(Read more: Osborne defends Carney on forward guidance)
The debate, at a CNBC panel at the World Economic Forum in Davos, was heated for an event where the emphasis is generally on consensus.
Summers said that it was a "fair comment" that he was wrong on U.K. austerity, after a better-than-expected return to growth in the U.K., but pointed out that the country is still not back to its pre-crisis growth peak. Osborne countered that the U.K. had a deeper recession than the U.S., and a much larger banking sector as a proportion of the economy, making it more difficult to recover.
Summers has repeatedly criticized Osborne's "Plan A" of public spending cuts, and the government's support of the mortgage market, saying last year: "I have had some difficulty following the logic of British policy."
"Both expose tax payers to risk of loss."
The two agreed on the importance of Europe to the U.K. recovery, however.
"The Chancellor highlights the similarities and the risks associated with the other countries of Europe and I would see Europe's difficulties in those increases in bond yields as centrally related to the fixed exchange rate, the absence of a central bank, and the uncertainties that would be created if for example a U.S. state were to pursue a massive fiscal expansion," Summers said.