The uplift began with the bid win and will continue as Tokyo gets in shape for the big event and then finally sees people from all over the globe arrive at its doorstep, Vambery said.
He puts the cultural and entertainment value of the 2020 games at "not less than $15 billion and as much as $30 billion," not including revenue from travel, hotels, shopping, etc.
For what it's worth—and its worth is debatable—the Tokyo government predicts a $30 billion economic boost from its Olympics. Investors turned bullish on the news of the winning bid, pushing the Nikkei index up 4 percent this week.
Standard & Poor's lent its own lukewarm blessing in a report, saying: "A well-run 2020 Games would not adversely affect our rating on the Tokyo Metropolitan Government."
Lure of the rings
It is true that past Olympics have not not delivered the hoped-for returns for some host cities.
It took Montreal three decades to pay down the $2 billion loss it incurred for the 1976 games. The British Olympic Association says London broke even on the 2012 games, though some are skeptical. And in Athens, which had 800 percent budget overruns, the $15 billion sunk into the 2004 Olympics seems rather profligate in light of events since.
(Slideshow: Olympic Cities: Booms and Busts)
The China Daily figures Beijing came out $140 million ahead after spending $43 billion. But even if the 2008 Olympics did not pay for itself, Vambery said, consider the value of the thrills and national pride of 1.3 billion people.
The 1964 Olympiad was likewise a coming-of-age moment for Japan, a country that had rapidly risen from the defeat and destruction of World War II. It also came at a sweet spot of development for Tokyo, which really needed the infrastructure built for the games.
This time, said Toyoda of Villanova, fewer venues will need to be built, and the costliest item is a new national stadium. But even that will not be so disruptive, as it will go on the site of the stadium built for the '64 games.
Japan's last round with the Olympic rings was the 1998 Winter Games, and one might look to Nagano to see how that city fared with its investment.
Unfortunately, that is not possible. A Japanese official ordered the records burned.
—By CNBC's Matt Twomey. Follow him on Twitter @Matt_Twomey.