Home ownership has declined dramatically, but even more stunning is that there are about 15 percent fewer homeowners under the age of 35 and between ages 34 to 44 than there were in 2005, according to Deutsche Bank researchers.
Renting is not only more necessary but more popular among younger demographics. Witness the 24 percent jump in multifamily building permits in October, according to the U.S. Census, reaching the highest level of permits since mid-2008.
(Read more: Do homeowners need underwater insurance?)
"The rise in multifamily permit activity is a justified response to the strength of renter," said Paul Diggle of Capital Economics.
That is why most large investors are clearly not dumping all their properties to take advantage of price gains: They are seeing too much rental demand. Still, they are slowing their purchasing and putting some homes on the market.
"The first-time homebuyer is still renting, and we still see a surge in apartment demand," Khan said. "We'll continue to see that. It's definitely not a sustainable market without the first-time homebuyer."
If investors are moving out of the market and even getting ready to sell some of their properties for much more than they bought them, housing could take take another turn for the worse. There just is not enough regular consumer demand to make up the difference.
(Read more: Map: Tracking the recovery)
"I don't see evidence that people think we are launching out on some great new era," said Shiller of today's homebuyer. "I don't find that they're as excited about the housing market as price increases would suggest."
Ken Rosen, chairman of Rosen Consulting Group, echoed Shiller, saying that the "extreme exuberance" is over. He told "Street Signs" that we will now see a "more normal market."
But as Khan pointed out, Rosen said, the question is whether first-time homebuyers will get the liquidity they need.
—By CNBC's Diana Olick. Follow her on Twitter
—CNBC's Drew Sandholm contributed to this report. Follow him on Twitter