A striking surge in home prices this fall was not enough to convince one of the nation's top housing economists that the recovery is on solid ground.
"We can't trust momentum in the housing market anymore," Nobel Prize-winning economist Robert Shiller said on CNBC's "Squawk Box."
Why not? Investors, specifically institutional investors, have vast sums of cash. They have bought about 100,000 homes, most of them previously foreclosed properties. They bought the homes in a limited number of markets, mostly in the West, pushing prices dramatically higher as competition for the properties increased. They are now renting them, and even selling bonds backed by the rental streams.
(Read more: Chinese buying up California housing)
The trouble, according to Shiller, is that investors are a fickle bunch, and if they see lower-than-expected returns they won't hesitate to dump the properties and move on to another trade.
"They've learned that there is short-run momentum in housing," said Shiller.
In fact, they may already be moving on. Institutional investor purchases represented just 6.8 percent of all sales in October, according to a new report from RealtyTrac. That is a dramatic drop from 12.1 percent in September and down from 9.7 percent a year ago. While one could point to the housing recovery and the related drop in the number of distressed properties, sales of bank-owned homes actually rose in October, both month over month and year over year, the firm said.
"There is notable weakness in the new-era, Fed-inspired investor, flipper/renter regions—that is, California, Arizona, Nevada—with a surge in supply," said housing analyst Mark Hanson.
(Read more: Pending home sales fall again)
Anika Khan, senior economist at Wells Fargo Securities, agreed that investor activity probably has peaked as home prices continue to rise.
"In a lot of those hard-hit markets, we continue to see the greatest price increases. ... A lot of this increase is exaggerated," Khan told "Street Signs." "However, the underlying fundamentals are still very positive, especially in those markets that have strong household formations, strong population growth and strong job prospects. ... We are seeing a recovery in the housing market."
Prices far not only higher for investors but for regular homebuyers—even more so now than they during the housing boom. Unlike the days of cheap and easy credit, they must use traditional and pricier 30-year-fixed mortgages with large down payments.
All-cash investors pushed the prices up, now up more than 13 percent from a year ago, according to the latest S&P/Case-Shiller report. That surge is far higher than income and employment growth, and has priced out regular buyers.
Home ownership has declined dramatically, but even more stunning is that there are about 15 percent fewer homeowners under the age of 35 and between ages 34 to 44 than there were in 2005, according to Deutsche Bank researchers.
Renting is not only more necessary but more popular among younger demographics. Witness the 24 percent jump in multifamily building permits in October, according to the U.S. Census, reaching the highest level of permits since mid-2008.
(Read more: Do homeowners need underwater insurance?)
"The rise in multifamily permit activity is a justified response to the strength of renter," said Paul Diggle of Capital Economics.
That is why most large investors are clearly not dumping all their properties to take advantage of price gains: They are seeing too much rental demand. Still, they are slowing their purchasing and putting some homes on the market.
"The first-time homebuyer is still renting, and we still see a surge in apartment demand," Khan said. "We'll continue to see that. It's definitely not a sustainable market without the first-time homebuyer."
If investors are moving out of the market and even getting ready to sell some of their properties for much more than they bought them, housing could take take another turn for the worse. There just is not enough regular consumer demand to make up the difference.
(Read more: Map: Tracking the recovery)
"I don't see evidence that people think we are launching out on some great new era," said Shiller of today's homebuyer. "I don't find that they're as excited about the housing market as price increases would suggest."
Ken Rosen, chairman of Rosen Consulting Group, echoed Shiller, saying that the "extreme exuberance" is over. He told "Street Signs" that we will now see a "more normal market."
But as Khan pointed out, Rosen said, the question is whether first-time homebuyers will get the liquidity they need.
—By CNBC's Diana Olick. Follow her on Twitter @Diana_Olick.
Questions?Comments? facebook.com/DianaOlickCNBC
—CNBC's Drew Sandholm contributed to this report. Follow him on Twitter @DrewSandholm