Investor demand, large and small, put a floor on home prices after the housing crash and ignited a recovery. While institutional investors made up a small percentage of those home buyers, they have garnered the most attention because of the bulk purchases they made and because they are a new entrant to the housing market. The concern now is what will happen if and when they decide to pull out?
Investor demand really started in Phoenix, Las Vegas and the hardest hit parts of California. Blackstone's Invitation Homes, American Homes 4 Rent, Colony Capital and American Residential Properties, to name a few, came in armed with billions of dollars in cash.
They bought thousands of distressed properties, and when the bargains began to dry up, they moved east to Atlanta, Charlotte, N.C., and Chicago. While there are no exact numbers available, institutional investors have purchased well over 100,000 homes, which is still a very small percentage of investor-owned homes in the United States.
Despite that share, economists and housing experts are concerned. In a survey of these experts, Zillow found that 79 percent believe that if investors pulled out of the housing market this year, then there would be a "significant" or "somewhat significant" impact on the markets where that demand had been highest.
While there has not been much selling, investor purchases already have slowed in markets like Phoenix and Las Vegas. That has pushed sales down and inventories up dramatically. Sales in Phoenix fell 17 percent in January from a year ago, while inventories were up 30 percent, according to California-based housing analyst Mark Hanson, citing Arizona Regional Multiple Listing Service data.
(Read more: Fewer affordable houses, even as credit eases)
"If I were a Phoenix real estate 'investor' sitting on a lot of house-price upside—or in the long process of readying dozens, hundreds, or thousands of houses for rent into a market about to get pounded for years with single-family rental and for-sale supply coming to market—I would push the 'sell button' on everything I could, immediately, on data such as these," Hanson said in a note to clients.
But the Zillow survey did not find the same sentiment, at least among economists and other housing analysts. Just 4 percent expected institutional investors to begin selling this year. Fifty-seven percent said investors would start selling within three to five years and 33 percent said investors would hold for six to 10 years.
"There is going to be no rush for the exits," said Stan Humphries, chief economist at Zillow. "For the investors, they bought homes in 2009-2010 at phenomenal fire sale prices. They're in markets where rents are increasing, so they're holding on to a cheap property that is massively cash-flow positive. I would see no reason why they would seek to unload those properties any time soon."
(Read more: US mortgage applications fall in latest week: MBA)
In fact, some of the largest players, like Blackstone and American Homes 4 Rent, are now securitizing the rental streams and selling those securities to investors for additional profit. Others say they have no intention to sell now, especially since they've finally gotten management systems in place and have reached economies of scale.
They have definitely slowed their buying, however, allowing smaller investors and more regular home buyers to get into the market. With home prices still rising at a faster-than-normal pace, higher mortgage rates and less investor demand, this spring's usually busy housing season will arguably be key to understanding where the housing recovery is headed. Until now, it has been fueled by a lot of cash and government-induced rates. As housing re-enters the real world, demand and credit will be the wild cards to keep sales and prices in the positive.