"Investors are concerned with a potential rise in interest rates," said Lawrence Yun, chief economist for NAR. "It makes it less attractive in a rising interest rate environment."
The drop has been long expected. Home prices jumped dramatically last year and are still higher by nearly 5 percent from a year ago, while the supply of cheap, distressed properties fell. When calculating for potential returns, the math simply doesn't work as well anymore for investors.
Read MoreHard hit Tampa housing making some strides
"The reduction in appetite from investors has put a temporary lid on home sales that has yet to be offset by the first-time home buyer, which is more interested in renting than buying," wrote Peter Boockvar, chief market analyst with the Lindsey Group. "Faster income growth, slower home price gains and more of an easing in credit standards is what is needed to bring them back."
First-time home buyers made up just 29 percent of August buyers and do not appear to be picking up any of the slack of investors. The first-time buyer share is historically around 35 to 40 percent of the market, and it usually increases in the fall, when large families move out of the market. First-time home buyers tend not to have children yet.
Realtors are hoping the decreased competition from investors will bring first-time buyers back to the market, but this younger cohort is still facing high levels of student debt, weak employment and income growth and soaring rents, which keep them from saving for a down payment on a home. Some of those who are more fiscally sound are still choosing to rent, especially as the single-family rental market has become more popular.
This is why investors, while not buying many more homes, are not rushing to sell the ones they already have, either. Rents are up over 3 percent from a year ago nationwide.
Read MoreAre we building too many homes?
"We are really happy with our portfolio of homes," said Aaron Edelheit, CEO of Atlanta-based The American Home, in a July 2014 interview with CNBC. "Our demand for rental properties is strong. We have 95 percent rented." American Home invested in distressed properties.
Edelheit, who operates mainly in the Atlanta and Charlotte, N.C., markets, said he is not buying any more homes. The company currently owns about 2,400 single-family rental homes, which it manages through an in-house network of property managers, rental agents and technicians.
"This is maturing into every other type of industry," noted Edelheit. "You will have consolidation. Business is going to be about execution and operations. It will be all about operations. We have an attractive company with attractive assets. Like anything else, it's about operating and running this business."
Read MoreHome builder confidence hits highest level since 2005
Others, however, are still buying, albeit shifting their strategies. Justin Chang, CEO of Colony American Homes, which was a huge player early on in the Phoenix and Southern California housing markets, said his company continues to buy, although at a reduced rate.
"The contours of the buying and the geographies are shifting, as one might expect," said Chang.