So where will investors find the best bets now? Detroit ranked No. 1 for rental returns, according to a new survey by RealtyTrac. The rankings were calculated by taking the 2014 fair market rent for a three-bedroom home multiplied by 12 (months) and then dividing that total by the median sales price of residential properties in the county.
Detroit's median home price of just under $45,000 will give investors a 30 percent annual gross yield, according to the report. The national median home price in February was $189,000, according to the National Association of Realtors.
Despite the big annual price jumps in Atlanta overall, one county in the metro area, Clayton, ranked second, likely because there are still a lot of homes in foreclosure in the market. The median home price there is just over $50,000, so investors would see a 28 percent annual return.
Read MoreReal estate's red-hot category: Apartments
"I think that parts of Atlanta have probably been picked over pretty well by some of the big investors, but certainly the fundamentals are still strong due to the relatively low home prices," noted Daren Blomquist, RealtyTrac vice president.
The underlying economy in Atlanta is also doing well and there is strong rental demand, some of it from those who lost their homes to foreclosure and may be seeking single-family rentals.
The worst markets for investors are just the opposite: High-cost areas with limited distress.
Not surprisingly, the New York City/northern New Jersey metro market ranks worst for investors, yielding just 3 percent annual gross returns. Rents may be high, but the cost to purchase a median-priced home hovers just under $900,000. San Francisco is also in the top three. Rental demand and prices there are very high, but home prices still don't make it worth an investor's while.
Read MoreHousing in recovery, but not its stocks
Large-scale institutional investors, who bought 354,000 properties just in 2013, according to RealtyTrac, have been reducing their purchases so far this year. Much of that is due to price appreciation and lack of distressed supply; perhaps less is due to investor demand. All-cash sales, which are largely by investors, still make up a historically high 35-40 percent of home sales today, according to several different surveys.
"That indicates to me that there is still very strong investor interest in the market, even though it may not be the big hedge funds," added Blomquist.