As investors move out, the market is left to regular, owner-occupant, mortgage-dependent buyers. Mortgage applications to buy an existing home are down 10 percent from a year ago, and a report Thursday from the Mortgage Bankers Association finds loan applications to purchase a newly built home fell 5 percent in June from May. While supplies of homes for sale may still be slim, sellers cannot claim the sky is the limit on price, especially with a tighter mortgage market standing in the way.
Accordingly, less than one quarter of the Redfin agents surveyed said it is a seller's market, down from 35 percent just three months ago. Sellers are in the weakest position in the Midwest, while buyers nationwide are walking away from bidding wars because homes are significantly less affordable and credit is still comparatively tight, according to Redfin.
Read MoreMortgage volume remains stuck despite lower rates
Members of the Federal Reserve's Board of Governors noted the disconnect in their most recent meeting, as noted in the minutes released Wednesday: "Despite attractive mortgage rates, housing demand was seen as being damped by such factors as restrictive credit conditions, particularly for households with low credit scores; high down payments; or low demand among younger home buyers, due in part to the burden of student loan debt. ... Several other participants suggested the possibility that more persistent structural changes in housing demand associated with an aging population and evolving lifestyle preferences were boosting demand for multifamily units at the expense of single-family homes."
Unfortunately, affordability concerns are not limited to the home-buying market. Rent is also rising and taking a larger chunk out of people's paychecks.