"Finding a house is the last hurdle for many buyers who have saved a down payment and gotten pre-approved for a mortgage, but low inventory levels like those we're seeing across the country can bring the homebuying process to a screeching halt," said Stan Humphries, chief economist at Zillow. "In many markets, there just isn't a lot to choose from in terms of homes on the market."
The bigger the city, the bigger the problem. Inventory fell in 19 of the nation's largest metropolitan areas. Supplies are also falling the most in the lower price ranges, making it even more difficult for already cash-strapped first-time buyers to get into home ownership.
The reasons for tight supply are manifold: Homebuilders are putting up single family homes at a far slower pace than historical norms. They cite a shortage of labor for at least some of that weakness, but they also are not seeing strong demand, due to their higher prices.
Read More Detroit: Tale of two housing markets
Another reason is negative equity. Millions of homeowners still owe more on their mortgages than their homes are worth, or they don't have enough equity in their homes to afford to move up, leaving them unable to list their homes for sale. Still other potential move-up buyers are afraid they won't be able to find a home to buy, so they stay where they are.
Supply is also directly connected to price. Supplies are lowest in markets where prices continue to rise, but supply is actually starting to ease in markets where home prices have flattened. Take Washington, D.C., for example. Home prices are flat from a year ago, and inventory is now up nearly 19 percent, according to Zillow. In Dallas, however, where prices are still rising, up 12.5 percent from a year ago, inventory is down nearly 20 percent.
"Sellers tend to want to hang in and get the last juice out of the orange," said Humphries. "If you think you might see 10 percent appreciation over the next year, is it rational to move where you might squeeze more out of the market? No."
Read MoreHomeownership rate drops to lowest since 1967
That seller psychology is only feeding the problem in some of the hottest markets. Low inventory only pushes prices higher, and makes more sellers want to wait. That in turn pushes supply lower, as housing demand rises. In addition, higher rents are keeping more first-time buyers from saving for a down payment.
"If we can get additional increase in supply, then price increases will begin to flatten out, which will be good for the economy, good for many first time buyers, but as long as we have limited supply, and that's what we have today, then prices inevitably will continue to rise," said Lawrence Yun, chief economist for the National Association of Realtors.