A sharp rise in mortgage rates over the last few weeks means it may already be too late for many homeowners to benefit from a refinance.
This just as thousands were gaining equity in their homes and finally becoming eligible.
At the same time, it is pushing some renters off the fence, fearing they too will miss the boat on the best conditions for home buying.
Refinances dropped 12 percent last week, while mortgage applications to purchase a home rose 3 percent and are now up 14 percent from a year ago, according to the Mortgage Bankers Association.
"Rates rose in response to stronger economic data and an increasing chance that the Fed may soon begin to taper their asset purchases," said the MBA's Mike Fratantoni in a release.
The Federal Reserve has poured billions of dollars into the mortgage market since the housing crash began, pushing mortgage rates to record lows, but it can't last forever. Recent remarks by Fed Chairman Ben Bernanke suggest the monthly mortgage market infusions may end sooner than later.
(Read More: Map: Tracking the US Real Estate Recovery)
That has pushed the rate on the 30 year fixed conventional mortgage to 3.90 percent, the highest level in a year and dangerously close to the emotional 4 percent barrier. This as home prices are jumping higher and faster than expected.
"It's amazing to see the frenzied pick-up in home buying, as renters get nervous that both home prices and rates will rise quickly," said Craig Strent, CEO of Maryland-based Apex Home Loans. "They are trying to catch the beginning of the curve here."
In Cincinnati however, Dan Green, a loan officer with Waterstone Mortgage, said his refinance clients are being hit the hardest, especially those who need lower-cost FHA loans. His home-buying clients seem more indifferent to the situation.
(Read More: Mortgage Rates Surge as Fed Tapering Fears Mount)
"Among the Main Street set, there is little awareness of this month's change in mortgage rates, let alone the changes of this week," added Green. "There's been very little panic among rate-shopping households. There's an acceptance, almost, a 'low rates couldn't last forever'-like attitude."
Still, rising rates could not come at a worse time for the housing recovery. Home prices rose over ten percent in March, according to the latest surveys from S&P/Case-Shiller. Every one percentage point rise in mortgage rates reduces the average home buyer's maximum purchase price by 11 percent, figures Green.
(Read More: Home Price Gains Go to Double Digits)
First-time home buyers will be hit hardest by rising rates, just as they were beginning to trickle back into the market. They made up just 29 percent of buyers in April, according to the National Association of Realtors, the lowest level in two years. Historically, they usually account for about 40 percent of the market.
The 30-year fixed mortgage hit a record low rate of 3.47 percent in December of last year. Even though it is still well below historical norms, this small rise is already taking its toll.
"In my world it's clearly slowing the market and pricing. Right now I have properties that are well-priced yet sitting on the market unsold," said David Fogg, a real estate agent in Burbank, CA. "Should rates continue to rise, values will likely soften."
(Read More: Borrowers Becoming Accidental Landlords)
Questions? Comments? RealtyCheck@cnbc.com