The rise in rates put the brakes on the housing recovery, sending both mortgage applications and home sales lower during the summer months. While home builders continued to tout demand and affordability, they could not help but notice fewer buyers in their showrooms.
"We are experiencing the same as others who have reported, decent spring followed by a poor summer," said Stephen Paul of Mid-Atlantic Builders. "Through June, sales were up 16 percent then dropped off the table in July and August."
Home builder confidence stalled nationally in September, after rising steadily, especially at the beginning of 2013.
"While builder confidence is holding at the highest level in nearly eight years, many are reporting some hesitancy on the part of buyers due to the sharp increase in interest rates," said Rick Judson, the National Association of Home Builders' chairman.
If interest rates retreat to where they were at the beginning of the year, mortgage refinances will likely rebound again, especially since they have dropped so dramatically in the past six months.
As for home sales, that is not an easy call. Sales have been hampered not just by rising mortgage rates, but by very low inventory, anemic construction, and still-pervasive negative equity among potential move-up buyers. Lackluster job and wage growth, especially among younger Americans, has not helped either.
(Read more: Tepper: Fed wants growth first, second, and third)
We also know that while the Federal Reserve may not be tapering now, it will have to eventually. Some say it should do so sooner rather than later.
"Rip this Band-Aid off already," said Peter Boockvar of the Lindsey Group. "There will never be the right time to cut back, and today was the perfect opportunity to do so because the market was ready for it. Playing games now over this with the market will not smooth the eventual ease."