Rising home prices are not the only factors hitting home affordability. Fees charged to lenders by Fannie Mae and Freddie Mac (known as "guarantee fees" for bundling and selling mortgages) began rising dramatically in the past month and are now at a high of 46 basis points, according to Capital Economics. These fees are passed on to borrowers in higher interest rates. This is one of the reasons why rates, still at historic lows, are not as low as the Federal Reserve had hoped when it announced another round of purchases of agency mortgage-backed securities.
(Read More: Housing Recovery Is Leaving Behind First-Time Buyers)
Congress raised guarantee fees in 2011 to pay for a payroll tax cut. There is yet another plan to raise fees further to fund immigration reform, although the bill is widely expected to fail.
"Dipping back into the housing piggybank to pay for unrelated policy items on the backs of America's homebuyers ends the wrong message at a time when the housing market is starting to show signs of recovery," wrote David Stevens, President and CEO of the Mortgage Bankers Association in a statement last month.
(Read More: Big Money Is Making Big Bets on a Housing Rebound )
Raising guarantee fees is another way for government to wind down the two mortgage giants it still backs, Fannie Mae and Freddie Mac, but that comes at a cost to borrowers who are already hampered by stricter underwriting standards.
"G-fees will continue to increase as a way to run down the GSEs' role in the mortgage market," writes Paul Diggle of Capital Economics. "Stronger mortgage demand suggests that would-be buyers are growing in confidence. Nevertheless, mortgage lending will continue to be held back by tight credit."
(Read More: Home Builders Beg for Skilled Workers)