Realty Check

The new mortgage landscape: What you need to know

Troels Graugaard | E+ | Getty Images

Home mortgage debt rose in the third quarter of this year for the first time since the end of 2008, according to a new report from the Federal Reserve.

It is a healthy sign for the housing market, arguably, but it is not exactly what it seems at face value. Mortgage debt is growing largely because banks are foreclosing on fewer homes, thereby wiping away less debt.

During the last few years, record-low mortgage rates prompted millions of Americans to refinance their home loans, but a sluggish market, dominated by all-cash investors, kept loans to purchase a home in the negative. That too, however, is beginning to change.

Housing affordability shrinking
Housing affordability shrinking

More than one-third of the housing market is still running on cash, but home sales rose in the first half of this year, pushing mortgage volume higher. The Mortgage Bankers Association is estimating that, thanks to that surge, the full-year 2013 purchase mortgage volume will be up 11 percent year over year.

Spring could also bring a new surge in sales, as potential buyers gain confidence from rising prices, but buyer demand will be hit head-on by a new mortgage landscape. The Federal Housing Administration (FHA), the government's insurer of home loans, just announced it would be lowering its loan limit in the very highest cost areas from $729,750 to $625,000, starting Jan. 1, 2014.

"As the housing market continues its recovery, it is important for FHA to evaluate the role we need to play," said FHA Commissioner Carol Galante, in a release. "Implementing lower loan limits is an important and appropriate step as private capital returns to portions of the market and enables FHA to concentrate on those borrowers that are still underserved."

(Read more: Skyrocketing rents hit 'crisis' levels)

The FHA has already raised premiums and fees, and the average credit score of its borrowers is far higher than it was during the housing boom.

Then late Monday, the Federal Housing Finance Agency (FHFA), overseer of Fannie Mae and Freddie Mac, announced it would again raise the fees it charges lenders, beginning in March. Those fees will likely be passed on to borrowers in higher rates.

"The new pricing continues the gradual progression towards more market-based prices, closer to the pricing one might expect to see if mortgage credit risk was borne solely by private capital," noted the FHFA's acting director, Ed DeMarco, in a release. "These changes should encourage further return of private capital to the mortgage market."

(Read more: Rising mortgage rates a boon to smaller lenders)

The push to bring private capital back to the mortgage market by both the FHA and FHFA would seem to indicate that government agencies and federal regulators believe the economy, and the housing market, can handle the changes and that investors are waiting at the ready. That is not necessarily a given.

"We appreciate the broader goal of attracting more private capital to the housing finance system, but we question the timing of the price hike, as it comes right as multiple other housing regulations take effect," wrote Guggenheim Partners' Jaret Seiberg of the hike in Fannie and Freddie fees.

A major new regulation, called the Qualified Mortgage (QM), also goes into effect in the new year, and that could raise borrowing costs even more, especially for borrowers who don't qualify for the new standards. Qualified Mortgages are designed to protect both borrowers and lenders by limiting risk and providing banks with legal protections. The rules, however, are strict, and while lending will take place outside of QM, it will do so at a far higher cost.

"For anybody to venture into that space, the investor requirements are going to have to have enough yield to offset the risk of losing one case," notes David Stevens, president and CEO of the Mortgage Bankers Association. "They are going to expect very high margins."

(Read more: Soaring new home sales: Not what they seem)

Some lenders are already quoting rates as high as 10 percent for non-QM loans, even 30-year fixed loans. While rates may not end up as high as that when the rules take effect, they will undoubtedly be higher for a large number of borrowers with lower credit scores and higher debt.

By CNBC's Diana Olick. Follow her on Twitter .

Questions? Comments?