One major area of weakness in the mortgage market today is loans backed by the Federal Government, with application volume down nearly 37 percent from a year ago, according to the MBA. The Federal Housing Administration (FHA), which insures but does not originate home loans, is largely to blame for that. It has raised insurance premiums and gone after the big banks aggressively for making bad loans.
"The real question for me is should we be in the FHA business at all," said Jamie Dimon, CEO of JPMorgan Chase, on a conference call Tuesday related to the bank's second-quarter earnings. "We want to help there, but we can't do it at great risk to JPMorgan."
Read MoreJPMorgan beats Wall Street, sees trading drop
JPMorgan has agreed to pay over $600 million to the FHA and the Department of Veterans Affairs for defective loans. It originated far fewer FHA loans in the second quarter of this year.
"Until they come up with a safe harbor or something, we are going to be very, very cautious in that line of business," said Dimon, adding that he was "confused" by how the FHA treated his bank.