It's unclear how much Wells Fargo—the nation's largest mortgage provider—needs to rightsize to adjust to the new normal. The bank has already cut 3,000 jobs in origination and servicing, but activity has been bouncing around.
A brief drop in mortgage rates over Labor Day weekend caused refinancing activity to pick back up. News this week that the Federal Reserve would maintain its pace of bond buying sent yields lower, which in effect will cause mortgage rates to fall.
(Read more: Mortgages: To lock or not to lock?)
The average rate on a 30-year fixed mortgage for the week ended Sept. 13 was 4.5 percent, versus 3.53 percent in May, according to the Mortgage Bankers Association.
With the mortgage boom officially over, banks are closing facilities and units that have been servicing customers looking to refinance their home loans.
Other Wall Street banks are making similar moves, as a sharp rise in rates has kept consumers from taking out or refinancing mortgages.
JPMorgan laid off more than 2,000 employees in early August—about half of them in originations, according to a person familiar with the situation. The bank had said in February that it would cut 17,000 jobs (largely in the mortgage servicing unit, which handles bad loans) by the end of 2014.
The August round of layoffs represented the first time the bank had moved to downsize its origination business, which surged as mortgage rates went to historic lows.
In August, Bank of America notified 2,100 employees that their jobs were being cut; Wells Fargo has laid off more than 3,000 since July. Citigroup confirmed in early September the July closure of an office in Danville, Ill., that affected 120 jobs.
"The Danville facility was originally established to handle the surge in demand for refinancing," said Citi spokesman Mark Rodgers, who added that other telephone sales positions supporting mortgage banking would be cut.