"I feel like we got so close to the dream of keeping our house and suddenly it's gone," Ms. Herman said.
Some of the problems, analysts and regulators say, come down to the speed. The specialty servicers have not upgraded their technology or infrastructure to accommodate the glut of new mortgages.
Even more troubling, some regulators say, the servicers benefit when they work through the troubled loans as quickly as possible. That has raised questions about whether the companies are pushing homeowners into foreclosure or offering mortgage modifications that will keep homeowners treading water, but ultimately cause them to fall even further behind.
The servicing companies say they have bolstered customer service, including employing more Spanish-speaking representatives and offering flexible call hours.
"If these companies can do a better job rehabilitating the borrower, that is a good development," said Wilbur Ross Jr., a board member of Ocwen, which says it offers more subprime mortgage modifications than many peers.
(Read more: If investors bail on housing, what then?)
But some borrowers say that dealing with the specialty servicers is even more vexing than working with the banks, especially when long-promised loan modifications don't materialize.
The Hermans of Columbia Falls, Mont., said that despite almost daily calls to Nationstar, they still could not get an explanation of how their permanent loan modification from Bank of America, which reduced the balance on their mortgage by nearly $80,000, could disappear.
"I don't even know how to get a human on the line," Mr. Herman said.
Nationstar said that the couple never had a permanent loan modification and added that it had since offered the Hermans a new modification.
But behind Mr. Herman's exasperation is what separates the specialty servicers from the largest banks, according to regulators. The specialty servicers, the regulators say, do not offer the same attention to customer service that banks did.
Flaws in computer systems can further compound delays. At Ocwen, there is a dizzying number of computer codes, approximately 8,400 different varieties, to categorize issues within borrowers' files like a job loss, according to a person briefed on the matter. Many of these codes, the person said, are duplicates.
Mr. Lawsky's office, which installed an independent monitor at the company, is examining whether computer issues are wrongfully pushing homeowners into foreclosure. Ocwen says that they are not aware of any improper foreclosures.
The servicers also have relationships with companies that can benefit from foreclosures.
William Erbey, Ocwen's chairman is also the chairman of Altisource Residential, which buys up delinquent mortgages and owns foreclosed homes turned into rentals. Altisource's loans are serviced by Ocwen. According to securities filings, Mr. Erbey recuses himself from issues that relate to both companies and Ocwen adds it has a "strictly arms-length business relationship" with Altisource.
Specialty services may also be profiting at the expense of the investors who own the mortgages. Typically servicers get a fixed fee from investors for handling the mortgage payments, no matter if the borrower is up to date or has fallen behind.
But the dynamic of that business has changed, in part, because the specialty servicers are buying the rights to collect payments at discounts, along with the loan advances — the money that the servicers pay to investors to cover any delinquent payment. The sooner the servicer can make the loan current again, the sooner investors pay back the servicers' advance in full. That kind of arbitrage could incentivize servicers to offer modifications that cause borrowers to default again, investors say.
Borrowers like Ms. Darden of Maryland, meanwhile, must contend with the changes in the market. "I just don't know how much more of this I can take," she said.
—By Jessica Silver-Greenberg and Michael Corkery of The New York Times