In Bangladesh, Li & Fung has been tied to several calamities. It arranged the production of clothing for Kohl's at one factory where 29 workers died in a fire in 2010. It brokered some work at another in 2011 where more than 50 workers who made Tommy Hilfiger clothing were injured and at least 2 died in an explosion and a stampede.
And last year, Li & Fung was responsible for some garments produced at the Tazreen Fashions factory, when 112 workers died in November in a fire after many of them were ordered to continue working even though alarms had sounded.
Such episodes highlight the often hidden role played by sourcing companies in trying to feed the West's seemingly insatiable demands for ever cheaper merchandise. Worker advocates say that Li & Fung and others make accountability more difficult by adding a layer of insulation between reputation-conscious retailers and often poorly treated workers, allowing businesses to avoid bad publicity and legal liability when things go wrong.
Sourcing companies face an inherent conflict: they are expected to find low-cost factories for clients, but also to blow the whistle if the factories violate safety standards. Some critics say that the scale of Li & Fung's operations and the speed at which it shifts production from one site to another give owners less incentive to improve their factories and make it difficult for Li & Fung to deliver on its pledges of carefully vetting its suppliers.
"We make our best effort to weed out bad factories," said Bruce Rockowitz, chief executive of Li & Fung. "But we don't always succeed."
(Read More: Death Toll in Bangladesh Building Collapse Climbs to 160)
Mr. Rockowitz added that Li & Fung employees conduct rigorous on-site audits—unlike many competitors—to ensure that the company does business only with factories that adhere to safety regulations. In the case of Tazreen, Li & Fung had acquired a new subsidiary that placed orders at the factory, but the changes sought by Li & Fung had not been made 11 weeks later when the fatal fire occurred, a company spokesman said.
By usually relying on long-term relationships with best-in-class suppliers, Mr. Rockowitz said, the company avoids dangerous work sites. Industry consultants agree that the long experience of Li & Fung—it was founded in 1906 during China's Qing dynasty—helps it ensure compliance with varying local labor standards and say that it tries to be conscientious in its oversight, especially when compared with newer and smaller rivals.
Industry insiders also emphasized its ability to provide savings for consumers. Fred Gehring, the chief executive of Tommy Hilfiger, a client, said Li & Fung had so much buying power that it extracted better deals from factories than his company could if it dealt with them directly.
Asked about his company's role in depressing wages, Mr. Rockowitz said that the business simply responds to consumer demand for bargains. "We definitely are a part of bringing the prices down, there's no question about that, because we are arbitraging factories and countries all the time," he said. "But it has to be a safe factory."
Retailers turn to Li & Fung because its resources make it uniquely equipped to find the Mexican port that can accept a shipment sooner, to persuade a Chinese fabric maker to cut an extra thousand square feet of silk faster, and to coax a factory in Bangladesh to fill an order more cheaply.
"Little John Waynes," Li & Fung calls its in-country hagglers, for their cowboy-like independence. Factory owners, though, said the nickname referred more to the company's take-it-or-leave-it style at the bargaining table.
"They usually get what they ask for," said Tipu Munshi, the Bangladeshi factory owner of the Sepal Group, which makes jeans and other clothing for Target, Walmart and Kmart.
Li & Fung's ability to exert pressure on factories can have unfortunate consequences, said Guadalupe Palma, director of Warehouse Workers United, a labor advocacy group. "Every extra penny you squeeze from a factory," she said, "is a step closer to that factory cutting the kind of corners that lead to deadly disasters."
While it has some of the lowest labor costs in the developing world, Bangladesh, with rampant corruption, congested roads and frequent unrest, is a tough place to do business.
Making clothes is also different from mass-producing other goods, like filing cabinets or batteries. Styles change fast. Fabrics react differently when cut in a high-humidity country but worn by customers in an arid one. Factories, which often consist of just sewing machines on tables, can appear and disappear overnight.
(Read More: EU considers trade action after Bangladesh factory collapse)
"Li & Fung is your insurance policy for when logistics go wrong," said Mr. Hertzman, of Sourcing Journal.
When a container ship arrives in the United States with a half-million Tweety Bird shirts for Warner Brothers that look like cheap knockoffs because they are the wrong shade of yellow, it is the sourcing company, not the factory, that will get the unhappy call, he said. When street protests delay a shipment of khakis bound for Gap and the factory says it cannot afford to move the items by air, it is companies like Li & Fung that will have to cover the cost of the plane, he said.
Citing privacy concerns, Li & Fung officials declined to discuss specific clients or details about services. But industry consultants and former employees or partners offered some insight.
Li & Fung and other sourcing companies track harvesting schedules because many factories rely on migrant workers. They watch weather reports to advise drivers which trucks should be outfitted with roof-mounted dinghies for use when roads wash out. In Dhaka, Bangladesh's capital, sourcing agents check in daily with political and labor officials who can offer warnings about demonstrations that could shut down production.
They also try to guard against abuses. They send undercover informants into factories to check for blocked fire exits, for example, or arrive early for scheduled factory inspections to check for child labor violations. When retailers are alerted to worker complaints in factories, they often turn to their sourcing companies to investigate. But sourcing companies can have little incentive to highlight problems at plants that they introduced into the supply chain.
In June 2011, for example, more than 280 workers vomited and fainted in a three-day period at the King Fashion Garment Company factory in Phnom Penh, Cambodia, which made dresses and knitted tops for brands including H&M, a former Li & Fung client. The sourcing company investigated and cited a "bad smell" at the factory and malnourishment as likely causes. A Li & Fung spokeswoman added that the business encouraged the plant to upgrade its ventilation and improve wastewater drainage.
Police and medical officials, though, said that exhaustion from overwork was probably to blame, while a United Nations labor agency pointed to the possibility that chemicals used to treat fabrics could be responsible. Li & Fung has continued to do business with the factory.
In 2007, more than a dozen garment workers at the PT. Mulia Knitting Factory in Jakarta, Indonesia, who were making clothes for Polo Ralph Lauren and Tommy Hilfiger were fired, allegedly for trying to form a union—the kind of dismissal that violates Indonesian law. Li & Fung investigated and did not find any violations of workers' rights, a spokeswoman said.
But labor advocates found that Li & Fung did not interview any of the dismissed workers and conducted all employee interviews in the factory, often with managers present. In explaining why it would not sever ties to the factory or push for reforms, Tommy Hilfiger cited the Li & Fung findings.
"Li & Fung claims to be monitoring factory conditions, but they don't publicly release their investigation reports or even the full list of the factories they use, so it's impossible for independent organizations to assess the effectiveness of their monitoring," said Tim Connor, a former labor rights advocacy coordinator for Oxfam.
Balking at an accord
To help prevent future catastrophes, worker advocates have recently promoted an international accord that sets mandatory safety standards for garment factories. Most American retailers have not signed the agreement, citing liability concerns, especially since they cannot guarantee that their subcontractors or sourcing companies will follow the rules.
Li & Fung and other sourcing companies have also balked, saying they cannot make that decision on behalf of their clients, the retailers.
In April, a dozen retailers gathered in Geneva to discuss how much to pay into a fund for families of the dead Tazreen workers. Though there was evidence that some of the clothing made at the factory was destined for Walmart, Sears and Disney, the three companies refused to pay into the fund. But Li & Fung, which had played a role in brokering some of the contracts for work at Tazreen, agreed to pay injured workers and the families of those killed roughly $1,200 each.
"Part of what Li & Fung sells" to retailers "is the right not to attend these meetings or pay into these compensation funds when people die," said Suri Gurumurthi, a business school professor at the University of North Carolina at Chapel Hill.
In the meantime, Mr. Rockowitz said his company was considering South America and sub-Saharan Africa as possible places for growth.
"But I wouldn't write Bangladesh off" because of the recent factory disasters, he said. It still has some of the cheapest labor in the world, he pointed out. For factories to get safer, clothing prices would have to go up, he said.
"So far," he added, "consumers have just not been willing to accept higher costs."
—By The New York Times' Ian Urbina and Keith Bradsher. Kitty Bennett contributed research.