From the start, troubles with online health exchanges
WASHINGTON — In March, Henry Chao, the chief digital architect for the Obama administration's new online insurance marketplace, told industry executives that he was deeply worried about the Web site's debut. "Let's just make sure it's not a third-world experience," he told them.
Two weeks after the rollout, few would say his hopes were realized.
For the past 12 days, a system costing more than $400 million and billed as a one-stop click-and-go hub for citizens seeking health insurance has thwarted the efforts of millions to simply log in. The growing national outcry has deeply embarrassed the White House, which has refused to say how many people have enrolled through the federal exchange.
Even some supporters of the Affordable Care Act worry that the flaws in the system, if not quickly fixed, could threaten the fiscal health of the insurance initiative, which depends on throngs of customers to spread the risk and keep prices low.
(Read more: A fix to Obamacare tech problems? May take years )
"These are not glitches," said an insurance executive who has participated in many conference calls on the federal exchange. Like many people interviewed for this article, the executive spoke on the condition of anonymity, saying he did not wish to alienate the federal officials with whom he works. "The extent of the problems is pretty enormous. At the end of our calls, people say, 'It's awful, just awful.' "
Interviews with two dozen contractors, current and former government officials, insurance executives and consumer advocates, as well as an examination of confidential administration documents, point to a series of missteps — financial, technical and managerial — that led to the troubles.
Politics made things worse. To avoid giving ammunition to Republicans opposed to the project, the administration put off issuing several major rules until after last November's elections. The Republican-controlled House blocked funds. More than 30 states refused to set up their own exchanges, requiring the federal government to vastly expand its project in unexpected ways.
The stakes rose even higher when Congressional opponents forced a government shutdown in the latest fight over the health care law, which will require most Americans to have health insurance. Administration officials dug in their heels, repeatedly insisting that the project was on track despite evidence to the contrary.
Dr. Donald M. Berwick, the administrator of the federal Centers for Medicare and Medicaid Services in 2010 and 2011, said the time and budgetary pressures were a constant worry. "The staff was heroic and dedicated, but we did not have enough money, and we all knew that," he said in an interview on Friday.
Administration officials have said there is plenty of time to resolve the problems before the mid-December deadline to sign up for coverage that begins Jan. 1 and the March 31 deadline for coverage that starts later. A round-the-clock effort is under way, with the government leaning more heavily on the major contractors, including the United States subsidiary of the Montreal-based CGI Group and Booz Allen Hamilton.
(Read more: Rx for Obamacare? Feds mull rebuilding website
One person familiar with the system's development said that the project was now roughly 70 percent of the way toward operating properly, but that predictions varied on when the remaining 30 percent would be done. "I've heard as little as two weeks or as much as a couple of months," that person said. Others warned that the fixes themselves were creating new problems, and said that the full extent of the problems might not be known because so many consumers had been stymied at the first step in the application process.
Confidential progress reports from the Health and Human Services Department show that senior officials repeatedly expressed doubts that the computer systems for the federal exchange would be ready on time, blaming delayed regulations, a lack of resources and other factors.
Deadline after deadline was missed. The biggest contractor, CGI Federal, was awarded its $94 million contract in December 2011. But the government was so slow in issuing specifications that the firm did not start writing software code until this spring, according to people familiar with the process. As late as the last week of September, officials were still changing features of the Web site, HealthCare.gov, and debating whether consumers should be required to register and create password-protected accounts before they could shop for health plans.
One highly unusual decision, reached early in the project, proved critical: the Medicare and Medicaid agency assumed the role of project quarterback, responsible for making sure each separately designed database and piece of software worked with the others, instead of assigning that task to a lead contractor.
Some people intimately involved in the project seriously doubted that the agency had the in-house capability to handle such a mammoth technical task of software engineering while simultaneously supervising 55 contractors. An internal government progress report in September 2011 identified a lack of employees "to manage the multiple activities and contractors happening concurrently" as a "major risk" to the whole project.
(Read more: One Washington gets Obamacare right
While some branches of the military have large software engineering departments capable of acting as the so-called system integrator, often on medium-size weapons projects, the rest of the federal government typically does not, said Stan Soloway, the president and chief executive of the Professional Services Council, which represents 350 government contractors. CGI officials have publicly said that while their company created the system's overall software framework, the Medicare and Medicaid agency was responsible for integrating and testing all the combined components.
By early this year, people inside and outside the federal bureaucracy were raising red flags. "We foresee a train wreck," an insurance executive working on information technology said in a February interview. "We don't have the I.T. specifications. The level of angst in health plans is growing by leaps and bounds. The political people in the administration do not understand how far behind they are."
The Government Accountability Office, an investigative arm of Congress, warned in June that many challenges had to be overcome before the Oct. 1 rollout.
"So much testing of the new system was so far behind schedule, I was not confident it would work well," Richard S. Foster, who retired in January as chief actuary of the Medicare program, said in an interview last week.
But Mr. Chao's superiors at the Department of Health and Human Services told him, in effect, that failure was not an option, according to people who have spoken with him. Nor was rolling out the system in stages or on a smaller scale, as companies like Google typically do so that problems can more easily and quietly be fixed. Former government officials say the White House, which was calling the shots, feared that any backtracking would further embolden Republican critics who were trying to repeal the health care law.
Marilyn B. Tavenner, the administrator of the Centers for Medicare and Medicaid Services, and Kathleen Sebelius, the secretary of health and human services, both insisted in July that the project was not in trouble. Last month, Gary M. Cohen, the federal official in charge of health insurance exchanges, promised federal legislators that on Oct. 1, "consumers will be able to go online, they'll be able to get a determination of what tax subsidies they are eligible for, they'll be able to see the premium net of subsidy," and they will be able to sign up.
But just a trickle of the 14.6 million people who have visited the federal exchange so far have managed to enroll in insurance plans, according to executives of major insurance companies who receive enrollment files from the government. And some of those enrollments are marred by mistakes. Insurance executives said the government had sent some enrollment files to the wrong insurer, confusing companies that have similar names but are in different states. Other files were unusable because crucial information was missing, they said.
Many users of the federal exchange were stuck at square one. A New York Times researcher, for instance, managed to register at 6 a.m. on Oct. 1. But despite more than 40 attempts over the next 11 days, she was never able to log in. Her last attempts led her to a blank screen.
Neither Ms. Tavenner nor other agency officials would answer questions about the exchange or its performance last week.
Worried about their reputations, contractors are now publicly distancing themselves from the troubled parts of the federally run project. Eric Gundersen, the president of Development Seed, emphasized that his company had built the home page of HealthCare.gov but had nothing to do with what happened after a user hit the "Apply Now" button.
Senior executives at Oracle, a subcontractor based in California that provided identity management software used in the registration process that has frustrated so many users, defended the company's work. "Our software is running properly," said Deborah Hellinger, Oracle's vice president for corporate communications. The identical software has been widely used in complex systems, she said.
The serious technical problems threaten to obscure what some see as a nationwide demonstration of a desire for more affordable health insurance. The government has been heavily promoting the HeathCare.gov site as the best source of information on health insurance. An August government e-mail said: "35 days to open enrollment." A September e-mail followed: "5 days to open enrollment. Don't wait another minute."
The response was huge. Insurance companies report much higher traffic on their Web sites and many more callers to their phone lines than predicted.
That made the flawed opening all the more disappointing to supporters of the health plan, including Timothy S. Jost, a law professor and a consumer representative to the National Association of Insurance Commissioners.
"Even if a fix happens quickly, I remain very disappointed that the Department of Health and Human Services was not better prepared for the rollout," he said.
Robert Pear reported from Washington, Sharon LaFraniere from New York and Ian Austen from Ottawa. Quentin Hardy contributed reporting from San Francisco, and Kitty Bennett contributed research.