They will form a new company for the venture, which will be headquartered in Boston, a hub for biotechnology and medical research. It will be "free from profit-making incentives and constraints."
Gawande will start July 9. He currently practices general and endocrine surgery at Brigham and Women's Hospital and is a professor at the Harvard T.H. Chan School of Public Health and Harvard Medical School.
He is founding executive director of Ariadne Labs, a health systems innovation center. He's also written numerous books, including "Being Mortal" and "The Checklist Manifesto."
Gawande will transition to chairman of Ariadne Labs from executive director, the organization said in a press release. He will not give up his positions at Harvard or Brigham and Women's Hospital and will keep writing, including for The New Yorker.
"I'm thrilled to be named CEO of this healthcare initiative," Gawande said in a statement. "I have devoted my public health career to building scalable solutions for better healthcare delivery that are saving lives, reducing suffering, and eliminating wasteful spending both in the US and across the world.
"Now I have the backing of these remarkable organizations to pursue this mission with even greater impact for more than a million people, and in doing so incubate better models of care for all. This work will take time but must be done," he added. "The system is broken, and better is possible."
The trio hasn't laid out a specific plan for the new company, which has yet to be named. The overarching vision is to lower health-care costs for employers.
J.P. Morgan's Dimon told CNBC earlier in June that this is a long-term initiative. He said there are a lot of ideas out there and things they know can be done better, pointing to fraud, administrative costs, overuse and underuse of various drugs, and specialized procedures.
Gawande has been vocal about his ideas to improve health care and lower costs. He has advocated more integration in health care, saying in a 2012 TED Talk that the ones getting the best results at the lowest costs have found ways to get all the different pieces to come together into a whole.
"There's a famous thought experiment that touches exactly on this that said, what if you built a car from the very best car parts? Well it would lead you to put in Porsche brakes, a Ferrari engine, a Volvo body, a BMW chassis. And you put it all together and what do you get? A very expensive pile of junk that does not go anywhere. And that is what medicine can feel like sometimes. It's not a system," he said.
Despite momentum around the venture, some health-care experts have been skeptical about whether Buffett, Dimon and Bezos, while business icons, can simplify the current system.
Andy Slavitt, who was chief of the Centers for Medicare and Medicaid Services during the Obama administration, is among the consultants the trio spoke with. He praised the decision to have Gawande lead the effort.
"There are few better people in health care than [Gawande]," he said. "They are fortunate to have him, principally because of his moral leadership."
Buffett said in a statement that the trio was looking for "talent and dedication" when interviewing candidates.
"We said at the outset that the degree of difficulty is high and success is going to require an expert's knowledge, a beginner's mind, and a long-term orientation," Bezos said in a statement. "[Gawande] embodies all three, and we're starting strong as we move forward in this challenging and worthwhile endeavor."
— CNBC's Bertha Coombs contributed to this report.