He arrived at JPMorgan with Mr. Dimon in 2004 from Bank One, serving as the chief financial officer through the financial crisis and then as the head of treasury and securities services. He was promoted to co-head of the investment bank, along with Daniel Pinto, nearly two years ago. Mr. Pinto will now become sole chief executive of the investment bank.
Mr. Cavanagh is now joining Carlyle, serving as co-president alongside a veteran of the investment firm, Glenn A. Youngkin.
(Read more: Morgan Stanley investors await Fed's buyback blessing)
"This is a rare opportunity to join a premier global investment firm during a time of swift change for the industry," Mr. Cavanagh said in a statement. "We accomplished an immense amount at J.P. Morgan and I am grateful to my colleagues, especially Jamie Dimon, for their friendship, support and confidence. Carlyle is an established innovator and I look forward to partnering with Glenn and the rest of the Carlyle team to help take the firm to the next level of success."
Mr. Dimon said in a statement: "I have worked with Mike Cavanagh for more than 20 years. He's a highly talented executive and has been an integral part of our management team, as our C.F.O. for six years and as co-C.E.O. of the corporate and investment bank. He's also a special person and we wish him well in his choice to take on a new challenge. While we would prefer he stay at the firm, we are glad he's going to a valued client in Carlyle. I know the whole operating committee joins me in thanking him for his years of service to our firm."
—By Michael J. de la Merced of The New York Times