"[Immelt] came in, in an aura of a coronation. This big thing, he's succeeding Jack Welch," Langone said on "Squawk Box," though he admitted he had voted for Immelt to succeed Welch. "On paper, [Immelt] looked fabulous. He really did and you can't take that away from him. Look at the success he had at GE" before he became CEO.
But the idea that "GE was transformed" after Welch was really a "synonym for destruction," Langone contended, adding he believes there's a good chance that GE will be "busted up."
Langone said GE may be a case in which an activist investor might be able to make a difference. Billionaire Trian Partners boss Nelson Peltz, a major GE shareholder, is pushing for changes. Ed Garden, a founding partner of Trian, may be the vehicle as a GE board member.
GE, with new CEO John Flannery, is in the midst of a major restructuring effort. Three new directors were nominated this week. Adding insult to injury, shares of GE briefly fell below $14 each on Monday after news late Friday that the Justice Department could take action in connection with alleged subprime mortgage violations. On Friday, GE also said it will restate earnings for 2016 and 2017 as it adopts a new accounting standard.
Immelt, who served in various other leadership roles at GE before CEO, led the company for 16 years until the latter half of 2017. During his tenure, the stock lost about 38 percent of its value. Immelt came under fire from critics over leadership decisions that left GE cash-strapped. On his way out the door, Immelt found himself defending the company's practice of having an empty business jet follow his corporate plane on several trips around the world.
Welch wanted Immelt to also have 20 years at the helm to put his stamp on the company, Langone said, but stressed that Welch, who stepped down in 2001, was a hard act to follow. "Jack ran the business. Jack to me was one of the finest managers I've ever since in my life. He knew everything about his business."
"In 1997, when Jack Welch felt he didn't have enough shelf space for light bulbs at Home Depot, he was screaming at me and Bernie on the golf course one day about what bad guys we were and how wrong this is," Langone said, referring to one of Home Depot's other co-founders, Bernie Marcus.
Langone was forced to leave GE's board during Immelt's time as CEO because of a dust-up with then-New York Attorney General Eliot Spitzer. Immelt was not immediately available to respond to CNBC's request for comment. For its part, GE said it has "no comment" on Langone's remarks.