"How could the most valuable company in the world 18 years ago now be scurrying around for money?" said Langone, who was asked to leave the board in 2005 by then-CEO Jeff Immelt. "What happened to General Electric is a tragedy, and the whole story needs to be told."
Langone appeared on "Squawk Box" on the morning it was revealed that GE filed confidentially for an initial public offering of its health-care unit, GE Healthcare, as part of the larger company's turnaround efforts. The GE Healthcare spinoff will set the highly profitable unit on its own course, said Langone, who served on GE's board for six years beginning in 1999.
Restructuring aside, Langone argued GE itself has needed at least one person on its board to ask the "tough questions," criticizing the company for making what he called a series of poor deals over the years. He said one unnamed director described the company as "too much cheerleading" and "not enough supervision"
"You have no idea the lives we impact, betting on us to do the right thing," said Langone, also co-founder of Home Depot. "Do you know the number of people who have retired from GE that have seen most of the net worth wiped out?"
GE did not immediately respond to CNBC's request for comment on Langone's remarks.
When Immelt took over at GE in 2001 from then-CEO Jack Welch, the stock was already turning over, as the dot-com bubble of the 1990s burst and took the broader stock market lower as well. Immelt navigated GE through the aftermath of the Sept. 11, 2001, terrorist attacks and the 2008 financial crisis.
Immelt came under fire by critics for poor leadership decisions as CEO that left GE cash-strapped. On his way out the door, Immelt found himself defending the company's practice of having an empty business jet follow his corporate plane on several trips around the world.
John Flannery, who look over for Immelt, was then ousted from the top job after just 14 months. Flannery was replaced Oct. 1 with Larry Culp, who had been a GE board member since April. Culp previously was CEO of Danaher from 2000 to 2014.
GE shares were actually trading up about 4.5 percent early Wednesday, after longtime analyst Jeffrey Sprague of Vertical Research Partners issued his first buy rating on the stock in a decade. But the stock was bouncing off levels not seen in a decade. Last week, GE touched the financial crisis lows of $6.66 per share.