Jeff Immelt, the CEO at General Electric, has been unable to get his company's share price anywhere near the heights achieved by his predecessor Jack Welch. Steve Ballmer had similar problems following Bill Gates at the helm of Microsoft.
And after a strong start in following the iconic Steve Jobs, Apple CEO Tim Cook has had some major struggles in the past 18 months and the company's share price is well off its highs.
One problem for any successor is that Buffett, by dint of his long stewardship of Berkshire and the outperformance he has delivered over most of the past half century, has the freedom to make mistakes or underperform the stock market in ways that could cost other CEOs their jobs.
"I feel sorry for the person who follows in his shoes," said Dave Sather, president at Sather Financial Group, which invests more than 5 percent of its $365 million of assets under management in Berkshire.
(Read more: Warren Buffett's 5 key investing don'ts)
"Buffett's in a very unique position... to where he can just think 100 percent independently. I worry that the next person will not have quite the same flexibility to be a dealmaker."
Certainly, Buffett's profitable deal-making has been a boon to Berkshire for decades, and has at times been transformative for the company.
In his annual letter to shareholders this weekend, Buffett noted that Berkshire's performance trailed the Standard & Poor's 500 in 2013—and in fact, even on a five-year basis it lagged that benchmark for the first time under Buffett. Berkshire's net worth per share rose about 91 percent in the five years, while the index gained about 128 percent.
Still, since 1965, Berkshire's book value per share has risen to $134,973 from $19. The nearly 20 percent annualized gain is twice the rate of growth in the S&P 500.
Buffett has said that Berkshire is likelier to outperform the S&P when markets are weak or moderate, in contrast to the surge in equities last year. While it lagged over the last five years, it has outperformed over the last six, including during the 2008 financial crisis.
Berkshire plans eventually to split Buffett's three roles as CEO, chief investment officer and chairman. The only one of those positions where succession has been clearly flagged is the last: Berkshire plans to make his son Howard non-executive chairman.
Berkshire's board had previously identified three internal candidates for the CEO job, though it has never named any of them, with one of those selected to take over immediately in an emergency.
But in a regulatory filing on Monday, Berkshire eliminated its previous statement about the number of candidates and said only that "certain current" managers of subsidiaries were CEO candidates.
Speculation over succession has mounted in recent years, including after Buffett underwent radiation therapy treatments for prostate cancer in 2012.
Buffett on Saturday praised some of the people who investors believe could be among his potential successors—including Ajit Jain (the head of his insurance operations), Greg Abel (the chief of his power utility MidAmerican), and Matthew Rose (who leads his railroad company BNSF).
(Read more: I'd vote 'yes' on Keystone pipeline: Warren Buffett)
He also pointed out that portfolio managers Todd Combs and Ted Weschler—who could be candidates for the chief investment officer job—had outperformed not only Berkshire's returns but those of the S&P 500 last year, though their portfolios of more than $7 billion each are small compared with the massive one that Buffett himself oversees.
The strong bench may not mean a lot in the days immediately after Buffett disappears from the scene, investors say.