Three of Berkshire's major holdings, Wells Fargo, Coca-Cola and American Express, have been in the portfolio for years. Wells and AmEx have been spectacular performers over the past twelve months as financials rebounded from the credit crisis. Coke has been roughly flat. (Over the longer run, of course, they've helped make him one of the world's richest billionaires.)
(Slideshow: Berkshire Hathaway's 15 Biggest Stock Holdings)
It's Buffett's most recent big stock pick that's trailing the pack.
IBM is up 7.2 percent since the end of 2011's third quarter, when Buffett finished accumulating the bulk of what is now a $12.6 billion stake. Not bad, but well short of the S&P's 49 percent surge over the same period.
Even by Buffett's favorite metric, book value, a measurement of the value of assets on a company's balance sheet, Berkshire is lagging the S&P.
In his annual letter to shareholders last March, Buffett was disappointed that Berkshire's per-share book value increase of 14.4 percent in 2012 was less than the S&P index's 16 percent increase. A "subpar" year, he said, even though it was a $24 billion gain.
"It's our job to increase intrinsic business value — for which we use book value as a significantly understated proxy — at a faster rate than the market gains of the S&P," he wrote, but predicted Berkshire would beat the S&P by only a "small margin" over time, in part due to its large size.
(Read more: Berkshire cuts Kraft, Mondelez stakes)
Berkshire is beating the S&P by one important measure that directly affects its investors: stock price. Berkshire's Class B stock is up 30 percent year-to-date, compared to the S&P's 18.2 percent gain. For the last twelve months, Berkshire is ahead 37.2 percent to 20 percent.
Berkshire's shares, of course, represent a lot more than just its own stock holdings, and business has been good for many of the subsidiaries, including Buffett's 2010 acquisition, Burlington Northern Santa Fe.
So, perhaps Buffett hasn't entirely lost his touch, but there's no denying his stock picks have been below par, at least lately.
Even he acknowledged that in his March shareholders letter.
After praising Berkshire's two new portfolio managers for beating the S&P by "double-digit margins" in 2012, Buffett used very small type to add, "They left me in the dust as well."