A separate amended filing for the second quarter, shows that most of the shares, 31.2 million of them, were added during the three months ending June 30.
The portfolio snapshot filing for the second quarter included a line disclosing that "confidential information" had been omitted, often a signal that Buffett is accumulating a position and is trying to avoid having the price be inflated by copycat buyers.
Based on the most recently available holdings data, the new stake would be enough to make Berkshire the 6th-biggest holder of Exxon, with 0.92 percent of the shares outstanding from the world's biggest publicly traded oil company.
At Thursday's closing price of $93.22, the stake was worth $3.74 billion. That's roughly 4 percent of the $99.5 billion market value of Berkshire's publicly-disclosed stock portfolio and was the company's seventh biggest holding on the list.
In after-hours trading, the stock gained almost one percent to $94.07 as of 7:45 pm ET.
(Read more: Berkshire Hathaway's 15 biggest stock holdings)
Berkshire hasn't held any Exxon Mobil since the end of 2011, when it sold a relatively small holding of 421,800 shares.
The large size of this new stake indicates Buffett himself made the decision to buy, not one of Berkshire's two portfolio managers who typically handle smaller amounts of stock.
Answering a viewer's question on "Mad Money," CNBC's Jim Cramer said, "There's going to be a continuing run in Exxon now that Warren Buffett took a 40 million share stake. For three straight days, you're going to hearing nothing except how great Exxon is because of the big buyback and Warren Buffett likes it. So, why don't you wait three days, then sell, sell, sell."
On "Fast Money," trader Steve Grasso said he didn't really like the trade. "I think it's very Warren Buffett-esque. I think it's a boring, slow-moving, dividend-paying stock... You have to believe in crude rallying from here, even though it is a very diversified play, but I think there are other places to put your money."
Guy Adami, however, said he thinks the stock has a "real chance" to trade up to $96.50 to $97 after bouncing from its low earlier in the year.
Oppenheimer oil analyst Fadel Gheit told Reuters that Buffett "likes buying big, established global brand names, and Exxon is a good flight-to-quality stock... The stock has also lagged the market in the last three and five years. That makes it a typical Warren Buffett holding."
Even so, Buffett's dive into big oil is something of a surprise. In May 2012, Buffett told CNBC he had made a mistake buying shares of ConocoPhillips for Berkshire, saying he has no special insight into the future movements of oil prices.
He does, however, like to buy stocks when the price goes down and that certainly happened during the second and third quarters.
In the first half of April, Exxon shares fell almost five percent to a second quarter low close of $86.08.
Then, after soaring to their 52-week high of $95.49 on July 23, they fell about 10 percent to end September at $86.04.
Since then, Exxon has rallied back into the $90s, to close Thursday at $93.22, a gain of more than 8 percent.
While we don't know exactly when Berkshire bought its shares, it's likely it picked them up at the lower end of the quarters' range and has already made a paper profit as the stock rebounded in the first weeks of the fourth quarter.
An Exxon spokesman is quoted by Reuters as saying the oil giant "appreciates the confidence" investors have in the company.
In the new filing, Berkshire also revealed it has sold more of that Conoco stake that had Buffett apologizing to shareholders in early 2009, reducing its holdings by 44 percent to 13.53 million shares. They're worth just under $1 billion at Thursday's close.
Berkshire held almost 84 million shares as of the end of September, 2008, at the height of the credit crisis.
In another large percentage move, Berkshire slashed its GlaxoSmithKline holdings by 77 percent to 345,819 shares during this year's third quarter.
That cut, however, represents only around $59 million worth of stock, relatively small by Berkshire standards.
- By CNBC.com's Alex Crippen with contributions from Reuters