The company's shares fell $8.96 at Tuesday's open after a , costing the Oracle of Omaha some $712.9 million.
He is also IBM's largest shareholder, with almost 20 million more shares than the next-biggest investor.
To be sure, Buffett takes a very long view, and he added to his IBM holdings earlier this year. As recently as May, he said he expected to make "considerable" money on IBM over the long term.
But it's also not the first time IBM has cost Berkshire a lot of money on a bad quarter.
Last October, IBM's sharp drop after earnings wiped just over $1 billion off Berkshire's books.
On Monday, IBM posted second-quarter adjusted profit of $3.84 per share on $20.81 billion in sales. Those were both down 13 percent from the prior-year period.
IBM's crucial "strategic imperatives"—made up of cloud computing and analytics—saw revenue growth of about 20 percent. That wasn't enough to ease Wall Street's skepticism, though.
"I think the challenge for the company is can they grow these strategic imperatives faster than the rest of the company is declining," said David Nelson, chief strategist at Belpointe Asset Management, in an interview Monday with CNBC's "Closing Bell."