"I'm not sure what this means except he is moving money around and he thinks that the refining side of the business is probably a better bet because there is going to be a lot of oil around for a long time," he told CNBC's "Squawk on the Street."
Matthews, who has authored several books about Buffett and Berkshire Hathaway, was referring to Berkshire's nearly $4.5 billion stake in Phillips 66. The disclosure, announced on Monday, reveals a stake that is about 11 percent of the company.
Shares of Phillips 66 got a boost on Monday off the news and were last up over 2 percent.
"He's adding to his position," Matthews said. "Does this mean people should run in and buy? I don't think so. I mean, he just sold a couple of other positions in the oil field. I don't think this is terribly meaningful. I think it's more adding to the portfolio."
Matthews was previously a Berkshire shareholder but sold out after the Precision Castparts deal.
Doug Terreson, head of energy research at Evercore, did not agree. Speaking on the same show on Monday he said that the move could be encouraging for other investors.
"We do think that you should read something into Warren Buffet's announcement today because valuations in the energy sector are near decade lows and sentiment is near decade lows," he said. "So for those investors with intermediate to longer-term time horizons, you're not going to get sentiment and valuation much better than you're getting it today, so therefore its not completely surprising that some of the deeper-value, longer-term investors like Warren Buffett are participating here."