Warren Buffett said managing billions of dollars is easy compared to the job boxing champ Floyd Mayweather has when he enters the ring.

As he was attending Saturday night's WBC/WBA welterweight championship boxing match between Mayweather and Marcos Maidana in Las Vegas, Buffett was interviewed by Showtime's Jim Gray.

Sitting next to Mayweather, Buffett was asked how Mayweather's "intensity" compares to "managing all those billions." The billionaire rejected Gray's suggestion that it must be "nerve-wracking" to be responsible for so much money. "Oh no. This guy's got a tough job. I've got an easy job."

Buffett said he "enjoys sports" but had never been to a championship fight before. "It's a real thrill... I'm 84-years-old so it's about time."

Read MoreBuffett can also pick 'em in Vegas

Mayweather said he admires Buffett and others such as Microsoft co-founder Bill Gates and Mark Cuban, owner of the NBA's Dallas Mavericks.

"Those are the guys, when I was young, that I always looked up to." Mayweather has his own business, dubbed "The Money Team."

Buffett visited Mayweather as he was training in late August and posed for a mock "faceoff" photo.

Mayweather's Tweet of Buffett photo

Mayweather easily defeated Maidana in a unanimous 12-round decision. It was too easy a job as far as some in the crowd were concerned. The AP reports they "sometimes booed the lack of action."

Read MoreBerkshire Hathaway's 15 biggest stock holdings

While he was in Vegas, Buffett made his first-ever bet on a sports book, putting $550 on the University of Nebraska. The team won handily, making Buffett a winner, too.

—By CNBC's Alex Crippen