Billionaire investor Warren Buffett told CNBC he really missed the boat on Amazon.

While long admiring Amazon founder Jeff Bezos, Buffett said on "Squawk Box" that "I don't have a good answer" for not investing in the e-commerce giant.

"Obviously, I should have bought it long ago," the Berkshire Hathaway chief reflected. "I didn't understand the power of the model."

"It's one I missed big time," he said. "Retailing is tough for me to figure out" because the internet has swept in and offered shoppers variety and low prices at their finger tips.

Bezos is a "terrific business person," Buffett said, marveling at how Amazon's founder could have thought about taking over the world by "selling books online."

Of course, Amazon now sells practically everything over the internet, while also getting into other businesses such as cloud computing and video production.