Ballmer said on "Squawk Box" he recalled the offer being "$24 billion, when the company was itsy bitsy."
At the time, Zuckerberg said no, Ballmer said. "I respect that." He added, "You have to have a willing seller."
Zuckerberg's decision to pass certainly worked work well for him.
Facebook has a current market value of around $374 billion. And according to Forbes, Zuckerberg is the fourth richest person in the world with a net worth of nearly $56 billion, as of Friday.
Ballmer, now the owner of the Los Angeles Clippers, was CEO of Microsoft from 2000 to 2014. Forbes puts Ballmer at No. 15 on its billionaire list, with a net worth of about $28 billion.
During Friday's CNBC interview, Ballmer also revealed that, despite speculation to the contrary, he never wanted to buy Twitter. There had been rumors that Ballmer and Saudi Prince Al-Waleed bin Talal, both major Twitter shareholders, were interested in making a bid.