The changes will likely include a shake-up of some top executives, according to an All Things D report, which cited sources close to the situation.
(Read More: Big Trader Connects With Microsoft)
Satya Nadella, president of the servers and tools division; Tony Bates, president of Skype; and Don Mattrick, president of the interactive entertainment business, may get expanded roles from the revamp, according to the report.
However, sources also said that the changes are still being figured out and could shift at any time, according to the report.
This isn't the first time that a revamp in the company's organization has been mentioned. Ballmer spoke about Microsoft shifting to a "devices and services" focused company in his last shareholder letter.
"This is a significant shift, both in what we do and how we see ourselves—as a devices and services company. It impacts how we run the company, how we develop new experiences and how we take products to market for both consumers and businesses," Ballmer said in the letter.
Microsoft didn't immediately respond to a request for comment.
_ By CNBC's Cadie Thompson. Follow her on Twitter @CadieThompson.