"The issue with internal candidates is that Microsoft has cultivated a holding-company style culture, so very few execs are broadly exposed to all areas of the business," said Al Hilwa, an analyst at tech research firm IDC.
Here's a list of potential internal candidates, with their pros and cons, based on conversations with analysts and insiders. All except Raikes and Thompson are executive vice presidents.
Satya Nadella, cloud and enterprise
PRO: A 21-year Microsoft veteran, he knows its inner workings, especially the hot areas of servers, data centers and online services. Recently promoted to run the new cloud-and-enterprise unit, he controls the infrastructure behind the services side of Microsoft's reframed vision.
CON: Although he was once a vice president in the Office unit, he might struggle to impose authority over the all-powerful Windows and Office factions—the wellsprings of Microsoft's profits and famously antagonistic toward each other.
Tony Bates, corporate strategy
PRO: Came to Microsoft two years ago as CEO of the acquired online-chat company Skype, which represents the new wave of Internet-centric, consumer-focused technology that Microsoft has had difficulty replicating. He so impressed his new boss that Ballmer put him in charge of corporate strategy and relations with developers and PC makers.
(Read more: Microsoft 'a modest growth business': Pro)
CON: May not have been at Microsoft long enough to know how to wrench it into a different shape, and his narrow specialty in the telecommunications and router field may not be broad enough to run such a large, software-based company.
Terry Myerson, operating systems
PRO: A young entrepreneur whose Web software company Microsoft bought in the late 1990s, he might bring a start-up mentality to the top job. Ballmer recently chose him to run the full range of operating systems (still the heart of the company ) ranging across Windows PCs, tablets, phones and the Xbox game console.
CON: His last assignment was running the Windows Phone unit, which won praise for its clean, stylish software but has not come close to making Microsoft a major player in the smartphone market.
Qi Lu, search and Internet
PRO: The former Yahoo executive is a heavyweight in online search and advertising, with 20 U.S. patents. He now runs the applications-and-services group, which is in charge of putting the established software businesses, such as the Office suite, onto the Web. It is a crucial part of Ballmer's reorganization plan.
CON: Under his stewardship, the Bing search engine has cost Microsoft billions of dollars without threatening Google's dominance.
Julie Larson-Green, Xbox gaming console and Surface tablet
PRO: A 20-year veteran of Microsoft and an acolyte of recently departed Windows chief Steven Sinofsky, she has intimate knowledge of the Office and Windows units, having led the redesign of both products.
(Read more: In Microsoft, young tech sees itself)
CON: Is now in charge of the devices-and-studios unit, leading Microsoft's foray into making its own computers and other hardware. The Surface tablet has had poor sales, despite initial enthusiasm. She may be marked down for her close involvement with the tepidly received Windows 8.
Eric Rudder, research and technology
PRO: A fixture in the background at Microsoft for two decades, this deeply tech-savvy exec now runs Microsoft's long-term research unit and sets overall technical strategy. He is the nearest the company has to a big thinker in the mold of Bill Gates.
CON: Never having been a business unit leader, he may not have the experience to deal with the sharp-elbowed internal politics of Microsoft to survive as CEO.
Kevin Turner, COO
PRO: Microsoft's chief operating officer for the past eight years, the former Wal-Mart exec is the power behind the company's fearsome sales operation.
CON: A professional salesman and motivator, he does not have an engineering background, which could be a liability.
Jeff Raikes, philanthropy, ex-Office chief
PRO: Bill Gates, who is on the committee to choose the next CEO, picked this former leader of the Office unit to be the chief of his foundation. As Microsoft undergoes a critical transition, his long experience, understanding of Gates' thinking and steady hand might be an effective combination.
CON: Immersed in the world of philanthropy for the past five years, he may be out of touch with the latest technology trends. The same generation as Ballmer, his appointment might be viewed as a continuation of the old guard.
John Thompson, search committee leader
PRO: It is not beyond the realm of possibility that the man leading the CEO committee may end up being considered by it. The former IBM executive went on to be CEO of computer security firm Symantec, giving him experience both of a large company reinventing itself and an understanding of the enterprise software market.
CON: He joined Microsoft's board only last year and has no direct experience managing the company. His current day job is CEO of the little-known, privately held cloud-computing firm Virtual Instruments.