Microsoft's next CEO isn't who you think

Microsoft CEO Steve Ballmer
Yoshikazu Tsuno | AFP | Getty Images
Microsoft CEO Steve Ballmer

Now that Steve Ballmer is out, there's a ton of speculation about who will be taking the helm at Microsoft. While there are plenty of names being floated as potential candidates, odds are the company's next CEO is probably not anyone you would expect.

And that, Microsoft watchers say, is precisely what the company needs: fresh blood.

(Read more: Microsoft CEO Ballmer to step down within 12 months )

"Bill (Gates) would like the successor to be someone that he knows and someone that he is comfortable with. I don't think that person is necessarily an obvious person in the company," said Rick Sherlund, head of U.S. technology research at Nomura Securities, on CNBC's Squawk on the Street.

"I do not believe there is a successor in waiting. ... Unfortunately, at Microsoft there has been enormous turnover of senior people under Ballmer, so we are left with no obvious choices here."

But the fact that there is no obvious candidate within Microsoft may actually bode well for the company, because there are major changes needed that are only possible with someone from outside the company, analysts say.

"CEO changes are tough when you are in a business like this. Microsoft missed the transition to tablet, missed the transition to smartphones, those problems are all still there," said Dan Niles, CIO of AlphaOne Capital Partners. "The new CEO needs to be amazingly good because he is going to have his work cut out for him."

Pushed out?

Ballmer's announcement comes just one month after the company's corporate shake-up. While Ballmer had probably hoped that the push to turn the company around would help ensure his position, shareholder activism proved that the shake-up wasn't enough and investors wanted Ballmer out, analysts said.

(Read more: Will Microsoft investors win from shake-up?)

"This validates that the shareholder activism agenda is likely to be accomplished one way or another," Sherlund said.

The most notable investor who wanted Ballmer out was probably Gates, Niles said.

"The only way Steve Ballmer is gone is because Bill Gates wants him gone, in my opinion. There's no other way this happened," Niles said.

Microsoft's stock surged by 7 percent after news of Ballmer leaving, adding $24 billion to the company's market capitalization from Thursday's close.

However, investors shouldn't expect the growth to last in the long-term, Niles said.

(Read more: Steve Ballmer's bench is empty: Who's next at the helm? )

"In the near-term, the stock probably heads higher, but let's not forget this business has a lot of issues and there's a lot of precedence for replacing the CEO and still having the stock go a lot lower," Niles said. "I hope they do some out-of-the-box thinking and they get somebody from outside the company. If they bring someone from inside the company, I think that's going to be a very bad decision."

But who?

Some big names suggested as viable candidates include Steven Sinofsky, former president of the Windows Division; Scott Forstall, former senior vice president of iOS software at Apple; and even Bill Gates. And while these are all interesting candidates, it's unlikely any of these candidates would take the job, Milanesi said.

Sinofsky had a reputation much like Forstall (difficult to work with), so it wouldn't make sense for Microsoft to bring him back, said Carolina Milanesi, a technology analyst for Gartner. As for Gates, it's unlikely he would come back to run the company since he has moved onto other projects.

"It's not necessarily going to happen, but it would be interesting for the industry since Jobs is no longer with us. They were both seen as pillars in the industry," Milanesi said. "We don't have a leader like that in the industry right now, there's a lot of good leaders, but not necessarily such a big person personality wise."

—By CNBC's Cadie Thompson. Follow her on Twitter @CadieThompson.