In Fed chair race, Obama's loyalty to Summers could be key

Saul Loeb | AFP | Getty Images

President Obama and Larry Summers in file photo.

The oh-so-subtle behind-the-scenes battle to succeed Ben Bernanke as chairman of the Federal Reserve could come down to something as simple as a conference call.

When candidate Barack Obama was prepping for a crucial meeting in 2008 with President George Bush and his GOP rival, John McCain, in the thick of the fiscal crisis, Larry Summers took a lead role prepping the candidate on urgent economic issues. The meeting proved a turning point for Obama's campaign as he pulled away from Bush in the polls.

Subsequently, Summers was joined by heavyweights Robert Rubin and Warren Buffett on semiregular calls with Obama about the economy.

Not on the calls: Janet Yellen, the other presumed candidate for Fed chair.

As a consummate Washington insider, Summers knows that face time with a presidential candidate—especially one grappling with a crisis—is second in value only to face time with a sitting president. And once he got an office in the West Wing after Obama's 2009 swearing in, Summers had that, too.

But in the years Summers sat just one floor away from the president at 1600 Pennsylvania Ave., Janet Yellen was president of the San Francisco Fed—a job that doesn't come with a White House hard pass. Today, she's the vice chair of the Fed Board of Governors, a job description that does not require frequent visits to the Oval Office.

Several former White House insiders say that's the key difference between Summers and Yellen in the jousting for Bernanke's big chair: Obama knows Summers and trusts his judgment, and he doesn't have much of a relationship with Yellen.

(Read more: Obama likely won't announce Fed chair pick until fall)

"While economists are famous for 'on-the-one-hand this, on-the-other-hand-that,' Larry's given the president a lot of definitive advice," said one veteran of the Obama White House. "He has a high comfort level with Larry and a lot of faith in Larry's take."

That's not to say that the president is unaware of the rockier moments in Summers' history—his controversial 2005 comments on women in the sciences, or his sharp-elbowed style of bureaucratic infighting in Washington. But Obama discounts them.

"To the extent that Larry comes with baggage, it was pre-NEC baggage," said another former White House insider, referring to Summers' tenure as Obama's chair of the National Economic Council. "Working closely together for years tends to exorcise those demons."

Even former White House insiders who think Summers will get the nod acknowledge that his sometimes brusque personal style could play poorly at a high-profile, Bernanke-style Fed press conference.

"In some ways, he's not perfectly suited in terms of the communications element of the job," said the former insider. "With one poorly chosen adjective, you could shave 500 points off the Dow." That's one reason a Senate confirmation hearing for Summers, this person said, would be "rancorous."

(Read more: Wall Street wants Yellen, not Summers, as next Fed chief)

Still, the insider said the president has been thinking of Summers as Fed chair for a long time. "This is something they talked about way back in the day," the person said. "There wasn't a promise of a job, but in the course of their years working together Larry expressed interest in the job. Loyalty does mean something."

Obama is assessing all that as he makes his decision, which White House officials say isn't likely to come until the fall.

"If the president appoints Summers, it's because the president judges that he's mature enough not to bring this baggage with him," said the first insider.

A third former Obama White House insider says Summers' insider status is magnified by the fact that many Obama advisors weighing in on the decision owe their own jobs or career advancement to Summers.

"It just feels weird," said the insider. "It does seem to be a system built for identifying Larry as the pick."

—By CNBC's Eamon Javers. Follow him on Twitter: @eamonjavers