Multiple sources on Wall Street say Cohn is among the top contenders, along with former Fed Governor Kevin Warsh and FDIC Vice Chair Thomas Hoenig, to replace Yellen.
"He's the leading contender," said Christopher Whalen, an insider in the banking world and currently head of Whalen Global Advisors. "Every Fed chairman in recent memory going back even to (Paul) Volcker went through the White House in one way or the other. ... It would certainly make sense."
A White House spokeswoman said the chatter was "entirely speculation" and called the Beacon report and any others "inaccurate."
On a personal level, Whalen said he favors Warsh for the position but believes Cohn "wouldn't be a bad choice." Whalen thinks Cohn would "have no choice" but to advocate for keeping rates low during what Whalen believes will be an economic slowdown ahead.
There are several issues on the downside for Trump to pick Cohn.
Most notably, the president is relying on Cohn to usher his economic agenda through Congress and help with the implementation, particularly regarding tax reform. If Trump can't get a tax plan passed by next February, that would make it tougher to let go of Cohn.
Moreover, CNBC recently reported that Cohn actually may have pushed Trump to keep Yellen on board. However, it's not clear whether Yellen, 70, even would want to be given another them.
In the case that Yellen would want to leave, Cohn, along with Hoenig, Warsh and Stanford economist John Taylor, "would certainly be on the short list," Valliere said. "Cohn would be an interesting pick, though there would be some concern that he was politicized and the markets could view him with some suspicion that he is too close a political ally with Trump."
Then there's the Goldman connection. Critics have ripped into Trump for perpetuating the "Government Sachs" culture in Washington where so many of the firm's principals have found their way into high levels of government.
"It remains unclear though, especially this many months in advance of Yellen's term ending, as to whether there will be sufficient support for Cohn for this role within the West Wing at that time and whether he could even get confirmed given his lack of conservative credentials and academic background, not to mention his prominent ties to Goldman Sachs," Beacon said in its note.
Watch: Cohn backs splitting lending & investment banks