Lee has said that the chances of a non-Chinese becoming prime minister are better for the new generation of leaders but a lack of Mandarin, widely spoken here, could be a problem.
For some Singaporeans, though, the idea is as outlandish as a non-Malay prime minister in Malaysia or an Indonesian from outside the political heartland of Java becoming president.
In a book published two years before his death this year, Lee Kuan Yew, Lee's father and the deeply respected first prime minister of this tropical city-state, listed four ethnic Chinese men as the new generation of up-and-coming leaders.
Still, Shanmugaratnam's hustings performance in the run-up to last week's election was so impressive that even an opposition candidate, Paul Tambyah of the Singapore Democratic Party, openly longed for him to lead a grand coalition of parties.
"People would like to see Tharman around to set the tone for a new PAP leadership," said Catherine Lim, a long-time political commentator and critic of Lee Kuan Yew.
"It's time now for a completely different one, and the only person whom I can think of to set that tone convincingly and who can appeal to Singaporeans across ethnic groups would be Mr Tharman," she said.
Read MoreElection fever spurs tweeting spree in Singapore
Shanmugaratnam, 58, said in July he was not keen on the prime minister's job, though he expected Singapore to have a leader from one of its minority ethnic groups at some point. He was not available to comment for this article.
A transitional Prime Minister?
The PAP won almost 70 percent of the popular vote in the election, a stunning recovery from its record low of 60.1 percent in 2011. In his own district, Shanmugaratnam led a handful of lawmakers to a win with about 80 percent of the vote.
Analysts say that rebound was helped by a wave of patriotism after the death of Lee Kuan Yew and independent Singapore's 50th birthday celebrations, but also by a slight shift from unbridled capitalism to Western welfarism that was led by Shanmugaratnam.
In his campaign speeches, Shanmugaratnam pressed the right buttons for an electorate that has in recent years begun to question the hard-nosed growth-at-all-costs policies of the PAP that left many marginalized and struggling to make ends meet.
In a calm baritone and with his trademark avuncular style, he crunched numbers to show how social welfare is working.
He also explained changes the PAP has embraced after 50 years of unbroken rule, but conceded still more were needed.
"It used to be a top-down government, often quite heavy-handed," he told one rally. "It's no longer that way ... Strong leadership is listening, engaging, moving with people."
Read MoreSingapore ruling party wins decisive election victory
Shanmugaratnam spoke some Mandarin on the campaign, and when he quoted from an ancient Chinese poem at one rally the crowd exploded with cheers.
He was educated at the London School of Economics, Cambridge and Harvard, and spent most of his career at the Monetary Authority of Singapore, the island state's central bank and financial regulator.
He got into a legal tangle in the 1990s when he was fined for failing to protect the secrecy of official information after economic data was published in a newspaper ahead of its release. Shanmugaratnam had pleaded not guilty.
He is also well known on international circuits: a darling of international investors, he was appointed chairman of the International Monetary Fund's policy steering committee in 2011.
Eugene Tan, a law professor at Singapore Management University and a political commentator, said one obstacle for Shanmugaratnam is that he is seen as part of the prime minister's generation, when perhaps ideally a new generation would be coming forward.
"However, if it is assessed that a transitional prime minister is needed while the fourth generation is ready to take over, then ... Tharman is well-positioned to step up," Tan said.