His allies say he will rebalance the country's foreign policy, which tilted heavily towardsChina in recent years as Rajapaksa fell out with the West over human rights and allegations of war crimes committed at the end of a drawn-out conflict with Tamil separatists in 2009.
U.S. Secretary of State John Kerry was quick to welcome the successful election and commended Rajapaksa for accepting the verdict of the nation's 15 million voters.
Read MoreSri Lanka gears up for tight election race
"I look forward to working with President-elect Mithripala Sirisena as his new government works to implement its campaign platform of a Sri Lanka that is peaceful, inclusive, democratic, and prosperous," he said in a statement.
Indian Prime Minister Narendra Modi called Sirisena to congratulate the new leader of "a close friend and neighbor".
Rajapaksa had cold-shouldered New Delhi in recent years, but Sirisena told an Indian newspaper this week that "we will revert to the old, non-aligned policy".
"India is our first, main concern. But we are not against Chinese investment either. We will maintain good relations with China too," he told the Hindustan Times.
The Department of Elections said that of 7.6 million votes counted so far, Sirisena had taken 52.05 percent and Rajapaksa was trailing on 46.7 percent.
The results showed Rajapaksa remained popular among Sinhala Buddhists, who account for around 70 percent of the country's 21 million people, but Sirisena took his lead from the ethnic Tamil-dominated former war zone in the north and Muslims-dominated areas.
Rajapaksa won handsomely in the last election in 2010, surfing a wave of popularity months after the defeat of the Tamil Tiger rebels.
But critics say he had become increasingly authoritarian, with several members of his family holding key positions of power, and although the economy had blossomed since the end of the war, voters complained of the high cost of living.
Rajapaksa had called the latest election two years early, confident that the usually fractured opposition would fail to come up with a credible candidate. But he did not anticipate the emergence of Sirisena, who shared a traditional Sri Lankan dinner with him one evening and turned on him the next day.
Sirisena will lead a motley coalition of ethnic, religious, Marxist and center-right parties, which analysts say could hamper economic reform and encourage populist policies.
He has pledged to abolish the executive presidency that gave Rajapaksa unprecedented power and hold a fresh parliamentary election within 100 days.
He has also promised a crackdown on corruption, which would include investigations into big infrastructure projects such as a $1.5 billion deal with China Communications Construction to build a port city.
It is not clear if the port, to be built on land reclaimed from the ocean in Colombo, will be cancelled. However, his backers have said a casino licence given to Australian gambling tycoon James Packer's Crown Resorts will be withdrawn.