The loss was also keenly felt across the African continent.
"We are in trouble now, Africa. No one will fit Mandela's shoes," said Kenyan teacher Catherine Ochieng, 32.
Former Zambian President Kenneth Kaunda, an old ally of Mandela's in the struggle against apartheid, hailed him as "a great freedom fighter".
Politicians now 'nothing like Mandela'
For South Africa, the death of its most loved leader comes at a time when the nation, which basked in global goodwill after apartheid ended, has been experiencing labor unrest, growing protests against poor services, poverty, crime and unemployment and corruption scandals tainting Zuma's rule.
Many saw today's South Africa — the continent's biggest economy but also one of the world's most unequal — as still distant from the "Rainbow Nation" ideal of social peace and shared prosperity that Mandela had proclaimed on his triumphant release from prison in 1990.
"I feel like I lost my father, someone who would look out for me," said Joseph Nkosi, 36, a security guard.
Referring to Mandela by his clan name, he added: "Now without Madiba I feel like I don't have a chance. The rich will get richer and simply forget about us. The poor don't matter to them. Look at our politicians, they are nothing like Madiba."
The crowd around Mandela's home in Houghton preferred to celebrate his achievement in bringing South Africans together.
For 16-year-old Michael Lowry, who has no memory of the apartheid system that ended in 1994, Mandela's legacy meant he can have non-white friends.
"I hear stories that my parents tell me and I'm just shocked that such a country could exist. I couldn't imagine just going to school with just white friends," Lowry said.
Tutu tried to calm fears that the absence of the man who steered South Africa to democracy might revive some of the ghosts of apartheid.
"To suggest that South Africa might go up in flames — as some have predicted — is to discredit South Africans and Madiba's legacy," Tutu said on Thursday. "The sun will rise tomorrow, and the next day and the next ... It may not appear as bright as yesterday, but life will carry on."
May hurt ANC in long-term
Zuma and his ruling African National Congress face presidential and legislative elections next year which are expected to reveal discontent among voters about poverty and unemployment 20 years after the end of apartheid.
But the former liberation movement is expected to maintain its dominance in South African politics.
Mark Rosenberg, Senior Africa Analyst at the Eurasia Group, said that while Mandela's death might give the ANC a sympathy-driven boost for the next elections, it would hurt the party in the long term.
He saw Mandela's absence "sapping the party's historical legitimacy and encouraging rejection by voters who believe the ANC has failed to deliver on its economic promises and become mired in corruption".
Mandela rose from rural obscurity to challenge white minority rule — a struggle that gave the 20th century one of its most respected and loved figures. He was among the first to advocate armed resistance to apartheid in 1960, but was quick to preach reconciliation and forgiveness when the white minority began easing its grip on power 30 years later.
He was elected president in all-race elections in 1994 after helping to steer the divided country towards reconciliation and away from civil war.
Mandela was awarded the Nobel Peace Prize in 1993, an honor he shared with F.W. de Klerk, the white Afrikaner president who released him in 1990.
In 1999, Mandela handed over power to younger leaders better equipped to manage a modern economy, a rare voluntary departure from power cited as an example to African leaders.
This made him an exception on a continent with a bloody history of long-serving autocrats and violent coups.