Although the PP got more votes than any other party, it and the rival Socialists fell short of overall majorities in most areas. The two parties will have to negotiate coalitions with minority parties in the 13 of Spain's 17 regions that voted on Sunday alongside more than 8,000 towns and cities.
Spain has virtually no tradition of compromise politics and the fragmented vote is likely to result in weeks of pact-building in the regions which hold substantial devolved power and determine spending in key areas like education and health.
"Market sentiment towards Spain may be favourable but the political scene is becoming a lot more fragmented, boding ill for the formation of a stable and strong government after the parliamentary vote later this year," said Nicholas Spiro, analyst at Spiro Sovereign Strategy.
The PP got its worst result in countrywide municipal elections since 1991 and lost its absolute majority in regional bastions Madrid and Valencia, where potential left-wing coalitions could send the party into opposition for the first time in 20 years.
Read MoreHas Europe's populist tide peaked?
"I used to vote for the PP but they are burnt out, they have been in power for too long. It's time to clean the slate," said Nacho, a 56-year-old doctor in Valencia who voted for Ciudadanos.
At the PP headquarters in Valencia, dozens of shell-shocked supporters, many of them young activists, fought back tears as they received the news their party would also likely lose control of the city to a left-wing coalition.
In Madrid city, where there has been a PP mayor since 1991, Rajoy's party marginally beat a leftist platform backed by Podemos and headed by 71-year-old retired judge Manuela Carmena. But there as well the Podemos-backed alliance is likely to team up with the Socialists to win power.
Podemos, often compared to Greek radical left party Syriza, had toned down its policies in recent weeks, scrapping more extreme ideas like defaulting on the national debt.
Read MoreWealth gap widens in developed world
"It's time for total change," said 31-year-old teacher Natalia Cendejas in Madrid's old quarter, Lavapies, where immigrants and the working class rub shoulders with bohemians and tourists.
In Barcelona, another left-wing coalition headed by former community activist Ada Colau and backed by Podemos beat pro-independence parties Convergencia i Unio (CiU) and Esquerra Republicana de Catalunya (ERC), in a setback for the Catalan separatist movement.
"The people have won here, not a bunch of initials," Colau said to cheering supporters, referring to the scramble of initials representing traditional Spanish political parties.