The European Commission in Brussels said in a statement that the vote went against the principle of free movement of people. It said it would examine the implications for its relations with Switzerland, taking into account the position of the government, which had urged citizens to vote 'no'.
"For us, EU-Swiss relations come as a package," said Hannes Swoboda, a member of the European Parliament. "If Switzerland suspends immigration from the EU, it will not be able to count on all the economic and trade benefits it is currently enjoying. We will not allow ... cherry-picking."
In a nail-biting vote, 50.3 percent backed the "Stop mass immigration" initiative, which also won the required majority approval in more than half of Swiss cantons or regions, Swiss television said.
The outcome obliges the government to turn the initiative, spearheaded by the right-wing Swiss People's Party (SVP), into law within three years.
It reflects growing concern among the Swiss population that immigrants are eroding the nation's distinctive Alpine culture and contributing to rising rents, crowded transport and more crime.
Net immigration runs at around 70,000 people per year on average. Foreigners make up 23 percent of the population of 8 million, second in Europe only to Luxembourg.
(Read More: Will emerging markets become a euro zone-style risk?)
"This is an enormously important decision because the direction must now be shifted," SVP politician Luzi Stamm told Swiss television. "The Swiss population have said that, instead of free movement of people, quotas have to be introduced."
The Swiss system of direct democracy - which allows for up to four national referenda per year - means popular dissatisfaction can be translated into action relatively easily.
However, such concerns are being echoed around the EU's wealthier countries, where anti-immigration parties such as the UK Independence Party look set to make big gains in elections to the European Parliament in May.
"I fear a 'yes' from Switzerland would set off a further round of debate about free movement of persons in the EU," European Parliament President Martin Schulz told the Swiss newspaper NZZ am Sonntag ahead of the vote.
The provisions of the initiative require the restriction of residence permits for foreign nationals, including cross-border commuters and asylum seekers, according to quotas, the government said in a statement.
These limits will now need to be defined at a legislative level, it said.
Nestle and Hayek
Foreign professionals have helped to power Switzerland's economic success story over the past 150 years, from German-born Henri Nestle, who gave his name to the world's largest food company, to Nicolas Hayek senior, who founded Swatch Group, the world's biggest watchmaker.
The European Union is Switzerland's biggest trading partner, buying 110 billion Swiss francs' ($122 billion) worth of goods in 2013.
(Read More: Swiss banks agree to US tax deal, more expected to follow)
Opponents of the move say it could exacerbate a shortage of skilled workers in Switzerland, the home of Roche, Novartis, UBS and other multinationals filled with foreign professionals.
"Explanatory and constructive talks with the EU are needed urgently," the Swiss Banking Association tweeted after the result.
Swiss voters generally have a history of voting down proposals that they feel could hurt their country's economic success story or threaten its competitiveness.
Last year, they rejected a proposal to cap the salaries of top executives at 12 times that of the company's lowest wage, amid warnings from industry leaders that the economy could suffer.
But immigration has become a growing concern. In 2009, Swiss voters defied government advice by backing a ban the construction of minarets, and in 2010 they voted to automatically deport foreigners convicted of serious crimes.
Follow us on Twitter: