This decision is expected to put more pressure on the new Greek government to come to a deal on the country's bailout program.
Global X Funds' Greek ETF fell more than 10 percent on the news.
Read MoreGreece, Ukraine and Russia: History lessons
The ECB's move is "not as big a deal as it seems," TrendMacro CIO Donald Luskin wrote in a note, especially in light of Wednesday morning's announcement that the ECB had authorized Greece's central bank to use "Emergency Liquidity Assistance" for Greek banks as needed.
"If you put both of today's policy moves together, what it means is that the Greek government—not the Eurosystem—will be on the hook for the collateral. But there will still be a funding mechanism for the banks, in case there is a serious run on them. That's the important thing," Luskin wrote.
Greece's new government has sought to overhaul the nation's bailout program with European Institutions. Greek Prime Minister Alexis Tsipras met with European Commission President Jean-Claude Juncker and the president of the European Union's parliament, Martin Schulz, in Brussels earlier on Wednesday.
Read MoreUK warns Greek debtrow 'biggest risk to global economy'
Speaking at a news conference after their talks, Schulz said that Tsipras' trip around Europe showed that Greece wanted to embrace the region, rather than separate itself from it. He added that "both sides are fighting for mutual understanding" and both had to find compromises in negotiations over Greece's bailout program.
Schulz added, however, that "it is a difficult time again and we don't yet have the necessary solutions."
For his part, Tsipras said that the history of the EU was one of disagreements that ended in compromise and accords.
"We have the willingness to find a mutually acceptable agreement and compromise for our common future," he said.
—CNBC's Holly Ellyatt and Ben Berkowitz contributed to this report.