"I have said from the get-go that I think a special prosecutor is the way to go, but now with what's happened it is the only way to go," Schumer told reporters on Tuesday.
The FBI had been investigating Russia's alleged interference in the 2016 presidential election as well as any possible links to the Trump campaign, as part of its counterintelligence mission.
Schumer called on Deputy Attorney General Rod Rosenstein to appoint a special prosecutor in the Russia probe.
"Mr. Rosenstein, America depends on you to restore faith in our criminal justice system, which is going to be badly shattered after the administration's actions today," Schumer said.
The New York senator said that Comey's firing was part of a "deeply troubling pattern" of behavior from the White House. Schumer cited the Trump administration's similarly abrupt removal of former acting Attorney General Sally Yates and former U.S. attorney Preet Bharara.
Yates was fired when she refused to defend Trump's first executive order on immigration, which would have restricted travel from seven Muslim-majority nations. Bharara was dismissed when he refused to resign, as the White House had demanded of Obama-appointed federal prosecutors.
Many congressional Democrats also called for a special prosecutor, including Sen. Mark Warner, vice chairman of the Senate Select Committee on Intelligence.
"The only way this Administration can begin to demonstrate a commitment to the rule of law, which has so far been sorely lacking, is to cooperate fully with the ongoing congressional investigations and to support the appointment of an independent special counsel," Warner said in a statement.
Sen. Bernie Sanders said that "it is clear that whomever President Trump handpicks to lead the FBI will not be able to objectively carry out the Russia investigation."
Trump's sudden termination of Comey sparked similar calls from notable Republicans as well, including Sen. John McCain, who said he was disappointed by the president's decision.
"I have long called for a special congressional committee to investigate Russia's interference in the 2016 election. The president's decision to remove the FBI Director only confirms the need and the urgency of such a committee," McCain said in a statement.
Sen. Richard Burr, chairman of the Senate Intelligence Committee, said he was "troubled by the timing and reasoning" behind Comey's removal, saying he considered the fired FBI director "to be a public servant of the highest order."
"Director Comey has been more forthcoming with information than any FBI Director I can recall in my tenure on the congressional intelligence committees. His dismissal, I believe, is a loss for the Bureau and the nation," Burr said in a statement.
Rep. Justin Amash, who serves on the House Committee on Oversight and Government Reform, said his staff is looking at legal provisions for establishing an independent commission on Russia.