"This is not something a president should be doing or can do," said Larry McDonald, head of the U.S. macro strategies at ACG Analytics. "It's just noise, and it's pretty far removed from reality."
"This is not a dictatorship. We have bankruptcy judges and the rule of law," McDonald said. "But it is scaring the bond market."
Municipal bond insurers with exposure to Puerto Rico's debt also saw their stocks fall. Shares of MBIA, Ambac Financial and Assured Guaranty declined 8.4 percent, 5.5 percent and 2.9 percent, respectively.
Mick Mulvaney, director of the Office of Management and Budget, tried to walk back Trump's comment on Wednesday. In an interview, he told CNN: "I wouldn't take it word for word with that. I talked to the president at length yesterday as we flew home on Air Force One."
"The primary focus of the federal effort is to make sure the island is safe and that we're rebuilding the island," he said.
Puerto Rico currently holds more than $70 billion in debt in part because of years of government overspending. Cate Long, founder of Puerto Rico Clearinghouse, told CNBC in an email that in aggregate, about 75 percent of it is held by retail investors.
Senate Minority Leader Chuck Schumer told reporters that government needs to find a "fair solution" to solve Puerto Rico's debt problems. "Maybe the horrible, horrible devastation that's been caused to Puerto Rico will make our colleagues understand that we have to have a far fairer solution to Puerto Rico's problems," he said.
Last year, Congress passed the PROMESA law that created Financial Oversight and Management Board. Its goal is to help Puerto Rico negotiate with its creditors.
PROMESA also created a mechanism known as Title III that allowed U.S. territories such as Puerto Rico to enter a form of bankruptcy, a financial tool the island did not have access to previously.
"The president can't just wipe out the debt," said Jeff Lipton, managing director, head of municipal research and strategy at Oppenheimer. "He doesn't have the power to wipe it out. To my knowledge, he cannot issue an executive order even though Puerto Rico is a United States territory."
However, Long of Puerto Rico Clearinghouse said Trump's comments could help debt-restructuring proceedings move faster. "So far it has been mired in court litigation and needs to be moved to a consensual negotiation which was required by law (Promesa), but the PR Oversight Board skipped over," Long said.
Puerto Rico has also been contending with a shrinking population in recent years. Between 2010 and 2016, the island's population has shrunk by 8.4 percent, according to the U.S. Census Bureau. The mass exodus from Puerto Rico could accelerate after Hurricane Maria hit the island two weeks ago.
—CNBC's Patti Domm and Reuters contributed to this report.