U.S. indexes were moving between small gains and losses for most of the day Wednesday before Fitch put its report out at 3:30 p.m. ET. The Dow was down 75 points shortly before the report came out.
Banks, which would have the earliest and deepest losses from an escalation of Europe's financial crisis, fell by the most. Bank of America and JPMorgan Chase each closed about 4 percent lower.
The Dow Jones industrial average closed down 190 points at 11905.97.
In its report Fitch said bank exposure to Greece, Ireland, Italy, Portugal and Spain is "manageable" but further contagion poses a serious risk.
"Fitch believes that unless the euro-zone debt crisis is resolved in a timely and orderly manner, the broad credit outlook for the U.S. banking industry could worsen," the rating firm said.
Fitch's current outlook for the industry is stable, reflecting improved fundamentals at most banks combined with ratings lower than at pre-crisis levels. However, Fitch warned that risks of a negative shock are rising and could alter its outlook.
U.S. banks have reduced direct exposure to stressed European markets considerably over the past year, in Fitch's view.
The Associated Press contributed to this report.