Financials closed 1.45 percent lower as the greatest S&P 500 decliner. Credit card stocks traded sharply lower after a Tuesday 8K filing from Synchrony Financial forecast an increase in net charge-offs and higher reserve builds in the coming months. In a note, Deutsche Bank called the announcement a "huge negative surprise" and was concerned the rising rate of reserve coverage indicated poorer credit quality. Shares of Synchrony closed 13 percent lower, negative for the year so far.
The Dow Jones industrial average closed about 57 points lower after earlier falling 136 points. American Express, Home Depot and Goldman Sachs contributed the most to declines.
Treasury yields turned higher in late trade, with the 10-year yield near 1.62 percent and the 2-year yield near 0.72 percent. Earlier, both yields touched their lowest since mid-February.
"The 10-year (Treasury yield) is now changing for the market from every pullback to be short-lived in scope to be potentially more damaging," Sarhan said.
Early morning in London, the German 10-year bund yield fell into negative territory for the first time to hit a low of minus 0.034 percent. The U.K. 10-year gilt yield hit a fresh record low of 1.115 percent. The Japanese 10-year yield hit a record low of negative 0.168 percent.
Peter Boockvar, chief market analyst at The Lindsey Group, said the decline in the bund yield into negative territory was due to "momentum and speculating." There's a sentiment of "I don't care if I'm buying something with negative yield, if I can flip it and buy something with more negative yield," he said.
U.S. crude oil futures settled down 39 cents, or 0.80 percent, at $48.49 a barrel.
We're "following crude," said Jeremy Klein, chief market strategist at FBN Securities. "You've got the Fed tomorrow. Everyday you have seemingly a disappointing poll on Brexit. People are just not putting on a lot of risk right now."
"Everyone's waiting for tomorrow and almost more importantly, next Thursday," he said.
The U.K. is set to vote June 23 on whether to leave the European Union. The latest Brexit poll showed a gain in the "leave" camp, according to a Reuters report of a TNS online poll published Tuesday. Late Monday, The Sun newspaper endorsed Brexit.
Sterling was near $1.4109 after hitting its lowest against the dollar since mid-April. The U.S. dollar index was up 0.6 percent, with the euro around $1.121 and the yen around 106.1 yen against the greenback.
The Federal Open Market Committee kicked off its two-day meeting Tuesday and Fed Chair Janet Yellen is scheduled to hold a press conference at the meeting's conclusion Wednesday afternoon. The central bank is also set to release its statement and summary of economic projections.
All 41 respondents to CNBC's Fed Survey expected Fed policymakers to keep rates unchanged at the meeting this week.
The Nasdaq composite closed mildly lower. Earlier, the index attempted gains, helped by gains in shares of Apple and Yahoo. The iPhone maker's stock gained 0.12 percent for the day, while Yahoo closed 2.55 percent higher.
"I think people are sort of going ahead of Yellen because they rightly assume Yellen is going to be dovish," said Peter Boockvar, chief market analyst at The Lindsey Group. He said he is also watching U.S. market action after the European market close and that markets were likely too complacent about the possibility of Brexit.
European stocks were lower, with the German DAX off about 1.4 percent and the STOXX Europe 600 falling 1.9 percent.
In U.S. economic news, retail sales rose a more-than-expected 0.5 percent in May. Ex-autos, gasoline, building materials and food services, retail sales rose 0.4 percent.
"I though it was a pretty solid report given we're seeing deflation out there for a lot of prices," said Steve Rick, chief economist at CUNA Mutual Group. "Basically seeing consumers didn't roll over in the month of May even though we didn't see much job growth."
Import prices rose 1.4 percent, their largest gain in more than four years, while export prices jumped 1.1 percent in May. Business inventories rose 0.1 percent in April.
In other economic news, the NFIB (National Federation of Independent Business) survey said its small business optimism index rose 0.2 points to a reading of 93.8 last month, remaining below the 42-year average of 98.