Art Cashin, UBS' director of floor operations at the NYSE, said there is a lot of background nervousness in the markets because of events around the world.
"You are experiencing some of that [flight-to-safety trade]," Cashin noted. "Some of the risk monitors indicate there's come geopolitical concern out there."
Anxiety over Brazil's presidential election, pro-democracy protests in Hong Kong, political clashes in Spain and continued fears about ISIS dragged down the Dow Jones Industrial Average as many as 178 points. Brazil stocks are down across the board with Petroleo Brasileiro falling double digits; the iShares MSCI Brazil Capped ETF (EWZ), which holds a basket of Brazilian stocks, is at a six-month low and having its worst day in three years. Hong Kong closed at a two-and-a-half month low and Spain lost more than 1 percent as bond yields rose.
Read MoreArt Cashin: Global growth concerns weigh on markets
Despite the global worries, U.S. stocks recovered some of their gains mid-morning as market watchers said investors may be buying into the U.S. as part of a flight-to-safety trade. Major indexes lost ground after the European close, though, showing no rush into U.S. stocks.
Cashin warned that geopolitical concerns may stay at the forefront for the rest of the week.
"At the end of the week, we have Yom Kippur and the hajj to Mecca," he said. "That will be very tempting to terrorists to try to disrupt."
—By CNBC's Kristen Scholer.