"It is shocking that we can expect to have a 3.5 to 4 percent growth rate in the second quarter of this year, and have a 2.5 yield on the 10-year. One of those is wrong. My guess is it would be the latter," said Art Hogan, chief market strategist at Wunderlich Securities.
Hogan said stocks were also spooked by the declines in Nasdaq and the Russell 2000, which were down for a second day after Monday's one-day bounce.
"We got a double-barreled excuse to be a risk off day. We didn't get a bounce on the Russell and it kind of breaks down support. 1050 will be key. The jury is still out on whether a correction in the Nasdaq and Russell is preceding a correction in the S&P and the Dow," he said.
Read More'Shell shock' bond move sends yields to new low
After ending at record highs Tuesday, the Dow fell 101 points Wednesday to 16,613 and the S&P 500 lost eight points and closed at 1,888. The Russell, meanwhile, lost 1.6 percent closing at 1,103, and the Nasdaq ended on the psychological 4,100, off 29 points.
The 10-year followed the lead of European bonds, which saw yields decline as investors bet the European Central Bank would move ahead with stimulus.