"The breadth of the market has been deteriorating and it's finally caught up with the big-cap names people were hiding in," said Peter Boockvar, chief market analyst with The Lindsey Group.
Adding to the negative tone was the steep drop in King Digital, creator of the "Candy Crush" game. The IPO opened below its $22.50 offering price, and proceeded to fall 15 percent on the day.
Traders said while the IPO was viewed as overpriced by some, it could be a possible negative for the overall IPO market which has been very active lately.
Read MoreNear record for S&P dividends and share buybacks
The Dow fell 98 points to 16,268, and the S&P 500 was off 13 at 1,852. Paul LaRosa, chief market technician at Maxim Group, said the Nasdaq and Russell 2000 broke key support levels and look to be headed lower.
The small-cap Russell 2000 ended the day down 1.9 percent, its worst performance since Feb. 3. It is now down 5 percent from the all-time high it set on March 4. The Nasdaq closed down 1.4 percent to 4,173 on Wednesday, and is now off 5 percent from its March 4 intraday high.
"Trees don't grow to the sky, so there's got to be corrections. That's the way the market works. Certain sectors and certain stocks have gotten extremely overbought and that leads to a selloff," said LaRosa, adding it's yet to be seen whether the market is on track for a substantial correction or just a pull back.
Banks may be a positive influence on the market Thursday, after many of the major names announced dividend hikes and share buyback plans after clearing the Fed stress test. Shares of JPMorgan, PNC and Wells Fargo, among others, moved higher in late trading. But Citigroup did not, as its capital plan failed to pass muster. Its stock sunk nearly 6 percent.
Read MoreStocks have become 'absurd': Bill Fleckenstein
While stocks were drifting earlier Wednesday, traders turned their focus to action in the options market where there was talk of a big put buyer that unsettled stock traders. Patrick Kernan, who trades S&P 500 options at the CBOE, said he saw a few investors making bets that could be construed as bearish for stocks.
"We had a big put spread buyer. We had a couple of bigger trades. One seemed like a directional trade where a guy was just getting straight short the market," said Kernan, who works at Cardinal Capital. "We had a guy who bought 10,000 April 1,790/1,850 puts spreads which I assumed was directional. We definitely saw some trade today that felt like bets on near-term downward movement."
"That's the big thing that was overhanging the market, but I think the high beta names are coming off because the hedge funds are getting hurt," said one trader.
Read MoreDr. Doom: Watch out if the Fed makes this mistake
Meanwhile, bonds caught a bid and yields fell. David Ader, chief Treasury strategist at CRT Capital, said part of the reason may be a quarter-end rotation.
The other factor was a strong five-year note auction. The yield on the five-year broke its recent feverish range, and was trading at 1.67 percent after the auction. Since the Fed's meeting last week, the five-year yield moved higher, rising to between 1.7 and 1.75 percent.
"The first and foremost thing is to recognize that the issue du jour is the five year and it had gotten very cheap over the last couple of weeks," said Ader. "It got so cheap in fact that what was for us at least for the five-year auction a record nondealer bid."
The five-year yield moved higher after the Fed meeting last week and comments from Fed Chair Janet Yellen that the Fed could move to raise rates six months after it ends its quantitative easing program. It cut another $10 billion a month in QE bond purchases at its meeting last week.
Read MoreMarket frothiness in 'inning seven': IPO analyst
"There's been divergences in the stock market for days and today did not just come out of nowhere," said Boockvar. "We're one meeting away from QE being cut in half and that changes behavior. There's a reason why the air is coming out. To me, it goes back to we're getting closer to the taper."
Ader said the "real' buyers helped boost the bond market Wednesday, as did weak durables goods data, which showed a decline in core orders.
"Maybe people want to get some paper on the books ahead of quarter-end," he said.
As for stocks, the major indexes are lower or close to flat on the quarter. The Dow is down 1.8 percent for the quarter so far, while the S&P 500 is off 0.3 percent and the Nasdaq is off 0.1, and the Russell is off 0.7 percent.
Data Thursday includes jobless claims data at 8:30 a.m. ET, and the final look at fourth-quarter GDP, expected at 2.7 percent. Pending home sales are reported at 10 a.m. There are also a few earnings reports, including Lululemon Athletica, Winnebago, Worthington Industries, Gamestop, Signet Jewelers, Red Hat and Voxeljet.