U.S. stocks closed lower, with selling accelerating in late afternoon trade after oil broke below $40 a barrel and there was more evidence of a weak manufacturing sector in the Fed's Beige Book. (Tweet This)
The sell-off began after Fed Chair Janet Yellen reaffirmed the case for a rate hike at the Fed's Dec. 16 meeting but reiterated that the decision will be data dependent.
"The question of whether the Fed is making a mistake still lingers out there," said Art Cashin, head of NYSE floor operations for UBS.
The Beige Book said economic activity grew at a modest pace in most regions. The Fed said consumer spending increased in nearly all districts while the manufacturing sector remained mixed, under continued pressure from the strong dollar, low commodity prices and weak global demand.
Traders also said the market was on edge because of a mass shooting in San Bernardino, California, during the afternoon trading session.
Art Hogan of Wunderlich Securities said the market was nervous about the shootings after the coordinated terrorist attacks in Paris last month. There was no evidence the California shootings were terror-related.
"It's hard to know but even if it's not, it sure feels like it," Hogan said. "First it's Paris, then it could be on our shores."
Stocks turned lower and extended losses after 2 p.m., ET, as oil broke $40 a barrel and the Beige Book raised concerns for some traders that economic growth was still sluggish.
Dow Jones industrial average intraday performance
WTI crude oil January '16 intraday performance
The S&P 500 closed down 1.1 percent with energy falling 3.1 percent to lead all 10 sectors lower. The Energy SPDR (XLE) had its worst day since Sept. 28.
The Dow Jones industrial average ended nearly 160 points lower, closing in negative territory for the year after ending higher for 2015 on Tuesday.
The Nasdaq composite closed about 0.6 percent lower after earlier trying to hold slight gains.
"People squaring up their positions and taking off some risk ahead of ECB and jobs Friday," said Jeremy Klein, chief market strategist at FBN Securities. He noted some traders could be concerned the European Central Bank's Thursday decision on monetary policy might not be as stimulative as the market has priced in.
Treasury yields held earlier highs amid the afternoon news, with the at 0.94 percent and the 10-year yield at 2.18 percent in the close.
"I just think (the Beige Book) another nail in the coffin in terms of a rate hike this month," said Peter Cardillo, chief market economist at First Standard Financial. He said most of the selling was due to oil.
U.S. crude settled down 4.6 percent at $39.94 a barrel, below $40 for the first time in three months. Trade in oil was volatile as investors reacted to headlines on production ahead of the OPEC meeting later this week.
Weekly crude oil inventories showed an unexpected rise of 1.2 million barrels.
"Crude is such a big part of this (selling) right now," said JJ Kinahan, chief strategist at TD Ameritrade.
The Dow transports closed down 2.1 percent to below the index's 50-day moving average, with Kansas City Southern leading decliners.
Earlier, the major averages traded mixed, while the U.S. dollar index spiked to hit its highest level since April 2003 after Yellen said in prepared remarks that the U.S. economy has come a long way, but the Fed's decision on rates could still be swayed by data before its December meeting.
Yellen also testifies Thursday morning before the Congressional Joint Economic Committee.
"I think people are realizing that a quarter-point rise in the federal funds rate isn't the end of the world," said Bryce Doty, senior fixed income portfolio manager at Sit Investment Associates.
The U.S. dollar index dipped below 100 after earlier touching a high of 100.51, topping the 100.39 high from March to briefly trade at levels not seen in more than 12 years.
The index held near 100.00 in late trade, with the euro recovering the $1.06 level, while the yen traded near 123.24 yen against the dollar.
Gold settled down $9.70 at $1,053.80 an ounce, off session lows of $1,049.40 an ounce, its lowest level since Oct. 2009.
"The Fed has a sort of an unspoken agreement that they're going to give a consistent message going into (the Fed meeting," said Sharon Stark, managing director and fixed income strategist at D.A. Davidson.
"The employment report on Friday is going to be important. If it's weaker than expected then there may be questions about maybe the Fed won't hike, but I don't think (that would derail the Fed) given the strength of the ADP report," she said.
The Federal Open Market Committee meets Dec. 15 to 16 and could raise short-term interest rates for the first time in nearly a decade.
"Given past rhetoric, the fading of financial market instability and Chinese concerns, and generally robust U.S. domestic data, her likely goal will be to elevate the market's expectation for a December rate hike from just above 70 percent to close to 90 percent. In this way, the Fed will manage to avoid too much disruption on the Dec. 16 decision day," RBC Global Asset Management Chief Economist Eric Lascelles said in a note.
Investors are also looking ahead to the European Central Bank's decision on monetary policy due Thursday morning ET.
"It's a lot to talk about," said Maris Ogg, president at Tower Bridge Advisors. "I don't think any of it's terribly important. We probably know what's going to happen."
Markets generally expect further easing from the ECB. The probability for a December hike in the United States has held steady above 70 percent, according to CME's FedWatch tool.
San Francisco Fed President John Williams said in a Reuters report Wednesday it will take a few years to get short-term U.S. interest rates, now near zero, back up to the "new normal" of about 3.5 percent,
Ahead of Friday's November jobs report, ADP data showed November private payrolls topped expectations at 217,000.
Revised third-quarter productivity rose 2.2 percent, while unit labor costs rose 1.8 percent.
U.S. stocks closed near session highs Tuesday, the first trading day of December, shaking off intraday pressure from the weak manufacturing data.
Over the last 10 years, the was up 70 percent of the time in December with an average return of 1.27 percent, according to analysis using Kensho.
In corporate news, Yahoo closed up 5.75 percent after the Wall Street Journal reported the company's board will discuss selling the flagship internet business, how to maximize the value of its stake in Alibaba, and consider the future of CEO Marissa Mayer at meetings this week.
Earlier, the Nasdaq composite attempted to hold slight gains as shares of Apple struggled for gains and Qualcomm traded higher. The Nasdaq 100 closed lower after briefly trading above its closing high of 4,719.06.
Qualcomm closed about 5.2 percent higher after briefly jumping more than 8 percent in opening trade after news the chipmaker entered a 3G/4G .
An SEC filing showed billionaire investor David Tepper's Appaloosa Management has taken a 9.25 percent stake in TerraForm Power, and is considering potential claims over its relationship with parent SunEdison.
The closed down 23.12 points, or 1.10 percent, at 2,079.51, with energy leading all 10 sectors lower.
The Nasdaq composite closed down 33.08 points, or 0.64 percent, at 5,123.22.
The CBOE Volatility Index (VIX), widely considered the best gauge of fear in the market, held near 16.
About four stocks declined for every advancer on the New York Stock Exchange, with an exchange volume of 954 million and a composite volume of 3.9 billion in the close.
Disclosure: CNBC's parent NBCUniversal has a minority stake in Kensho
—CNBC's Patti Domm and Peter Schacknow contributed to this report.
On tap this week:
7:30 a.m.: Challenger Job-Cut Report
7:45 a.m.: ECB rate decision
8:30 a.m.: ECB President Mario Draghi news briefing
8:30 a.m.: Cleveland Fed President Loretta Mester
8:30 a.m.: Initial claims
9:45 a.m.: Services PMI
10:00 a.m.: Fed Chair Yellen at Joint Economic Committee on economic outlook
10:00 a.m.: ISM nonmanufacturing
10:00 a.m.: Factory orders
1:10 p.m.: Fed Vice Chair Stanley Fischer on financial stability and shadow banks
OPEC meets in Vienna
8:30 a.m.: Employment report
8:30 a.m.: International trade
10:15 a.m.: Philadelphia Fed President Patrick Harker welcoming remarks at policy forum
11:45 a.m.: ECB President Draghi at Economic Club of NY
3:45 p.m.: St. Louis Fed President James Bullard on policy challenges
4:10 p.m.: Minneapolis Fed President Narayana Kocherlakota on policy renormalization
*Planner subject to change.
More From CNBC.com: