With the Federal Reserve's first rate hike out of the way and little economic news in the last two holiday-shortened trading weeks, stock market action has been tightly correlated to moves in oil and other commodities.
"We did somewhat track the movements in WTI earlier. ... Other than that the tone has been set by very limited overseas action. (It's an) extremely light volume day," Larson said.
"For the limited people that are in, it could be closing up positions for year-end," he said.
Trade volume this week was among the lowest of the year, ahead of Friday's New Year's Day holiday when markets are closed. However, volume Thursday topped 5 billion across exchanges, surpassing last New Year's Eve volume of 4.58 billion shares.
"Probably more urgency to sell than to buy. That's why we're down," said James Meyer, chief investment officer at Tower Bridge Advisors.
"It's a day of portfolio adjustments. I think you're going to find some people taking some money off the table and switching to more conservative (assets). So bonds are up and stocks are down," he said.
Read MoreLots of fireworks but no grand finale for stocks
U.S. crude oil futures settled up 44 cents, or 1.2 percent, at $37.04 a barrel. However, WTI lost more than 30 percent for 2015, for its first two-year losing streak since the late 1990s.
Brent settled at $37.28 a barrel for its third-straight year of decline.
Oil held higher in intraday trade but remained near seven-year lows. The U.S. oil drilling rig count showed a decline of 2, according to Baker Hughes.
Brent turned positive in intraday trade as a fierce storm in the North Sea forced oil firms to evacuate platforms and shut down production Thursday.
Also on Thursday, Iran President Hassan Rouhani ordered his defense minister to expand Iran's missile program, in response to a U.S. threat to impose sanctions over a ballistic missile test Iran carried out in October.
"I think this is going to be a short-lived bounce in energy prices," said Robert Pavlik, chief market strategist at Boston Private Wealth.
U.S. crude oil fell more than 3 percent Wednesday to settle below $37 a barrel, contributing to the Dow Jones industrial average's decline of about 117 points Wednesday.
Natural gas traded about 5 percent higher. Inventories showed a decline of 58 billion cubic feet in the week ended Dec. 25, according to StreetAccount.
In a light day of economic news, the Chicago Purchasing Managers' Index (PMI) came in at 42.9 for December, down from 48.7 in November.
"This really undermines those expectations for the new year to be that year of improvement," said Lindsey Piegza, chief economist at Stifel.
"This is not the type of data the Fed wants to see after the monumental change in December. We may see more downside in the economy," she said.
Weekly jobless claims came in at 287,000, up from 267,000 the week before.
Treasury yields held lower, with the 2-year yield around 1.06 percent and the 10-year yield near 2.27 percent in the close.
The U.S. dollar traded higher against major world currencies, with the euro falling below $1.09.