"It could come under a lot of pressure especially as the contango in the market is telling everyone to store all the barrels they can. This is indicative of supplies continuing to build," said Andrew Lipow, president of Lipow Oil Associates. Contango is when futures contracts get progressively higher priced into future months, and it is a bearish sign for oil prices.
Energy stocks led to the decline in equities Thursday, as the sector fell 1.3 percent. The worst-performing sector was health care, dragged down by Pfizer, UnitedHealth and other insurers. The S&P 500 was down 2 at 2,081.
The S&P energy sector is, however, up more than 2 percent for the week, as WTI is 2.8 percent higher week to date after a steep decline last week.
Read MoreETF: Play an emerging markets comeback for cheap
"Energy stocks have bounced back here, but it's been their relative trend that's the weakness and their relative trend looks close to oil," said Ari Wald, technical strategist at Oppenheimer Asset Management. "I think if oil were to come under pressure, it would be energy stock underperformance you would see."
"This is all playing into what's going on in the dollar. The real close correlation is dollar strength and commodity weakness, and that's something longer term we think continues to play out. We think the dollar is in a secular advance. Ultimately, it reaches parity with the euro," said Wald.
Read MoreThese high-risk liquid alt strategies also offer high rewards
The U.S. currency which has been consolidating, and the dollar index Thursday lost a half percent in its biggest drop since late October. Some strategists said the greenback's move was a case of "sell on the news" after the Fed's minutes Wednesday reinforced the idea the central bank could raise rates in December if the economy does not fall apart.
The euro was higher at $1.07 to the dollar Thursday, but traders are watching Draghi's comments at the European Banking Congress on Friday morning, since talk of further easing could put pressure on the currency, particularly as the Fed moves to raise rates.
Read MoreHere we go again? More confusion on Fed and rates
Fed speakers are likely to continue to drive their message that they would like to hike rates, if the economy continues to improve. Bullard speaks at 9 a.m. EST on the economy and monetary policy, and Dudley is scheduled to make remarks at 11:15 a.m. ET on the economy and regional conditions at Hofstra University.
Deutsche Bank's chief U.S. economist, Joseph LaVorgna, said the Fed minutes Wednesday were convincing enough for him to change his view on the rate hike. He had expected a March rate rise but now expects it in December, as do many other economists.
Read MoreA sigh of relief in IPO land
"They were broadly consistent with the hawkish statement we got in October. They imparted a relatively high bar for a nonmove," said LaVorgna. "And the fact the markets are increasingly comfortable with December liftoff, the Fed will likely move off zero."
The yield on the policy-sensitive two year was higher late afternoon Thursday, at 0.89 percent, but the 10 year was slightly lower at 2.24 percent.
There are just a couple retail earnings Friday — Foot Locker and Abercrombie & Fitch, before the bell.