"The boost to stocks came directly correlated to what happened in oil, so it's hard to say it was anything but that," Janney's Luschini said.
"You had some hope of an inventory drawdown, but that didn't hold," said Art Hogan, chief market strategist at Wunderlich Securities. "You had a build in distillates and that sort of rolled over the market."
Read MoreSigns point to market trough: Bernstein strategist
The S&P 500 closed slightly lower, as materials weighed. The Dow and S&P extended their losing streak to four straight days.
Yellen delivered her remarks to Congress at 10 a.m. ET, in which she notes that, if the U.S. economy were to disappoint, the Fed would have to reconsider its rate hike path.
"Bottom line on Yellen's testimony, of the 3 possible options I laid out in my morning note, I believe the Janet Yellen testimony fell into Door Number 2, the non committal one. I say this because she acknowledged all the risks to the downside and positives on the upside to the outlook without leaning in any one direction, thus leaving us with a 'let's play it by ear', middle of the road message," Peter Boockvar, chief market analyst at The Lindsey Group, said in a note.
"I don't recall her ever saying that before," said Peter Cardillo, chief market economist at First Standard Financial.
Read MoreBlackRock: Negative rates 'new toy' around world
Yellen answered questions after delivering her remarks, saying "I don't think it will be necessary to cut rates but like I said monetary policy is not on a preset course."
"She's put on the perfect hedge," Sarhan said. "She doesn't want to lose credibility and say she was wrong."
Randy Warren, chief investment officer at Warren Financial Service, said "I think think she's capitulated quite a bit. ... The market has basically said that you're crazy if you think you're raising rates this year."
Dow futures traded sharply higher Wednesday, jumping more than 150 points as European equities rallied.
"The pivot point in action today is Deutsche Bank saying they were considering buying back bonds," Wunderlich's Hogan said.
Shares of the German bank gained 10.2 percent Wednesday, but remained about 35 percent lower for the year. Meanwhile, the pan-European STOXX 600 index closed 1.88 percent higher.
"[European banks] were never forced to clean up their balance sheets the way ours were, and that's coming back to bite them," said Maris Ogg, president at Tower Bridge Advisors.
Read More Cramer: Why Disney stock is down
Investors also kept an eye on U.S. oil prices, which seesawed Wednesday.
"There was talk that Iran was willing to work with the Saudis, and that calmed the market," First Standard's Cardillo said.
Prices briefly jumped more than 3.5 percent after the Energy Information Administration said that U.S. oil inventories fell 0.8 million barrels, before closing 49 cents lower, or 1.75 percent, at $27.45 a barrel.
The oil market has been maligned by oversupply concerns throughout the year, pushing U.S. crude down about 24 percent this year.
WTI in 2016
"WTI is flirting with with that January low," Sarhan said. "If that low gets taken out on a closing basis, you could see another leg lower for oil."
The benchmark 10-year yield traded at 1.70 percent, slightly below Tuesday's close.
Jon Adams, senior investment strategist at BMO Global Asset Management, said the recent rally in government bonds around the world is reflective of global recession fears. "Once the market realizes that monetary policy [around the world] is still fairly supportive, I think yields will go moderately higher."
Read More Carl Icahn nurses paper losses on energy bets
The dollar fell about 0.3 percent against a basket of currencies.
Earnings season continued Wednesday morning, with Time Warner and Humana, among others, reporting before the bell. Cisco Systems, Tesla Motors, Twitter and Whole Foods are scheduled to report after the bell.