The second-best performing sector in the S&P 500, financials advanced with the continued rally in yields as the sector is expected to benefit from higher interest rates. The S&P Regional Banking ETF (KRE) gained 1.08 percent. The SPDR S&P Bank ETF (KBE) hit its highest level since October 2008.
PNC Financial, Capital One, BB&T, Zions Bancorp, People's United Financial and KeyCorp all touched multi-year highs.
"The logic that's infused here is encouraging," said Art Hogan, chief market strategist at Wunderlich Securities. He noted how better economic data supported higher yields and gains in financials, while the rise in oil prices helped the energy sector.
Energy reversed early gains to close slightly lower despite crude settling up more than 3 percent to above $60 a barrel. The U.S. Energy Information Administration raised crude oil production forecasts for 2015 while lowering those for 2016. Caterpillar was the second-best performing blue chip.
Apple closed 0.3 percent lower to weigh on the Nasdaq, which briefly fell below its 50-day moving average.
The Job Openings and Labor Turnover Survey showed 5.376 million job openings in April, the highest since December 2000.
Peter Cardillo, chief market economist at Rockwell Global Capital, said the jobs data confirms optimism in the labor market. "This is one more reason to believe yields are likely to move higher here and pressure equities."
Traders also noted investors selling government debt to hedge and make room for corporate bond issuance.
Read More What 'tantrum'? Investors still love bond funds
Wholesale inventories showed an increase of 0.4 percent in April, above expectations of a 0.2 percent rise.
U.S. small business confidence increased to a five-month high in May of 98.3, the best read since December, according to the National Federation of Independent Business.
Stocks struggled to hold slight gains in a mixed trading session Tuesday.
"We're a little bit still in 'good news may not necessarily be good for the market because it might bring the Fed into the picture,'" said Ben Pace, chief investment officer at HPM Partners. "I think in general the market's been handling the bump up in yields pretty well."
The S&P 500 held above a support level of 2,070 on Tuesday, after the index closed below its 100-day moving average on Monday.
"Two of our market internal measures are flashing green, suggesting the SPX will see an oversold bounce in the days ahead," BTIG Chief Techinical Strategist Katie Stockton said in a morning note. "Short-term oversold conditions are widespread."
Read MoreMarket's 'memory' reveals next move for oil
The Dow transports remained in correction territory, closing 0.3 percent lower after attempting to reverse Monday's 2 percent decline. Airlines plunged, with Southwest Airlines and Alaska Air falling 4 and 3 percent, respectively.
Southwest reported an approximately 6 percent decline in May passenger revenue per available seat mile.