On Wednesday, U.S. stocks closed higher by nearly 1.5 percent or more after the Federal Reserve's October meeting minutes showed most members of the Federal Open Market Committee thought a December hike would be appropriate.
"One of the things that's been missing in the market has been confidence in the Fed and the Fed delivered the message the market needed to hear (yesterday in the minutes)," said Mike Baele, managing director, The Private Client Reserve, U.S. Bank.
Read MoreFed speakers up next after minutes confirm hiking message
Federal Reserve speakers Thursday continued to emphasize a gradual pace of tightening.
Atlanta Fed President Dennis Lockhart said he is "comfortable with moving off zero soon." He told reporters after the speech the Fed may need to change its criteria for progress on inflation after it starts raising rates.
Earlier, Cleveland Fed President Loretta Mester, said on CNBC's "Squawk Box" that policymakers are not boxed in on rates and that she doesn't think the Fed is behind the curve.
Fed Vice Chairman Stanley Fischer is scheduled to discuss emerging Asia at the San Francisco Fed at 4:45 p.m. ET.
Read MoreHere we go again? More confusion on Fed and rates
On the data front, initial jobless claims came in at 271,000. The Philadelphia Fed Manufacturing survey's diffusion index for current activity came in for November at 1.9, its first positive reading in three months.
U.S. October leading indicators rose 0.6 percent, slightly more than expected.
"It's not the best of economic environments, but it's not getting worse and that's good news,' McCain said.
The 10-year Treasury note yield edged lower to near 2.24 percent, while the 2-year yield was up slightly at 0.89 percent.
The U.S. dollar held more than half a percent lower against major world currencies, with the euro at $1.073 and the yen at 122.86 yen against the dollar.
Overnight, the Bank of Japan kept monetary policy unchanged. The People's Bank of China said it will lower lending rates for loans under the standing lending facility. The European Central Bank meeting minutes showed the central bank considered adding more stimulus.
European stocks closed higher, with the DAX outperforming with gains of more than 1 percent. Asian equities also ended higher following gains in the U.S. market.
"Our market internal measures support near-term upside follow-through, particularly given the positive seasonal influences that are upon us," BTIG Chief Technical Strategist Katie Stockton said in a morning note. "Up-volume overwhelmed down-volume by more than 9-to-1 yesterday after a collection of contrarian extremes were registered last Friday."
Traders also eyed Square and Match, which began trading Thursday.
Square closed up 45.2 percent. The stock briefly gained more than 50 percent in its trading debut after opening at $11.20 a share. The stock priced below the expected range at $9 a share.
Tinder parent Match jumped 22.8 percent in its first day of trade. The stock priced at the low end of the expected range at $12 per share.
Read MorePisani: Strong Square showing is a weight lifted
"There's consternation in the market that demand for these large tech IPOs may not be as strong as hoped for," Self said.
Investors also watched geopolitical developments following the Paris terror attacks that killed more than 100 people.
See CNBC's full Paris coverage here
A French prosecutor confirmed Thursday that Abdelhamid Abaaoud, the suspected Belgian mastermind behind the attacks, was among the dead identified in Wednesday's raid by French forces in the northern Paris district of Saint-Denis.
"I think people will have a third eye on terrorist-related events around the world," Larson said.