"The time is about right," said Art Cashin, director of floor operations at UBS. "I'm just a little concerned about the volatility of the move."
Cashin said the volatility could continue due to Friday's quarterly expiration of options and futures. "It's going to be a huge options expiration. People are talking near record volume, and it may be eating up some of that (today)," he said.
(Read more: El-Erian: Fed taper begins—what happens next?)
The Dow, at first lower on the Fed news, exploded higher, gaining 292 points to 16,167. Traders said while the Fed took away some of its easy policy with one hand, it managed to soothe the market with reassurances it would leave short-term rates near zero for a very long time.
The S&P 500 also closed at a record, up 29 points at 1,810, and the Nasdaq reached a new 13-year high.
"I think this is the response of the market saying this is exactly what we needed. We got the right amount of taper, and we got it at the right time. I was like most people. I thought the odds were less than 50-50 that it was going to happen. The market's reaction tells me this is exactly what the market wanted," Frederick said.
He said a strong December could also lead to a positive January. He said in 14 of the last 18 times when December was an up month, so was the following January. Last year, the market was up 0.7 percent in December and more than 4 percent in January.
Frederick said one of the big issues facing the market was the federal budget, and the prospect of another round of congressional battling. But the Senate followed the House and voted Wednesday to adopt a two-year budget, removing a major hurdle markets would have faced early next year.
While the stock market responded to the idea that the Fed sees an improving economy, the bond market remained relatively tame. The 10-year yield was at 2.89 percent in late trading, off its highs of the day.
David Ader, CRT Capital chief Treasury strategist, was part of the minority in the market that expected a Fed taper. But he also expects the bond market to begin to price in more tapering as the January Fed meeting draws near with the prospect of further Fed action. That would mean higher yields, which have the potential to spook stocks.
But rates did not move much after Wednesday's Fed news. "I think it's incredibly significant that it came when we had tapering. The curve is doing very little on this."
Ader said the bond market was showing the first tapering event was priced in.
(Read more: Fed taper positive sign for economy, bad for bonds)
"Rates are going to rise again because we have housing starts blow through a million annualized pace, and we have industrial production up the biggest amount since 2008 last month," said Jim Paulsen, chief investment strategist at Wells Capital.
"You have retail sales up, while they were revised up the previous month and you have jobs up, and they were revised up the previous month. Stuff's on fire compared to where it was ... and I think that's why rates go up."
Paulsen said there's a risk now that inflation, currently nonexistent, could rear up. "I still think the real event isn't this one. The real event is when there's more of a panic surrounding Fed tapering and that will be if there's inflationary tones about it," he said. "In the period now, it's pretty good for stocks, pretty bad for bonds."
What to watch
There are a few economic reports of interest Thursday, including weekly jobless claims at 8:30 a.m. ET and existing home sales at 10 a.m. The Philadelphia Fed survey is also released at 10 a.m., and leading indicators are issued at 10 a.m.
Earnings are expected from Carnival, Darden and Rite Aid ahead of the opening bell. Nike, Red Hat and Tibco report after the close.