Fed Chair Janet Yellen, during a briefing, explained the central bank could still hike in October, but some forecasters see December as more likely. "In light of the heightened uncertainties abroad and the slightly softer expected path for inflation, the committee judged it appropriate to wait for more evidence, including some further improvement in the labor market, to bolster its confidence that inflation will rise to 2 percent in the medium term," Yellen said.
Read MoreFed leaves rates unchanged
The Fed also pared back its growth forecast for next year, and two of the 17 members moved their forecast for the first rate hike to 2016. There are now three members expecting a 2016 hike and a fourth now targeting 2017, with the rest expecting a first hike this year.
"It definitely makes it more uncertain. The beauty of an interest rate increase at this time would have been a removal of uncertainty and a removal of uncertainty would have increased confidence," said Don Townswick, director of equities at Conning. He has been expecting the first hike in December. "I think there will be more volatility in the market between now and December than if they had raised now."
Jack Ablin, CIO of BMO Private Bank, said the stock market's response was "buy the rumor, sell the news," and he remains cautious on equities.
Read MoreFed holds off, markets now betting on hike in 2016
"It doesn't change my view on the market. I think it's still risky. I think this erodes some Federal Reserve credibility. They're not data dependent. They're market dependent," he said. "They're not at 30,000 feet. They're at sea level."
Ablin echoed others who were disturbed that the Fed pinned the blame on global events, like China, rather than merely relying on its own readings on its dual mandate on labor and inflation.
While some market participants were concerned about the Fed's focus on foreign factors, CRT's chief Treasury strategist, David Ader, said the central bank did highlight concerns about its two mandates—Labor and inflation—when making its announcement.
Read MoreHere's what changed in the new Fed statement
In the bond market, yields immediately fell with investors selling the two-year sector where rates had risen on speculation of a rate hike. The two-year was yielding 0.67 percent late Thursday after trading above 0.80 percent ahead of the Fed news.
"I think you saw something happening today in that we had a relief rally in stocks. Then we pulled back lower on the day," Ader said. "We're now going to be climbing a wall of worry, looking at each piece of data."
Ader also said the Fed lost credibility in the eyes of the market Thursday. "The Fed is enhancing uncertainty by persisting with this view, which is not a good one," he said.
The S&P 500 fell 5 to 1,990 on Thursday and is now about 6 percent off its highs. But strategists say the market is vulnerable due to seasonality and the fact that Congress is making threats to shut down the government this fall.
Read MoreLessons from Lehman: Spotting the next crisis
There is also concern about earnings growth. Barclays on Thursday issued a note saying it expects no growth for S&P 500 companies' earnings in 2015.
"We're getting negative projections again," said Ablin. He added, however, that companies could beat like they did last quarter. "We're not looking at robust earnings growth."
There is no economic data Friday but stocks could react to options expirations.