Equities extended declines after Yellen said the "considerable period'' between the end of its quantitative easing program, known as QE, and the first rate increase from the Fed could be six months.
(Read more: Yellen indicates rate boost sooner than expected)
With QE forecast to wind down sometime near the end of the year, a six-month lag would move up the timetable for the Fed's first hike, which many market participants had been expecting in the second half of 2015.
"She certainly moved it up a little bit, and I don't think the market was expecting that at all because she is widely viewed as being more on the dovish side of the aisle than she is on the hawkish side,'' said Peter Kenny, CEO of Clearpool Group in New York. "That is not a particularly hawkish comment, but the fact of the matter is, it was not expected.''
The Fed also said it would cut its monthly purchases of U.S. Treasurys and mortgage-backed securities to $55 billion, from $65 billion.
The S&P 500 was within 1 percent of its record closing high, though economic bellwether FedEx hit a sour note in its outlook. Geopolitical concerns related to Ukraine also stayed in focus.
(Read more: Investors go to cash, sell stocks on Ukraine panic)
The Dow Jones industrial average fell 114.02 points or 0.70 percent, to end at 16,222.17. The S&P 500 slipped 11.48 points or 0.61 percent, to finish at 1,860.77. The Nasdaq composite dropped 25.71 points or 0.59 percent, to close at 4,307.60.
Equities had rallied to start the week, buoyed by easing geopolitical concerns, though trading volume has been light. The S&P 500 has climbed 1.7 percent over the previous two days, the best back-to-back performance for the benchmark index since early February.
Volume was light, with about 6 billion shares traded on U.S. exchanges, below the 6.7 billion average so far this month, according to data from BATS Global Markets.
Volume is expected to surge on Friday as options expiration takes place alongside multiple index rebalances. Credit Suisse estimates $14 billion in gross trading will stem from the S&P 500 index rebalance, with another $6 billion coming from rebalancing in other indexes.
Declining stocks outnumbered advancers on the New York Stock Exchange by a ratio of 3 to 1. On the Nasdaq, nearly two stocks fell for every one that rose.
A slew of economic data are expected Thursday including jobless claims, Philadelphia Fed survey, existing home sales and leading indicators. Nike is expected to post earnings after the closing bell.
—By Reuters. CNBC.com contributed to this story.