The major averages came off session lows after the ISM manufacturing PMI for May topped expectations with a 51.3 print. Consensus estimates from Reuters expected a read of 50.5, barely in expansion territory and off slightly from the previous month's report of 50.8.
Klein noted "there was definitely a real fear we were going to move below 50" and the data reinforced expectations for a Fed hike as soon as this summer.
Traders also attributed some of the late-session gains to first-of-month inflows.
The Dow Jones industrial average closed up about 2.5 points after earlier falling 122 points. Apple and IBM were the greatest contributors to declines.
Read More'Act now or risk another deep downturn,' OECD warns policymakers
"I think what's causing some of the sell-off (this morning was) questions about the strength of the U.S. economy and the global economy," said Chris Gaffney, president, EverBank World Markets.
The Fed's Beige Book released Wednesday afternoon said there was modest economic growth since the last report. The Beige Book also said tight labor markets were pushing up wages.
The major economic news for the week is the jobs report, scheduled for release Friday morning. Fed Chair Janet Yellen is set to speak on June 6. She is also scheduled to testify to the Senate Banking Committee on June 21, according to an announcement on the committee website.
Treasury yields edged off lows, with the 2-year yield higher around 0.89 percent and the 10-year yield steady near 1.84 percent. Earlier, both yields hit their lowest in two weeks.
The yen strengthened against the dollar, trading near 109.6 yen. The U.S. dollar index fell half a percent, while the euro was last near $1.119.
The "Japanese prime minister did as expected and delayed the consumption tax increase, but in doing so really raised concerns about global growth," Gaffney said.
"The data shows the U.S. economy is really chugging along. It's slow growth, low inflation," he said.
In other economic news, the final read for May on Markit's manufacturing PMI was 50.7. The flash read was 50.5, down from 50.8 in April.
Read MoreData dump to kick off June
May auto sales came in at a 17.45 million seasonally adjusted rate, according to Autodata. That was slightly above expectations for 17.3 million, according to economists polled by Reuters.
April construction spending fell 1.8 percent.
As widely expected, Japanese Prime Minister Shinzo Abe announced overnight a two-and-a-half year delay in a scheduled sales tax increase, putting plans for fiscal reforms on the back burner due to growing signs of economic weakness.
Asian stocks closed lower, with the Nikkei 225 down 1.6 percent and the Shanghai composite off 0.11 percent.
In China, the Markit Caixin manufacturing PMI fell to 49.2 in May, below expectations and down from 49.4 in April, and below the neutral 50.0 value for the 15th-straight month.
The official manufacturing PMI was unchanged from the prior month in May at 50.1. The official services PMI edged lower to 53.1 in May from 53.5 in April.