Earlier in the day, the ADP jobs report reported better-than-expected numbers. The report is viewed as an indicator of the all-important non-farm payrolls due Thursday, estimated to rise by 212,000.
"There is a catch-22 to the labor market," said Peter Boockvar, chief market analyst at The Lindsey Group. Increasing jobs indicates the economy is improving, but greater labor costs will likely lessen profit margins, he said. "You have to be careful on what you wish for."
The market has focused on low interest rates and low wage levels, but both of those may be reversing, Boockvar said. "That's what mucks up the Fed's plans—a higher rate of inflation than the Fed thinks."
Traders also looked to the bond market as Treasury yields on benchmark 10-year notes rose to the high end of the curve at 2.62 percent.
Morning market gains diminished when Fed Chair Janet Yellen said that monetary policy has limitations in addressing financial stability risks.
"I think the fact that she's seen an increase in risk-taking is probably causing the market to give up some gains," said Peter Cardillo, chief market economist at Rockwell Global Capital.
Investors may not fully appreciate the risk of future losses, Yellen warned on Wednesday, citing falling corporate bond spreads and volatility indicators as possible indicators. She said there was no need for a change in monetary policy focus in the meantime as it is not the best financial stability tool.
Read MoreYellen: Seeing pockets of increased risk-taking
"That said, I do see pockets of increased risk-taking across the financial system, and an acceleration or broadening of these concerns could necessitate a more robust macroprudential approach," Yellen said in prepared remarks for a speech at the International Monetary Fund.
"I think it's important that Janet Yellen endorses this not as a solution but as a guard and a guide to financial stability going forward instead of simply raising interest rates," said Bill Gross, founder and managing director of Pimco.
"If Fed funds going forward stop at 2 percent instead of 4 percent, which is historical, then stock prices at 17,000 and high yield spreads at 350 basis points over Treasurys are attractive and are less bubbly than some would imagine," Gross said on CNBC's "Street Signs."
May factory orders reported a greater decline than expected on Wednesday morning.
"I don't think it's inconsistent with an improving economy," said Art Hogan, chief market strategist at Wunderlich Securities. "The decrease is basically in line with expectations."
Read MoreThis is the hottest commodity of the future
The Dow Jones Industrial Average was within 35 points of 17,000 early Wednesday afternoon after missing the high by less than 2 points on Tuesday.
"17,000 is a great round number," Hogan said. "We should be there already."
Poor performance on new additions to the Dow have contributed to the index's sluggish rise compared to other indices since it crossed 16,000 last November.
Constellation Brands was the S&P 500's biggest percentage gainer, with shares up 2.9 percent to $90.91 after reporting earnings and revenue that beat expectations.
Shares of camera maker GoPro traded lower on Wednesday after days of consecutive gains that boosted prices to double the initial offering of $24 a share.
Google shares dipped slightly after announcing the purchase of streaming music service Songzalate Tuesday.
Rackspace Hosting sharesjumped nearly 8 percent to $36.44 after TechCrunch reported the cloud service provider may take itself private and was in talks with a private equity firm to fund the deal.
Shutterfly shares jumped nearly 14 percent to $49.49 after Bloomberg reported that the company was working with boutique investment bank Qatalyst Partners to find buyers for itself.
JPMorgan Chase shareswere down about 1 percent after Chief Executive Officer Jamie Dimon said he had been diagnosed with early stage throat cancer but would remain actively involved in the largest U.S. bank's business while in treatment.
Adding to a recent string of transactions that has lifted stocks in the healthcare sector, Roche said it would pay up to $1.725 billion to buy Seragon Pharmaceuticals, a privately-held U.S. biotech company that researches breast cancer treatments.