Expectations for the effectiveness of QE in keeping interest rates low are also changing. The percent of respondents who see QE lowering mortgage rates dropped to 42 percent, from 46 percent in April. This is against a backdrop of rising rates, where 30-year mortgage rates have risen above four percent for the first time in a year.
The number who see the Fed's QE helping keep bond yields low also dropped—to 46 percent from 54 percent in April. But an almost equal amount, 44 percent, don't believe the monthly purchases of mortgage and Treasury securities are helping drive down bond yields. Since early May, 10-year Treasury yields have risen in volatile trading from a low of 1.62 percent on May 2 to as high as 2.3 percent. On Monday, the 10-year was yielding 2.17 percent.
As for the stock market, 70 percent said that QE could help drive stock prices higher, down from 83 percent in April. But their year-end target on the S&P 500 increased from an average 1612 for Dec. 31, to 1655. They also forecast an average 1722 by June 30, 2014. The S&P 500 closed Monday at 1639. As rates have risen since early May, the S&P 500 is up 3.6 percent, but it has seen its most volatile trading of the year.
"The Fed will likely look to the next meeting to attempt to 'calm' markets," wrote Kevin Giddis, who heads fixed income at Raymond James, in response on the survey.
David Kotok of Cumberland Advisors wrote that the Fed's communication policy has "been fuzzy and caused an added volatility."
(Read More: Fed Needs to Choreograph Moves: Pro)
—By CNBC's Patti Domm. CNBC Senior Economic Correspondent Steve Liesman created the survey and reports on it ahead of Fed meetings.