About half say that advisers are expensive, and more than half would like the cost of financial advice to be more closely tied to performance. (Read more: Ultra-Rich Spend Less on Bling )
That doesn't mean, however, that multimillionaires don't like the advice they do get. The survey found that nearly three quarters of them are "satisfied" with their adviser, with only 29 percent of them saying they could do a better job of investing.
And half would follow their adviser if the adviser changed firms.
Still, the report said that investors want better communication from their financial advisers. Less than 25 percent of them rated the blogs written by their advisers are "excellent" and only half said the meetings with their advisers are "excellent." (Read more: Affluent Plan to Spend Less If Obama Wins )
"No longer can advisers talk about hot investments, " the report said. "They must be able to respond to difficult and probing questions about issues and investments investors may have found on their own. Investors will expect holistic advice with responses that fulfill their concerns about taxes, financial planning and numerous other issues."
-By CNBC's Robert Frank
Follow Robert Frank on Twitter: @robtfrank