Wealthy more worried about being seen as wealthy

US becoming more like rest of world
US becoming more like rest of world

Is success being vilified in America? The successful seem to think so.

A new poll from American Express Publishing and Harrison Group finds that 1 percenters no longer like to be seen as such.

One-third of members of the group said they "like it when others recognize me as wealthy." Though that number (taken in the fourth quarter of 2013) may sound high, it's down from 40 percent a year earlier. And it's far below the 53 percent who agreed with the statement in 2010.

(Read more: The poor should stop whining, says luxury CEO )

An Occupy Los Angeles protest in 2011.
Getty Images

Fully 28 percent say they worry about "being scorned for being in the top part of the economy," versus 24 percent who were concerned about that in the first quarter of 2013.

(Read more: How to become a millionaire? The answer's changed )

The recent and sudden move toward stealth wealth could have an impact on the elite economy. Thirty-eight percent of respondents agree that they "feel guilty purchasing luxury goods and services"—double the percentage who said the same thing a year ago.

It's easy to make too much of these numbers.

Nicole Miller's CEO comments on wealth 'nonsensical'
Nicole Miller's CEO comments on wealth 'nonsensical'

As venture capitalist Tom Perkins has discovered, the idea that the wealthy are being persecuted is exaggerated—among the general population and the wealthy themselves.

(Read more: The top three myths about income inequality)

As we see from recent millionaire-outlook studies and luxury sales, some may be sensitive to the political climate and talk of inequality. But others keep living and spending as they always have.

One thing is clear, however. People who have worked hard to achieve a higher level of financial success are realizing that it is a double-edged sword: It is rewarding but also makes them a public target.

—By CNBC's Robert Frank. Follow him on Twitter @robtfrank.