Most of the wealthy have a binary view of taxes: They either want less, or more.
But billionaire Ronald Perelman, who's listed by Bloomberg as the 28th richest man in America with $13.9 billion, has a more nuanced view. He believes income taxes should be high for the wealthy.
"I think wealthy people should be taxed more," he said. "I think if you can afford it, that's where the money's gotta come, and they've gotta raise the money. But I don't think it should be a tax that chills the desire or the interest or the financial reward for taking risk."
(Read More: Perelman's $100 Million Question: Can Entrepreneurship Be Taught?)
But (and here's where it gets interesting) he believes the estate tax should be abolished.
"I could get to the next thing that really bothers me, which is estate taxes," he said. "Double taxation. You're taxed on the creation, and you're taxed on the distribution."
Perelman's take gets some sympathy in Washington. While income taxes went up in the "fiscal cliff" deal for top earners, the estate tax was set at a more generous level. The tax was scheduled to go up to 55 percent on estates of more than $1 million. Instead, the tax was set at 40 percent for individual estates valued at $5.25 million.
(Read More: Corporate Climate Good for Earnings, Not Jobs: Perelman)
It seems unlikely that the estate tax will be abolished—or raised—anytime soon. Obama has proposed in his budget to hike the tax to 45 percent and lower the exemption to $3.50 million from $5.25 million.
But experts say any further hike in the estate tax is unlikely. So, however, is a reduction or elimination of the tax. A recent study in American Politics Research found that wealthier members of Congress were more likely to vote to repeal the estate tax.
Obama's proposal, however, does mean that the so-called permanent fix for the estate tax may not look so permanent.
-By CNBC's Robert Frank. Follow him on Twitter: @robtfrank