The trouble is, the rich as a group don't pay less than the middle class.
Buffett talks about half of the country's 400 highest earners paying less than 20 percent. And he says a few "actually paid nothing."
True enough. And that's an outrage to someone who pays the 25 percent or 35 percent official tax rate.
But what Buffett doesn't mention is that, on average, very few Americans actually pay the official tax rate, due to deductions and credits. On average, everyone pays less. When it comes to what Americans actually pay, the high earners and "wealthy" still pay more than twice the rate as the middle class.
The latest data from the IRS shows that people who make $1 million or more had an average tax rate of 20.4 percent in 2010. (Read more: The Millionaires Who Pay the Highest Tax Rate).
Tax filers who earned $30,000 to $50,000 paid an average rate of 4.8 percent, while those who made between $50,000 and $100,000 paid 7.7 percent. Those making under $30,000 had a negative effective rate, meaning they paid no federal income taxes after deductions and credits.
Put another way, millionaires pay a rate that's nearly three to four times the rate paid by the middle class.
Even the super-rich, or those who make more than $10 million and rely more on investment income taxed at 15 percent, have an effective rate of 20.7 percent.
"Very few people actually pay 35 percent," said Roberton Williams Senior Fellow of the non-partisan Tax Policy Center.
Yes, there are some super-earners who pay less than some in the middle class. But they are the exception not the rule. According to data from the Tax Policy Center, there would be 603,000 Americans in 2015 who make $1 million or more. Only about a third of them would be subject to the Buffett Rule, which means that two thirds already pay more than 30 percent.
Buffett's op-end implies that our tax system is regressive, that the rich pay less because of their power and lobbyist friends. The fact is, our tax system is progressive – the wealthy pay a higher rate than the rest of America. (Read more: What Capping Tax Deductions Would Cost the Rich)
This is not an argument for or against the Buffett Rule. There is surely symbolic value is making sure that everyone making $1 million pays more than the rest of America.
But it is largely that – symbolic.
-By CNBC's Robert Frank
Follow Robert Frank on Twitter: @robtfrank