The IRS already audits the wealthy at a relatively high rate.
People who earn more than $1 million a year are more than 12 times more likely to be audited than people who earn $200,000 or less. About one of every eight tax filers making $1 million or more were audited in 2011 – double the rate of 2009.
But a new report says the IRS should be targeting the wealthy even more.
A report from the non-partisan Government Accounting Office says that the IRS could get more bang for its audit buck by targeting high earners.
(Read more: What It Would Cost You to Own Downton Abbey)
It said that the net tax gap – the difference between taxes owed and taxes paid on time – was about $385 billion in 2006 (the period measured). It said this gap could be lowered by at least $1 billion by spending $124 million more on audits of high earners.
That may sound like a big expense. But it's less than 8 percent of the $1.6 billion the IRS spent on examinations over two years.
Pure math, of course, favors targeting those with higher incomes. The GAO said that audits of filers with incomes over $1 million generated $47.20 for every dollar they cost the IRS – even though those exams are more complicated. Examinations of lower-income earners' returns generated between $5.40 to $7.40 for every dollar.
(Read more: Why the Wealthy Are Relieved About the 'Cliff' Deal)
"The results of our analyses suggest that there is potential for IRS to increase the direct revenue yield of selected enforcement programs by hundreds of millions of dollars per year without significant (if any) adverse effect on the indirect effect that examinations have on revenues," the report said.
In short, it said the IRS be like Willie Sutton and go where the money is.
The IRS, which has already set up a special audit team to target the wealthy, may or may not act on the GAO recommendation. In a letter to the GAO, Steven T. Miller, an assistant commissioner at the IRS, wrote that the "IRS is committed to the optimal allocation of our enforcement resources; that is why we select workload strategically."
Correction: An earlier version of this story misidentified the author of the IRS letter to the GAO as James White. White, director of tax issues at the GAO, was the recipient of the letter.