Your Money

Why IRS customer service is bad (and may get worse)

IRS customer service bad, may get worse
IRS customer service bad, may get worse

If you have a question about your taxes, here's a tip: Be prepared to be very patient.

The Internal Revenue Service answered a smaller share of taxpayer calls and kept taxpayers on hold longer last year than in other recent years, a new report finds. What's more, budget cuts could make it hard for taxpayers to get help this year as well.

Andrew Harrer | Bloomberg | Getty Images

The IRS could answer only 61 percent of the calls it received from taxpayers during the 2013 fiscal year, National Taxpayer Advocate Nina E. Olson said in a report released to Congress on Jan. 9.

That means that the rest of the calls—about 20 million—just didn't get through, the independent taxpayer advocate said.

(Read more: Identity thieves gear up to steal your tax refund)

Taxpayers who got help had to wait a long time for it. The taxpayer advocate said callers who got through were on hold for an average of nearly 18 minutes before talking to a customer service representative during the 2013 fiscal year, which ended Sept. 30.

The level of service is a sharp drop from a decade ago. In the 2004 fiscal year, the IRS answered nearly 9 in 10 calls, and the average wait time was less than three minutes, according to the taxpayer advocate, an independent voice in the IRS that works on behalf of taxpayers.

Olson blamed a lack of resources. The overall IRS budget has been cut every year since the 2010 fiscal year, her report said, and the amount allocated for training has been slashed significantly as part of that.

It's not likely that things will get better this year. The spending bill passed by the House and Senate last week includes another cut to the IRS budget, even as the tax man grapples with problems including the continued threat of identity theft tax fraud.

ID thieves out to steal your tax refund
ID thieves out to steal your tax refund

Given its limited resources, the IRS said that in 2014 it will stop preparing tax returns for people who need help, such as those who are elderly or disabled. The agency also plans to answer only "basic" tax law questions, and only during the normal filing season through April 15. Instead, it will direct people to the website and other automated sources of information.

"(I)t is a sad state of affairs when the government writes tax laws as complex as ours—and then is unable to answer any questions beyond 'basic' ones from baffled citizens who are doing their best to comply," Olson said in the report.

(Read more: Marriage may not be magic bullet for poverty)

In a statement in response to the taxpayer advocate report, the agency said: "The IRS continues to make progress on a number of the critical issues outlined in the Advocate's report. It is important to note that the IRS must carefully balance limited resources to meet its dual mission of providing taxpayer service and enforcing the tax laws."

Outside experts say the lack of help is a concern both for taxpayers and for the government, which may not get all the money it should if people file their taxes incorrectly.

(Read more: The class divide: Marriage as a 'luxury good')

"It's a lose-lose situation," said Nina Crimm, a professor at St. John's University School of Law and an expert on tax issues. "It's a lose situation for … the average taxpayer, and it's a lose situation for compliance and collection of revenues."

Crimm's advice: Get started on your taxes early this year, so you aren't frantically trying to reach the IRS on April 14.

"They better start calling now," she said.

—By CNBC's Allison Linn. Follow her on Twitter @allisondlinn and Google or send her an email.