Thousands of Americans nationwide have been targeted since August by a phone scam in which fraudsters claim to be from the U.S. Internal Revenue Service and demand money for unpaid taxes, said the IRS' watchdog on Thursday.
The Treasury Inspector General for Tax Administration (TIGTA) said it has received more than 20,000 complaints from people, including recent immigrants, about the scam.
Thousands of victims have collectively paid more than $1 million to the scammers, TIGTA said.
"This is the largest scam of its kind that we have ever seen," said J. Russell George, the head of TIGTA in a statement.
(Read more: Big data used to catch fraudulent tax returns)
The fraudsters can manipulate victim's phone's caller ID so it displays the number of a local IRS office, TIGTA said.
In some cases, the fraudsters have also told victims parts of their Social Security numbers.
In cases where victims hung up, fraudsters have called back displaying a local police phone number on caller ID, TIGTA said.
Potential victims worried about their immigration status have been threatened with deportation, TIGTA said.
(Read more: Filing taxes online? How to protect yourself)
The scam has occurred in almost every state and the fraudsters have followed a uniform script, a senior TIGTA official said on a conference call with reporters.
The technology needed to manipulate caller ID displays is easily available to the public, the official said.
Major phone companies have been warned about the scam, as well as companies that provide voice-over-the-Internet call services, the official said.
Claudia Hill, a licensed tax preparer in Cupertino, California, said that in one week last month, four of her clients complained of such scam calls. Before this year, none of her clients had previously mentioned phone scams, said Hill, who said she prepares about 1,000 tax returns a year.
(Read more: Best & worst states for taxes: How's your state?)
"The IRS is not proactive enough in getting out in front of any of this mess," she said.