IRS scam continues long after tax-filing season

How to protect yourself from bogus IRS agents

Con artists may be evil, but they're not stupid. If they hit on a scam that works, they keep at it.

That explains why the bogus IRS agent scam continues long after tax-filing season is over. Telephone scammers have struck gold and they're not about to stop mining for more victims.

This swindle is incredibly simple and straightforward. The crooks pretend to be an IRS agent or someone from the U.S. Treasury Department calling about a problem with your tax return.

Read MoreBig data used to catch fraudulent tax returns

"They say you didn't pay enough or the money wasn't received, and the only way to remedy this and make sure nothing bad happens to you is to get money to them immediately," explains Lois Greisman, associate director at the Federal Trade Commission. "Some of them can become very threatening and very abusive."

The scammers typically threaten potential victims with arrest or deportation. They may also claim that they can revoke a license or shut down a business if they don't get the money right away.

To make their pitch seem more legit, they will often spoof the caller ID to make it display the IRS toll free number (800-829-1040).

If you hang up, another scammer may call, this time pretending to be with your local police department.

Whatever the exact pitch, the goal is always the same: To get your hard-earned money.

They may be willing to take a credit card payment, but typically they want you to go to the store and wire them money or buy a preloaded debit card and call them back with the card number and PIN.

Do that and your money is gone for good. Victims have lost hundreds, sometimes thousands of dollars.

In mid-March, J. Russell George, the treasury inspector General for Tax Administration, issued a news release in which he called this "the largest scam of its kind we have ever seen." At that time, TIGTA said it had received reports of scam attempts from more than 20,000 people and knew of thousands of victims who had collectively paid more than $1 million.

A spokesperson for the agency told CNBC the number of victims "has increased dramatically" since then. No arrests have been made.

Elaine Kuo talked to one of the scammers who identified himself as "Officer John Smith with the IRS." She was visiting her father-in-law in Maryland when the call came in. She describes the conversation as "very frightening and pretty threatening."

This "Officer Smith" said there was a warrant issued for her father-in-law because of income tax errors. He said law enforcement would be arriving within a half hour to arrest him.

Read MoreDon't fall for the 'phantom debt collector' scam

Taxes & identity theft a growing concern

When Kuo pressed for details about the problem, "Officer Smith" turned the call over to a co-conspirator, who claimed to be "Officer Alex Marshall".

"He said many many times that within the half hour law enforcement would be there and kept constantly asking, 'Do you want to be taken away today?' " Kuo recalled.

Marshall told her the matter could be resolved right away over the phone. That's when she hung up.

Recent complaints indicate that some of the scammers are now using robocalls—automated telephone recordings—to hook their victims.

Beverly Dyal of Olympia, Washington, got four of these calls all in one day. The first one came in at 7:15 a.m. The caller identified himself as "Officer Jason Miller" and he said:

"Now don't try to disregard this message and do return the call. If you don't return the call and if I don't hear from your attorney either, then the only thing I can do is wish you good luck as the situation badly unfolds on you."

How to spot the scam

The IRS follows certain procedures that are designed to protect your rights. If you understand how the agency operates, it's easy to spot the scams.

"We are not going to initiate contact with somebody via phone, email or any social media outlet and ask for personal or financial information," said Julianne Fisher Breitbeil with IRS media relations.

The IRS may call in certain situations, such as when the audit process has started and they want to schedule an appointment.

Read MoreFight fraud with remote control for credit cards

"But we are not going to call and say we need money," Breitbeil explained. "The scammers ask for immediate payment over the phone. We are never going to do that—never."

If you get a call and you're not quite sure what to do, hang up and call the IRS at (800) 829-1040.

If you've fallen for the scam, file a complaint. The FTC has more information on its website about how this scam works and how to report it.

—By CNBC contributor Herb Weisbaum. Follow him on Facebook and Twitter @TheConsumerman or visit The ConsumerMan website.