"The IRS notes this issue does not involve its main computer system that handles tax filing submission; that system remains secure," the agency said in a statement.
The agency's commissioner, John Koskinen, said on a Tuesday conference call that the hacking was conducted in an organized manner, and that there were about 200,000 attempts to illegally access information—about half of those were successful.
"We're confident these are not amateurs, these are actually organized crime syndicates that not only we but everyone in the financial industry are dealing with," Koskinen said.
Read MoreRefund fraud to hit $21B, and there's little the IRS can do
The IRS said thieves targeted the system from February to mid-May. The service has been temporarily shut down.
"In all, about 200,000 attempts were made from questionable email domains, with more than 100,000 of those attempts successfully clearing authentication hurdles," the agency said. "During this filing season, taxpayers successfully and safely downloaded a total of approximately 23 million transcripts."
Tax returns can include a host of personal information that can help someone steal an identity, including government old-age pension numbers and birthdates of dependents and spouses. However, the IRS said the thieves appeared to already have a lot of personal information about the victims.
The IRS said it is notifying taxpayers whose information was accessed.