Stop IRS Culture of 'Rot and Abuse': Rep. Boustany

Rep. Charles Boustany, R-La., told CNBC on Thursday he's confident congressional investigators will follow the "paper trail" to hold to account those responsible for the Internal Revenue Service's targeting of conservative nonprofit groups.

Boustany, head of the House Ways and Means Subcommittee on Oversight, said in a "Squawk Box" interview that he's been looking into allegations of IRS abuses for two years and he been faced with "stonewalling."

"We're going to reign in this culture of rot and abuse," he said. "The [Obama] administration should have known … what's going on. It's a very serious."

The House Ways and Means Committee—which has jurisdiction over all U.S. tax policy—is scheduled to hold a hearing into the matter on Friday. Outgoing IRS Acting Commissioner Steven Miller is expected to testify, as is the Treasury inspector general investigating the complaints, J. Russell George.

President Barack Obama announced the departure of Miller on Wednesday—the same day Attorney General Eric Holder appeared before the House Judiciary Committee. Holder warned that the Justice Department will be conducting a criminal investigation into whether false statements were made by top IRS officials.

"I think we're going to see a whole plethora of taxpayers coming forward with additional information," Boustany said. "This is only just getting started. It's a big deal."

The IRS inspector general report released on Tuesday said inappropriate criteria were used to identify tax-exempt applications for review over a period of more than 18 months beginning in early 2010. The ineffective management resulted in substantial delays in application processing and unnecessary information requests to be issued, according to the audit.

"The IRS is the most powerful organization, basically, in our federal government," said Boustany. "If a taxpayer is misleading or lies in their interactions with the IRS, there are severe penalties. It goes both ways. We're going to hold these individuals accountable."

On Wednesday, GOP House Speaker John Boehner raised the possibility of jail time for those responsible. (Read More: Jail Time for IRS Violations: Boehner)

Next week, the Senate Finance Committee will hold a hearing, while the House Oversight and Government Reform Committee will also take its turn.

By CNBC's Matthew J. Belvedere. Follow him on Twitter @Matt_SquawkCNBC.