Revelations in IRS Scandal 'Chilling': Rep. Paul Ryan

Rep. Ryan: No Immediate 'Hard Core' Cuts in Our Budget

House Budget Committee Chairman Paul Ryan told CNBC on Wednesday that investigating Obama administration scandals won't keep Republicans from pursuing sound fiscal policy.

In the Internal Revenue Service's targeting of conservative groups, what has been learned so far is "just chilling," Ryan said in a "Squawk Box" interview, claiming that the facts already known are damning.

"I see it as the arrogance of big government," he asserted, saying Congress is going to do its job on oversight and there's no need to appoint a special prosecutor who will come back "two years later after spending $20 million with some kind of conclusion."

(Read More: IRS Names Financial Manager New Chief Risk Officer)