Comey appeared Thursday morning before the House Oversight and Government Reform Committee, reiterating his contention that the FBI's recommendation was not at all tinged by political considerations.
"I want the American people to know we really did this the right way. You can disagree with us, but you cannot fairly say we did it in any kind of political way," Comey said. "We don't carry water for anybody. We were trying to do what the right thing is."
During the hearing, Comey said he stands by his recommendation not to prosecute Clinton — in keeping, he said, with years of precedent — but did allow that non-criminal penalties would be imposed on anyone under his purview who acted similarly.
"My conclusion was, and remains, no reasonable prosecutor would bring this case, no reasonable prosecutor would bring the second case in 100 years focused on 'gross negligence.' And so I know that's been a source of some confusion for folks: That's just the way it is, I know the Department of Justice, I know no reasonable prosecutor would bring this case," Comey said.
"I know a lot of my former friends are out there saying they would (bring such a case). I wonder where they were the last 40 years because I'd like to see the cases they brought on 'gross negligence,' nobody would, nobody did," he added.
Comey also addressed the significant differences between Clinton's actions and the case of former CIA Director and General David Petraeus, saying that "the Petraeus case, to my mind, illustrates perfectly the kind of cases the Department of Justice is willing to prosecute."
The hearing also saw Rep. Trey Gowdy of South Carolina unleash a blistering critique of Clinton, highlighting a series of apparent lies she told about her emails, and alleging that the FBI's decision will create a two-track justice system wherein elite members of society are held to different standards.
"It's not true, but I've heard it a lot," Comey said in response to a later question about whether Clinton received different treatment than another member of the public would.
One notable exchange saw Comey suggesting that the former secretary of state's level of sophistication may have been an issue in whether she understood whether emails contained classified information. Explaining that three emails on Clinton's server contained "(C)" markings on portions of body text — indicating confidential material — Comey said the FBI considered whether someone may not have understood that demarcation.
"I think it's possible — possible — that she didn't understand what a C meant when she saw it in the body of an email like that," Comey said of Clinton.
Still, Comey later noted that all of those emails were improperly marked, as they should have also included a header note that they included classified information. In fact, he said it "would be a reasonable inference" for Clinton to conclude that those emails did not include any classified material because of the absence of those headers.
When he announced Tuesday that he would recommend Clinton not be charged, Comey said the presumptive Democratic presidential nominee and her aides had been "extremely careless" in their handling of classified information. Yet he recommended that they not be prosecuted.
"Although there is evidence of potential violations regarding the handling of classified information, our judgment is that no reasonable prosecutor would bring such a case," he said during a news conference.
Attorney General Loretta Lynch said Wednesday she had accepted the recommendations of Comey and of her prosecutors that no charges be filed. Lynch had previously emphasized she would follow the FBI's lead on the case.
The FBI's announcement created instant backlash from congressional Republicans, who said they felt Comey had laid out a sufficient basis for criminal charges.
"The FBI's recommendation is surprising and confusing. The fact pattern presented by Director Comey makes clear Secretary Clinton violated the law," House Oversight Committee Chairman Jason Chaffetz, a Utah Republican, said in a release. "Individuals who intentionally skirt the law must be held accountable."
Experts have also been divided on the FBI's conclusion, alternately telling CNBC that the move was shocking based on their understanding of the law, and saying Comey was right to recommend against prosecution because investigators did not find intent to cause harm or reap personal gain from the transmission of classified documents.
— The Associated Press and CNBC's Tom DiChristopher contributed to this report.