Comey also addressed the significant differences between Clinton's actions and the case of former CIA Director and General David Petraeus, saying that "the Petraeus case, to my mind, illustrates perfectly the kind of cases the Department of Justice is willing to prosecute."
The hearing also saw Rep. Trey Gowdy of South Carolina unleash a blistering critique of Clinton, highlighting a series of apparent lies she told about her emails, and alleging that the FBI's decision will create a two-track justice system wherein elite members of society are held to different standards.
"It's not true, but I've heard it a lot," Comey said in response to a later question about whether Clinton received different treatment than another member of the public would.
One notable exchange saw Comey suggesting that the former secretary of state's level of sophistication may have been an issue in whether she understood whether emails contained classified information. Explaining that three emails on Clinton's server contained "(C)" markings on portions of body text — indicating confidential material — Comey said the FBI considered whether someone may not have understood that demarcation.
"I think it's possible — possible — that she didn't understand what a C meant when she saw it in the body of an email like that," Comey said of Clinton.
Still, Comey later noted that all of those emails were improperly marked, as they should have also included a header note that they included classified information. In fact, he said it "would be a reasonable inference" for Clinton to conclude that those emails did not include any classified material because of the absence of those headers.
When he announced Tuesday that he would recommend Clinton not be charged, Comey said the presumptive Democratic presidential nominee and her aides had been "extremely careless" in their handling of classified information. Yet he recommended that they not be prosecuted.
"Although there is evidence of potential violations regarding the handling of classified information, our judgment is that no reasonable prosecutor would bring such a case," he said during a news conference.
Attorney General Loretta Lynch said Wednesday she had accepted the recommendations of Comey and of her prosecutors that no charges be filed. Lynch had previously emphasized she would follow the FBI's lead on the case.
The FBI's announcement created instant backlash from congressional Republicans, who said they felt Comey had laid out a sufficient basis for criminal charges.
"The FBI's recommendation is surprising and confusing. The fact pattern presented by Director Comey makes clear Secretary Clinton violated the law," House Oversight Committee Chairman Jason Chaffetz, a Utah Republican, said in a release. "Individuals who intentionally skirt the law must be held accountable."
Experts have also been divided on the FBI's conclusion, alternately telling CNBC that the move was shocking based on their understanding of the law, and saying Comey was right to recommend against prosecution because investigators did not find intent to cause harm or reap personal gain from the transmission of classified documents.
— The Associated Press and CNBC's Tom DiChristopher contributed to this report.