Rakoff said some prosecutors' explanation for the lack of high-level cases is "implausible." He called the suggestion by top officials, including Attorney General Eric Holder, that some prosecutions would damage the economy—the "too big to jail" argument—"a doubtful proposition."
(Read more: Bank of America verdict spotlights US focus on civil cases)
It is rare for sitting judges to offer opinions about issues that could come before them—rarer still for them to grant interviews to journalists. But Rakoff said he has been careful not to cross any ethical boundaries, repeatedly pointing out, for example, that he has no opinion as to whether fraud actually occurred.
"I did my very, very best to couch my article in terms that were fully within the bounds permitted by judicial ethics," he said. As a result, he is not concerned that his comments might complicate a future case in his court.
Rakoff said his restraint did leave him open to what he called "a cheap shot" by Justice Department spokesman Brian Fallon, who complained in a statement following the article that "the judge does not identify a single case where a financial executive should have been charged, but wasn't."
"That would be inappropriate for a federal judge, even when giving his opinions as a citizen, to say I think X should have been prosecuted or Y should have been prosecuted, and I think the Justice Department well knew that I couldn't do that," Rakoff said.
(Read more: JPMorgan faces possible penalty in Madoff case)
Fallon also noted that the department has prosecuted thousands of defendants since the financial crisis and pointed to the record $13 billion settlement last month with JPMorgan Chase as an example of authorities being tough on big institutions for their role in the meltdown.
But Rakoff is critical of a growing emphasis by the Justice Department on prosecuting corporations instead of individuals.
Manhattan U.S. Attorney Preet Bharara, whose office handles most securities fraud cases, declined to comment on Rakoff's article. In the past, Bharara has argued that charging or threatening to charge a corporation can have a longer-lasting impact, forcing a company to change its culture.
"I have huge respect for Preet Bharara, a great U.S. Attorney by any measure. But even great men can make mistakes," he said.
"If you prosecute a CEO or other senior executive and send him or her to jail for committing a crime, the deterrent effect in my view vastly outweighs even the best compliance program you can put in place."