Gregg, a Republican who also served as governor of New Hampshire—agreed, telling CNBC: "It's inexcusable. Taxpayers' dollars should not be at risk for private investment."
"But the way Dodd-Frank is doing it," he continued, "is actually contracting, in many ways, credit on Main Street."
Now for where Gregg and Fisher disagree.
The Dallas Fed president and his colleague Harvey Rosenblum, executive vice president and director of research, have a proposal to address "too big to fail." Fisher said there are traces of it in the bank bill introduced by Sens. Sherrod Brown, D-Ohio, and David Vitter, R–La.
"What we are saying simply is any government guarantee program should only apply to the commercial bank portion of a complex bank holding company," Fisher said. By government guarantee, he said, he means deposit guarantees and access to the Federal Reserve discount window.
"Every other part of a complex bank holding company can still exist," he continued. "[But] if they screw up, they go under."
Let the market sort it out, he said. "That's capitalism at its best."
As for the capital requirement levels in Brown-Vitter, Fisher said they'll be negotiated.
"What they are trying to do is eliminate the cost of funding advantage that these larger institutions have. It's a very complex thing. There are other approaches to do so. We'll just see how that works."
Gregg said he prefers the approach of Sen. John Cornyn, R-Texas. "You basically create, on the too-big-to-fail issue, a bankruptcy option," Gregg said, "rather than have some sort of arbitrary capital requirements."
Gregg also said that the "London Whale" losses at JPMorgan Chase showed that the banking industry is "very well capitalized."
"Everybody is talking about 'the down side,'" he said. "'The good side' of what happened with that event is it didn't affect JPMorgan significantly. It didn't affect the markets."
"More importantly," he concluded, "the company continued to make money and the stock went up."
While Fisher didn't want to comment on whether Jamie Dimon should remain chairman and CEO at JPMorgan, Gregg said that Dimon should keep both roles and that calls for the contrary don't make any sense. The bank's shareholders are scheduled to vote on the issue Tuesday.
(Read More: Jamie Dimon Vote: A New Referendum on Governance)
"I'm speaking personally as a stockholder of JPMorgan—small, very, very small stockholder," Gregg said. "Why would you take one of the most talented people in the country, who runs one of the best banks in the country, one of the banks that's doing the most to make our country productive, and just stick a stick in his eye for no apparent reason other than, what I call, business political correctness?"
"Next, are you going to suggest we split the president's job between the presidency and the commander in chief?" he asserted.
"There are certain companies where the CEO is such a dominant figure that the CEO and chairman jobs should be mixed, should be together," Gregg said. "This is one that clearly should stay together."