At the heart of the issue is an inviolate social contract that bankers are supposed to honor. The government agrees to protect banks from collapse, and in return, bankers are meant to uphold the highest ethics when handling other people's money. But when law-breaking and other missteps proliferate at banks, it is a sign that the industry has stopped cleaving to the special contract, endangering taxpayers. And bad management can be a leading indicator of future financial problems at an institution. "It usually translates into losses down the road," Mr. Curry said.
The big question is whether regulators have the resolve to back up their tough words with meaningful punishments. Banks, for instance, have armies of lawyers who deploy strategies like refusing to turn over potential evidence to regulators. And the largest banks make such big profits these days that they can easily absorb the financial penalties the government throws at them. Also, notably, top bank executives did not voice their support for Mr. Dudley after he gave his sharply worded speech on culture.
(Read more: 'Too big to fail' is worse now—Commentary)
"I haven't yet seen bankers rushing to say Bill Dudley speaks truth on this issue," Cornelius K. Hurley, a professor at the Boston University School of Law, said.
Nonetheless, there are some signs that financial regulators have become more assertive since the financial crisis. The Office of the Comptroller of the Currency and the New York Fed, which each had a reputation for being too soft on the banks, have made important changes.
The regulators say that they have taken important steps since the crisis that will make it easier to crack down when they need to. Mr. Curry pointed to some new rules that his agency proposed in mid-January that, he said, could facilitate enforcement when rules are broken. "We are ratcheting up the potential consequences," he said. "This is something new."
The regulators also said that they have stepped up pressure on the banks' boards. The regulators hope that more independent-minded directors will demand changes if they see standards and practices slipping, especially in crucial areas like accounting and risk management. To help promote that push, Mr. Dudley said, the New York Fed has bolstered the stature of supervisors who interact with the boards and senior executives. "We've put some of our very best people in those spots," he said.
But boards may have very different priorities from regulators. Directors may not see the need for far-reaching changes if a bank is producing large profits that benefit shareholders.
JPMorgan Chase's board took steps to hold management accountable after the so-called London Whale trading scandal that engulfed the bank in 2012 and 2013. Still, in January, JPMorgan's board approved a large raise in the 2013 pay of Jamie Dimon, the bank's chief executive.
(Read more: 'Fabulous Fab' to pay over $825,000 in SEC fines)
Antony P. Jenkins, the chief executive of Barclays, which has been hit by its own scandals, took a different approach with his 2013 compensation. Earlier this year, he turned down a bonus worth $4.5 million.
And compensation is one area where bank regulators may need to do more if they want to do more to clean up bank culture, according to critics of the industry.
Wall Street's compensation practices can reward unhealthy levels of short-term risk-taking and entice bankers into ethical lapses. Acknowledging that, regulators around the world agreed after the crisis to overhaul bankers' pay, in part by requiring them to wait several years before they receive all of their bonuses. The hope is that bankers will behave better if they know their employers can easily take back the deferred part of their pay.
But there is evidence that large American banks are still deferring much less pay than their European peers. The Fed is in charge of regulating compensation at American banks. When asked whether the pay overhaul at American banks had gone far enough, Mr. Dudley said, "There is potential to defer more compensation for longer periods of time."
One particularly daunting challenge looms over the efforts to improve the ethics of banks. Some banks may be so large and complex that it would be difficult for managers to maintain a clean culture across all of their operations.
But Mr. Dudley said he would not allow size or complexity to be an excuse for ethical breaches. "Either the firm is not too complex, you can manage it, you do know what's going on," he said. "Or, if you don't know, that's sort of raising the question whether the firm is too complex to manage."
—By Peter Eavis of The New York Times