Lee added in a telephone interview that "over time, you've got to throw the towel in" if the benefits of rising rates keep failing to materialize.
Positioning for higher rates has cost the 20 largest U.S. banks somewhere between $2.5 billion and $3 billion of income each quarter, roughly 6 percent of their collective profits before taxes, said Marty Mosby, a bank analyst at Vining Sparks.
Those estimated costs come from banks refraining from investing customer deposits in longer-term bonds, and holding the funds instead in cash or short-term investments, which offer much lower returns.
Read MoreGlobal central banks manning the battle stations
In addition to foregone revenue, the strategy is also having more tangible costs.
In the fourth quarter, Bank of America marked down a bond portfolio that it uses to hedge interest rate risk by around $600 million, leading it to miss analysts expectations.
Regions Financial said on Tuesday that its net interest margin, a measure of how profitably it loans out its deposits, would decline even more in 2015 than it did in 2014 if rates stay low this year. Some loans that banks make, including mortgages, carry fixed rates, which in the current environment will generate lower returns for their whole lives for lenders.
Almost all major U.S. banks have received far more money from depositors than they can actually lend out, and instead of putting all of that extra cash to work by investing in bonds, many decided to hold off at least some of their investments until yields rose, so they could earn more. For much of last year, investors expected the Fed to start raising rates in 2015. If driven by heightened inflation expectations, those rate hikes could lift longer-term bond yields.
But as bond prices have risen and yields have fallen in recent weeks, banks' potential returns have slipped as well.
"They're stuck waiting for an event that may or may not happen any time soon," said Nancy Bush, a bank analyst at NAB Research.
Read MoreWhat markets do when ECB talks about buying assets
Bond yields in the United States have fallen in recent months for a series of reasons, including investors' increasing pessimism about the pace of economic growth in Europe and China, lowering yields around the world and making the United States look relatively attractive. Concerns about worldwide growth have also increased the appeal of U.S. bonds as a safe-haven.
The 10-year U.S. Treasury yield was 1.87 percent on Thursday, a steep drop from September, when it stood at 2.6 percent.
Shifting rates and bond yields can affect banks in multiple ways. For a big chunk of banks' investment portfolios, falling yields will boost banks' capital levels, which is a positive. And if the Federal Reserve lifts short-term rates, banks' floating-rate loans will generate more income.
But recent drops in yields are clearly cutting into earnings, hurting returns on both bond investments and fixed-rate loans. Banks will not usually compensate for lower bond yields by buying riskier securities such as junk bonds, because they prefer to take credit risk in their loan books.