The yield on the 10-year Treasury note has to get to 3 percent to 4 percent for banks to really benefit, he said. But much higher yields have their drawbacks as well, and will put pressure on banks' securities books and their mortgage-lending businesses.
According to a recent report from Moody's, as the Federal Reserve starts to withdraw its monetary stimulus, interest rates could rise sharply. "This could drive losses in banks' securities portfolios and would reduce mortgage originations and related revenue," the ratings agency wrote in a report.
As bank stocks rise on anticipation of higher rates, Miller said, banks "will not be able to deliver over the next couple of quarters as much as investors want, or as quickly."
Already, financials are up 20 percent this year, making them the market's best-performing sector.
Moody's writes that the reduced mortgage activity and pressure on securities portfolio will over time be "substantially offset by wider net interest margins."
Miller's case remains that interest rates should remain lower for a longer period because growth is still relatively subdued.
That should let investors continue to play the "mortgage trade" well into next year, said the analyst, who has outperform recommendations on Wells Fargo, PNC Financial and US Bancorp and equal weight on Bank of America.
(Read More: Yield Surge Sends Signal of a Scary Second Half)
Barclays analyst Jason Goldberg is more positive about big banks such as Citigroup and JPMorgan, which are trading at single digit price-to-earnings ratios.
"The group should benefit from the prospects of higher interest rates," he said. "As the economy expands, loan growth follows. Interest rates should rise, helping net interest margin, helping the capital markets—which aids fee income.
Banks have been good on expenses, and credit quality is very benign," Goldberg added. They're buying back more stock, which sets up a nice earnings equation."
—By CNBC's Justin Menza. Follow him on Twitter