Once Wall Street shook off its initial surprise at the departure of a longtime aide to JPMorgan Chase CEO Jamie Dimon, the parlor game begins again: Who will succeed Dimon when he retires?
Mike Cavanagh had worked by Dimon's side for some 26 years—first at Smith Barney, then Citigroup, Bank One, and, most recently, JPMorgan Chase. At JPMorgan, Cavanagh had been the firm's chief financial officer, head of Treasury services, and—a job viewed by many as a precursor to chief executive—co-head of the investment bank.
Read MoreTop JPMorgan China banker to resign
Cavanagh was seen next to Dimon at conference receiving lines, greeting clients, and in the last year, the two took meetings together with independent lawyers and advisors to discuss forthcoming capital restrictions and the future structure of JPMorgan.
Now, a relative newcomer to the firm is getting some nods. Gordon Smith joined JPMorgan in 2007, after some 25 years at American Express, to run the bank's then-fledgling auto finance division. Since then, he's taken the helm of Chase consumer and community banking, a unit that employs 170,000 people from call center employees to mortgage originators to senior executives.
Insiders tout the diversity of this experience as preparation for a job such as chief executive. Smith's tenure has been nothing short of impressive, they say, especially as scrutiny on financial products like payday loan and debit card fees has ratcheted up in Washington.
Read More
Smith's name has been put on par with that of another oft-mentioned contender, JPMorgan Chase Chief Operating Officer Matt Zames. In his current job, Zames is the one keeping the trains running on time. His skills came in handy at the 11th hour, when the bank was negotiating to buy a failing Bear Stearns in 2008, and when it needed to unwind a disastrous credit derivatives trade that lost the bank upwards of $6 billion.
While a skillful operator, insiders point to Zames' swift rise from a mortgage capital markets banker to one of the strategic forces behind a $200 billion market-cap firm. Others question his background as trader (he traded mortgage-backed securities for Long Term Capital Management, before its demise) as preparation to steer a bank as unwieldy as JPMorgan.
Finally, observers point to Mary Erdoes, chief executive of JPMorgan Asset Management, as an experienced executive who's intimately familiar with the firm, having joined pre-Chase JPMorgan in 1996. Erdoes has grown JPMorgan's assets under management to $2.3 trillion, though the unit still represents a fraction of the bank's overall revenue. Even so, Erdoes, at 46 years old, could be in pole position to inherit the bank if Dimon retires in the next few years.
CFO Marianne Lake, investment banking chief Daniel Pinto, and commercial banking head Doug Petno also have been mentioned in the list of potential successors.
Read MoreJPMorgan Chase to cut thousands more jobs
The question of who will run the bank falls not only on age—with Zames seen as possibly too young, and Smith possibly too old—but also on the form JPMorgan ultimately takes when Dimon decides to step down. At the bank's investor day in February, Dimon said it may be a "huge mistake" to split the bank based on the current financial landscape, that may be the right decision down the line, depending on how strenuous capital requirements become.
If the bank breaks up, an entirely new crop of executives could find themselves fighting over its pieces.
—By CNBC's Kayla Tausche. Follow her on Twitter: @KaylaTausche