The journey begins Friday, according to the bank's chair, Betsy Duke. His replacement will have to regain the credibility lost after the lender's fake accounts scandal helped to unearth a series of problems across the bank's sprawling operations, according to analysts and investors.
Most of the time, when the CEO of a big bank retires, the company's board has already picked a successor, and that person is usually already working at the bank. That happened recently when Goldman Sachs' Lloyd Blankfein resigned after 12 years and president David Solomon was made CEO.
The last time an American megabank entertained outside leadership was when Bank of America's Ken Lewis abruptly announced he would resign during the financial crisis. That job eventually went to an internal candidate, Brian Moynihan.
Wells Fargo's next leader will have to be comfortable working under the glare of regulators, politicians, media and customers. During his four-hour Congressional drubbing, Sloan said that the bank was operating under no less than 14 consent orders from regulators.
Still, much of the cleanup job has probably been done under Sloan's two and a half year tenure, and the new executive would enjoy a honeymoon period. And the lure of suddenly vaulting to the top of the nation's fourth biggest bank -- a behemoth with $1.9 trillion in assets -- at a crucial moment in the industry's evolution could be too hard to resist.
The first place to look, according to industry watchers, is at rival J.P. Morgan. The biggest U.S. bank by assets has been a training ground for CEOs, helping seed Barclays, Standard Chartered, Visa and First Data with their leaders.
J.P. Morgan chief financial officer Marianne Lake would be a strong choice because "she has the strong finance background working for a very complex organization," according to Citigroup analyst Keith Horowitz.
A former colleague of hers, Matt Zames, could also easily step into Sloan's shoes, Horowitz wrote. Zames was chief operating officer at J.P. Morgan and joined Cerberus Capital Management last year.
Gordon Smith, co-President of J.P. Morgan, came up in "nearly every conversation" with investors as a potential successor, Bank of America analyst Erika Najarian said in a March 11 note.
Management changes atop Goldman Sachs have also made two executives available for the position. Gary Cohn and Harvey Schwartz have both reportedly been considered as potential candidates, though Wells Fargo has pushed back on those reports.
Other potential names from the banking world include Richard Davis, former head of U.S. Bancorp and Bill Demchak, CEO of PNC. Demchak is another former J.P. Morgan executive.
One potential hurdle: Wells Fargo could be in the uncomfortable position of having to pay the candidate tens of millions of dollars in deferred compensation to join the bank.
That dynamic scuttled European rainmaker Andrea Orcel's plan to join Banco Santander as CEO earlier this year. Santander pulled his offer after saying it didn't want to pay him $50 million in deferred compensation for leaving his former employer, UBS.
As a result, executive chairman Ana Botin is now focusing on internal candidates for the CEO post, according to people with knowledge of her plans.
In the meantime, Wells Fargo's general counsel, Allen Parker, is interim CEO. The longtime corporate lawyer was one of a handful of new executives to join the bank after its fake account scandal broke in 2016.
"There's a highly accomplished leader out there who's going to look at the challenge and opportunity and say, "I'm a really good fit for Wells Fargo," Duke, the bank's chair, said Thursday. "That's the person we want at Wells Fargo."
With reporting from CNBC's Wilfred Frost, Dawn Giel and Michael Bloom.