Stumpf has recently come under fire after it was revealed that employees in Wells Fargo's community banking division opened about 2 million accounts without customer authorization, which resulted in the bank paying $185 million in penalties. Stumpf himself was grilled on Capitol Hill as he defended the bank's sales practices.
The board of directors elected President and COO Tim Sloan to succeed Stumpf as CEO of the bank.
"I am grateful for the opportunity to have led Wells Fargo," Stumpf said. "I have decided it is best for the Company that I step aside. I know no better individual to lead this company forward than Tim Sloan."
Wells Fargo stock closed the day down 0.3 percent, trading at $45.32. The stock has fallen 16.6 percent this year.
--CNBC's Christine Wang contributed to this report.