"The market's excited about more stimulus, or at least less aggressive tightening, and that's provided spark for the equity markets," said Michael Sheldon, chief market strategist at RDM Financial Group. "But while today's rally is encouraging, it will be interesting to see if we can hold onto all of this morning's gains."
Summers' surprise decision bolstered risk appetite as investors had expected him to take a more hawkish course regarding stimulus than other candidates if appointed. This leaves Fed Vice Chairman Janet Yellen as the front runner for the job. A well-known policy dove, Yellen would be expected to continue Bernanke's easy money policies.
(Read more: Wall Street wanted Yellen anyway: CNBC survey)
"The market will, at the margin, see his withdrawal as one which prolongs unorthodox policy for longer — partly because it moves the more dovish Yellen up the favorites list for the new job," said Deutsche Bank's Gael Gunubu in a research note.
Yields on U.S. government debt fell to their lowest levels so far in September following the news. Benchmark 10-year bond yields declined 8 basis points to 2.805 percent, near its session low and the lowest in two weeks.
Shares of homebuilders including Pulte and DR Horton spiked following the announcement. Summers was expected to rein in the government's stimulus program, which could have pushed interest rates higher, including rates for mortgages.
(Developing: 12 confirmed dead in DC Navy Yard shooting rampage)
Investors will be closely watching the two-day Federal Open Market Committee meeting, which ends on Wednesday. The central bank is expected to announce the start of the tapering in its monetary stimulus program. The latest Reuters poll showed economists expect the central bank cut its bond purchases by $10 billion, from the current $85 billion per month.
(Read more: Here it comes: Are you ready for the Fed to taper?)