Midday, the Dow Jones Industrial Average was on track to log its worst weekly drop since April – and the first back-to-back weekly loss since June.
"What happened yesterday was the worst possible world," Siegel said. "We had a strong labor market report with jobless claims, and we know the Fed looks at jobs for the tapering. And of course we have the weak guidance from the retailers."
(Read more: Jeremy Siegel bullish on earnings growth)
Siegel cited the rise in the U.S. Treasury yields but saw a robust demand for equities beginning in the fall.
"It hurts when it goes up," he said. "Now we're going catch 3 percent, I'm sure, on the 10-year – a little indigestion on the stock market."
"But if we can get some strength in the economy, people will look past that to a much better third and fourth quarter, and that's when buying starts in stocks."
— By CNBC's Bruno J. Navarro. Follow him on Twitter @Bruno_J_Navarro.
— CNBC's Stephanie Landsman contributed research to this report. Follow her on Twitter: @StephLandsman.
Trader disclosure: On Aug. 15, 2013, the following stocks and commodities mentioned or intended to be mentioned on CNBC's "Fast Money" were owned by the "Fast Money" traders: Josh Brown is long TLT; Josh Brown is long AAPL; Josh Brown is long XLU; Josh Brown is long XLF; Josh Brown is long FSLR; Enis Tanner is long GS; Enis Tanner is long NUE; Enis Tanner is long CF call fly; Enis Tanner is long SPY Puts; Enis Tanner is long DXJ; Enis Tanner is long UUP; As of /15 Michael Murphy is long BAC; Michael Murphy is long C; Michael Murphy is long MSFT; Michael Murphy is long LEN; Michael Murphy is long TOL; Michael Murphy is long F; Michael Murphy is long INTC; Michael Murphy is long CAT.