But Weiss also downplayed the decline in the market.
"We haven't seen a huge selloff," he added.
(Read More: Where the World's Wealthy Are Investing)
Josh Brown of Fusion Analytics said that there was no reason to be a buyer of stocks here.
"First of all, what's the rush? Most people who are running money are sitting on plus-10 percent returns," he said.
Brown also cited the uncertainty of the upcoming employment report.
(Watch Video: ADP: Private Sector Employment Up 135,000)
"Why would anyone want to make a huge bet in front of that?" he said. "Who needs to, other than people who've been trailing this rally?"
Brown said that a 4 percent correction and failing rallies were expected.
"I actually think that's the responsible stance," he added.
Trader disclosure: On June 6, 2013, the following stocks and commodities mentioned or intended to be mentioned on CNBC's "Fast Money" were owned by the "Fast Money" traders: Steve Weiss is long BAC; Steve Weiss is long C; Steve Weiss is long AAMRQ; Steve Weiss is long SODA; Josh Brown is long AAPL; Josh Brown is long XLF; Josh Brown is long MRK; Jon Najarian is long AAPL; Jon Najarian is long VZ; Jon Najarian is long EEM; Jon Najarian is long GLD; Jon Najarian is long TMO; Jon Najarian is long UVXY; Stephanie Link is long AAPL; Stephanie Link is long GS ; Stephanie Link is long JPM; Stephanie Link is long WFC; Stephanie Link is long CSCO; Stephanie Link is long FB; Stephanie Link is long EBAY.