On CNBC's "Fast Money," Kelly said investors would do well to buy U.S. Treasury bonds.
Japan's additional foreign investment could total about $200 billion going into the U.S. stock and bond markets, he added.
"Once again, everything is going to be manipulated, and eventually when the levee breaks, it's going to completely fall apart," he said.
Read More BOJ surprises with fresh stimulus, Nikkei surges 5%
Kelly also said he had made a winning bet by being short Japanese yen coming into the trading day, adding the massive Japanese stimulus program gave him pause.
"However, I felt this way before—and it was during Bear Stearns. Everybody cheered that Bear Stearns got a bailout from the Fed. And within three days, they were out of business," he said. "So, this is Japan bailing themselves out. they had no choice. They have to raise taxes. They are now monetizing their debt—100 percent monetizing their debt, and buying stocks. They're buying REITs. They're buying ETFs. It's insane."
Read MoreMohamed El-Erian: 'QE trade is evolving'
Kelly clarified he wasn't calling for a massive selloff in the near future.
"I'm not saying the market's going to fall apart on Monday morning," he said. "I'm just saying it's the same type of feeling where people are cheering a bailout they shouldn't cheer."
Stuart Frankel's Steve Grasso, who said he remembered the failure of Bear Stearns that set off the financial crisis in 2008, was more tempered in his view.
Read MoreContrarian fund manager's top 3 stock picks
"I just think that right now I'm worried about the market not falling on a cliff but maybe correcting a little," he said, adding the midterm U.S. elections could see contested races and runoffs, muddying control of the Senate.
OptionMonster's Pete Najarian took a different tack.
"When you look at yesterday, what was really rallying? Everything was rallying. It was the financials. It was technology. It was just about everything—industrials, pharma, all those names moving to the upside," he said, noting the CBOE Volatility Index was "cheap."
Read MoreCitigroup stock 'like free candy at these prices': Mike Mayo
"This is time again to start buying. Fourteen on the VIX. That means you buy S&P protection of your portfolio."