Cramer pointed to Thursday's and Friday's back-to-back IPOs of retailer Canada Goose and software company MuleSoft.
Cramer said the two shouldn't worry investors yet. He embraced Canada Goose as a rare sign of growth in a struggling sector, and called MuleSoft "a terrific company" that even after a 45 percent run right out of the gate, "seems like … it's got more room to run."
But if the two are the start of a trend, more of the same could lead to oversupply in the market, Cramer warned.
"I always says that what ultimately kills the bull is supply, because the stock market, like any other market, is all about supply and demand. When the market's roaring, more companies will want to come public, and sooner or later we get so many IPOs that the market's flooded with supply and it overwhelms the demand," the "Mad Money" host said.
That could result in the market pause so many analysts have been predicting since the rally began, Cramer said.
"Do you know that this might be my 20th IPO cycle in 37 years where I see the supply coming, and it looks to me like we're white water rafting together. And we hear the falls, but we're having such a good time we say to ourselves, 'Why stop?'" he asked.
"Well, I'll tell you why. The bottom of the waterfall is too far from here. Nobody makes it out alive," Cramer continued. "So, forgive me, but if IPOs keep coming at this torrid pace … I'm eventually going to have to tell you to get out of the raft."
Questions for Cramer?
Call Cramer: 1-800-743-CNBC
Want to take a deep dive into Cramer's world? Hit him up!
Mad Money Twitter - Jim Cramer Twitter - Facebook - Instagram - Vine
Questions, comments, suggestions for the "Mad Money" website? email@example.com