Just when the company seemingly found its footing and was on the way to a comeback in 2014, the rebound proved to be temporary. Since then, Gap has given back gains, including a 41 percent decline last year.
The first warning sign for Cramer was when its long-time CEO Glenn Murphy announced he would step down in October 2014. He handed the reins to Art Peck. By the end of the year, things were falling apart.
Read more from Mad Money with Jim Cramer
Cramer Remix: Forget it, OPEC is broken
Cramer game plan: Sizzling buys on an oil swoon
Cramer's tax day showdown: Intuit vs. H&R Block
At the time, Peck cited quality and fit issues as the reason for Gap's weak performance, which Cramer interpreted to mean that the people running the company thought they had a problem with the product.
The stock finally appeared to hit a rebound in February 2016, but those gains were erased quickly when Gap confirmed that same-store sales shrank 6.5 percent, with all three divisions down.
"I think the problems here began at their core Gap brand and then spread like wildfire through the rest of the company," Cramer said.
Even with all of its internal issues, the worst problem with Gap is something it may not have control over: the long-term decline of mall shopping.
Countless retailers have felt the effects of a downturn in mall traffic, as many consumers have migrated to shopping online.
Ultimately, Cramer does not want investors to fall into the Gap.
"Here is a company that lost its way quite some time ago. Gap is having an identity crisis when it comes to its brands, and management has some serious issues. Even if they fixed everything, this company would still be one of the biggest victims of the decline in shopping mall traffic," Cramer said.