"At the heart of it, I think that cupcakes by themselves do not attract enough people every day any longer," he added. "Maybe when Crumbs was 10 stores, it most certainly did."
But since going public in 2011, Crumbs has grown to many multiples of that size, which Slezak saw as a liability.
As the company's footprint soared to 80 locations, including a sizable mall presence, they began cannibalizing each other. More recently, the company has begun closing stores, and now has 65 locations in 12 states and the District of Columbia.
Read MoreStarbucks bows to complaints, brings back items
Last month, in a filing with the Securities and Exchange Commission, Crumbs said its accountants expressed "substantial doubt" about its ability to "continue as a going concern," unless it was able to raise additional funding.
Crumbs' rapid expansion has contributed to what Technomic Executive Vice President Darren Tristano said was a "saturation of restaurants selling cupcakes" in the face of demand that's begun to decline.
"We said it was likely going to be a fad with some long-term sustainability," Tristano said in a phone interview. "Clearly, consumers are moving on to something else. It's kind of in the rear-view mirror now."
The National Restaurant Association's "What's Hot" annual survey of chefs and data from market research firm The NPD Group reflect this.
The association had cupcakes on its "hot" list from 2007 until 2012, when they were taken off since their trendiness level had barely budged for several years, said spokeswoman Annika Stensson. Today's hottest desserts include hybrid desserts (such as the Cronut), savory ones, artisan ice cream, mini desserts and deconstructed desserts.
Retail cupcake servings dropped 5 percent from 2012 to 2013, according to NPD.
Read MoreThe worst tippers live here ...
Bonnie Riggs, an NPD restaurant industry analyst, attributed this decrease to the closure of cupcake shops. While Riggs said by email that she thinks cupcakes will continue to be a popular treat, she described the proliferation of cupcake stores a few years ago as a fad—and one that appears to be passing.
"Cupcakes were often priced too high, there wasn't a strong enough market (many shops were opened in smaller towns without the population to sustain the business), and there was too much competition from other cupcake shops and retail outlets," she wrote.