Meteoric sales can't last forever. While investors and analysts know that's true, there's still disappointment stemming from Michael Kors' latest earnings release. But maybe more sustainable sales growth is exactly what Kors needs for the long-run.
In its 12th quarterly report as a publicly traded retailer, the high-end handbag maker outpaced Wall Street estimates on the top and bottom lines, announced a $1 billion share buyback program—its first ever—and raised its fiscal third-quarter earnings forecast, but still Kors shares are selling off, recently down nearly 8 percent in midday trading.
Read MoreCoach snags new shoppers—at a cost
Kors sales guidance was not quite as bullish as some investors had hoped and sales in its biggest market are slowing, as is traffic, or the number of shoppers in its stores, according to executives on the earnings call.
"I think there's a lot of fear that the U.S. trajectory of the business is going in the wrong direction," Sterne Agee retail analyst Ike Boruchow told CNBC.
While Kors' international sales continue to grow, nearly 80 percent of the retailer's total sales originate from North America. Sales at stores open at least a year in North America grew nearly 11 percent, the slowest rate since going public in December 2011 and a far cry from the growth rate just seven quarters ago, when it topped 45 percent.
CEO John Idol said in response to a question on the earnings call, "I know there is concern on many of your parts about the North American comp[arable sales], but I think we are reaching levels [as you said before] that are more sustainable, but North America as a marketplace has plenty of growth left for us."
Idol also said the company thinks it will "grow probably long-term, in the high-single digits or very low double digits," adding this growth rate is "a more sustainable model for us."
Read MoreHere's where holiday shoppers plan to splurge
And maybe high single-digit growth isn't so bad. After all, part of the attraction for high-end brands is scarcity, and the more consumers that carry a Kors handbag, the more the sheen wears off the aspirational luxury name.
Despite its growing market share, Kors is still desirable. Piper Jaffray's latest biannual "Taking Stock with Teens" survey indicated Michael Kors was the top handbag brand among teens for the second straight survey.
Wells Fargo analyst Paul Lejuez told investors that while Kors has numerous growth opportunities, there is concern about the risk of over distribution. However, there's a fine line when it comes to what Wall Street will accept as growth slows.
Read MoreComparing Kate Spade to Michael Kors? Think again
"Coach has been the canary in the coalmine in terms of what happens to these red-hot fashion accessory brands over time, and you know Coach went from low double-digit positive comps to negative comps in just four quarters, so it's not outside of the realm of possibility that that could happen," said Boruchow, who recommends investors look at Kate Spade over Kors or Coach as this point.
Not all analysts are concerned about Kors. Jefferies analyst Randal Konik said he sees a buying opportunity in the stock's current weakness.
"We now see an attractive entry point as Europe and North America still represent significant growth catalysts that we believe the market is undervaluing," he said.