"Shareholders are more fickle than shoppers," Coach's chairman and CEO Lew Frankfort said on CNBC's "Closing Bell."
Frankfort told CNBC his company suffered in the fiscal second quarter from lackluster holiday sales and intensified competition in the U.S. department store space.
But Frankfort wouldn't get stuck in the negative saying, "Today is just a moment in time."
In addition to expecting 50 percent growth from the men's brand this year, Frankfort is focusing much of Coach's growth strategy on mobile.
"Remarkably 40 percent of consumers who visited us on the internet came via mobile devices in the second quarter," he said.
Frankfort said Coach is also looking for growth by expanding its reach in the shoe and clothing segments. Coach announced plans Wednesday to re-launch the shoe section at 170 of its 356 stores.
The dreaded inflection point when luxury goods become too mainstream for a brand's core consumer doesn't seem to worry Frankfort. "None of our research suggests that we are there," he said. "The accessories category is very brand loyal."
But Coach does cater to non-high end customers in addition to the traditional luxury consumer. Frankfort told CNBC the average cost of a Coach bag is less than $300.
"Coach offers an affordable product for what's in it," he said. "Our products are made out of the best materials. They are durable."
Coach announced second-quarter earnings of $1.23 per share missing analyst expectations of $1.28 Wednesday morning. The company also reported quarter-over-quarter revenue jumped 3.8 percent to $1.5 billion, falling short of analyst forecasts of $1.6 billion.
— By CNBC's Shannan Siemens. Follow her on Twitter @ShannanSiemens