Though several new names have appeared among Victoria's Secret's competitive set — among them, online retailer Adore Me and New York City-based Journelle — the brand has a "wide, deep moat" around its business, Jaffe said.
That's because other large players in the industry cater to a different audience. Chico's-owned Sonoma, for instance, targets an older customer, whereas Lane Bryant's Cacique is focused on plus-size shoppers.
Tubin added that Victoria's Secret has faced new players before. About a decade ago, a number of specialty shops and department stores tried to capture a piece of the market, but it never really materialized, he said.
"There's a technical expertise needed to make this stuff," Tubin said. "It's not just like making a T-shirt. There's fit concerns and there's strapping technologies."
What's more, as the largest player in the space, Victoria's Secret has the necessary factory relationships to produce items relatively quickly and inexpensively.
Overall, both Tubin and Jaffe agreed that what's made Victoria's Secret such a powerhouse is its ability to turn what used to be a commodity product into a fashion product.
But Jaffe said that doesn't mean the specialty store deserves to be valued like a consumer staples company, which tend to be less susceptible to economic or fashion swings.
"The fundamentals are so excellent," Jaffe said. "I just question what price you pay for this modest growth."