"There is still an interesting stigma attached to plus-size fashion and the woman who wears it," says Marie Denee, who wears a size 16 and studies the industry via her website TheCurvyFashionista.com. "Many think `Oh, she doesn't want to draw attention, live life, date, be confident, wear fitted clothes with bold colors and patterns,' when the exact opposite is true."
Carmen Barrington, 32, says that attitude has resulted in fewer plus-size options. Barrington, who wears around a size 22, lamented recently after a day of shopping at Forever 21, Lane Bryant and other stores that she sometimes can't even find decent plus-size clothing at retailers that specialize in it.
"It was a hot, annoying day, and I spent it trying on stuff, and came up with nothing," Barrington, who works in human resources, says. "There's this aversion to being associated with plus-size clothes."
To be sure, sizing is an inexact science. Women's sizes were developed in the 1920s as catalogues became popular and ready-to-wear clothing replaced tailor made or self-sewn items.
(Read more: As luxe outlets go online, off-price wars heat up)
But while a system of men's standard sizing based on chest sizes in the Army had worked well, a similar attempt to base women's sizes on bust measurements wasn't as reliable. Women's bust sizes are more variable.
In the 1930s, retailers began adopting even-numbered sizes commonly ranging from 14 to 24, says Alaina Zulli, a dressmaker who studies costume history. But those sizes bore little resemblance to those used today - a size 24 back then, for instance, would be a size 14 today - so the issues of not having enough plus-size fashions likely was not as pronounced.
The sizes stayed the same but the numbers decreased gradually, Zulli says, about 1 size a decade. This is known as "vanity sizing" because it gives women the impression that they're fitting into a smaller size.
Women's sizes, which today range from 0 to 24 but vary from store to store, haven't evolved much for decades. And for the most part, neither have the range of plus-size fashions.
As a result, the amount spent on women's plus-size clothing annually has only risen by one percentage point to 9 percent since 2011, the furthest back plus-size data available from research firm NPD Group.
"If the offering becomes stronger, women will spend more," says Alison Levy, a retail strategist at consulting firm Kurt Salmon. "There's a significant dollar opportunity in what is currently a very depressed apparel segment that retailers should be looking to exploit."
Some retailers have started to do just that. H&M, a European-based retailer that sells trendy clothing in the U.S. equivalent of sizes 1 through 16, last summer featured plus-size model Jennie Runk, who is a size 12 or 14, in its swimsuit ads. "Our aim is not to convey a certain message or show an ideal, but to have a campaign which can illustrate the collection in an inspiring and clear way," said Andrea Roos, an H&M spokeswoman.
Lane Bryant, a plus size retailer, said earlier this month that it is expanding into higher-end designer clothing. It will debut its first designer collaboration with Isabel and Ruben Toledo on a collection of holiday clothes and later a spring line.
"This is a big deal for us and we're treating it in that manner in every way we can," said Linda Heasley, Lane Bryant's CEO.
And online retailer Asos three years ago launched a plus-size category called Asos Curve with sizes 14 to 24. The company fits everything on a size 16 model to "ensure we are offering the right fit and comfort to our customers," says Natasha Smith, an Asos Curve buyer. The company wouldn't disclose sales figures but said they've been stronger each season.
"Our customer comes in all shapes and sizes and our range should reflect that," Smith says.
But for every chain adding to their plus-size offerings, there are many others that continue to cater to smaller sizes. Abercrombie & Fitch, for instance, has been criticized for only offering sizes 0 to 10 and its CEO's comments that the chain caters to "cool" and "attractive" kids.