Bigger stores could hurt retailers' profits this holiday

J.C. Penney store in Glendale, Calif., Aug. 16, 2013.
Patrick T. Fallon | Bloomberg | Getty Images

Although holiday sales are expected to post modest gains this season, the same may not hold true for retailers' bottom lines.

Higher store counts and larger shops, as well as an increase in orders when the sector showed signs of recovery back in the spring, could hurt many retailers' margins as they slash prices to move product.

"There's a whole ton of people that we believe are going to be inventory heavy," said Craig Johnson, president of Customer Growth Partners.

After trimming back on square footage during and in the aftermath of the recession, Johnson cited a reversal in this trend as a key reason for a potential inventory overload. As the consumer slowly regained strength, retailers across the spectrum started upping their store counts and building larger sales floors. This was seen at small retailers, such as jewelry and accessories retailer Charming Charlie and women's apparel retailer Francesca's, as well as at bigger names, such as Michael Kors, Tory Burch and Wal-Mart, Johnson said.

Year over year, Customer Growth Partners reported a 4 percent increase in square footage among retailers.

When this growth is coupled with Johnson's forecast for a 2.9 percent holiday sales increase—the slowest since the recession—the firm is projecting retailers will also see the first holiday season decline in store productivity per square foot since the recession.

What's more, Johnson said, demand has been on a "stairstep down" since July, and retailers will start to see an impact on their margins in the closing quarter.

(Read more: Retail sales get iPhone kick, but overall demand weak)

He noted, however, that it's important not to make a blanket judgment. Department stores and luxury retailers will likely take a bigger hit, as their orders were completed back in March and April, when consumers were coming off a cold winter and demand showed signs of recovery. On the flip side, retailers who chase inventory, such as TJX and Ross, were able to order later in the season.

"They can be much closer to gauging consumer demand," he said.

Wal-Mart CEO expects 'best Black Friday ever'
Wal-Mart CEO expects 'best Black Friday ever'

Although the National Retail Federation's Kathy Grannis also expects promotions to continue to be aggressive this season—citing Kmart ads that ran 105 days before Christmas—she said the group so far does not expect promotions to hurt retailers' margins.

Pam Goodfellow, insights director of Prosper Insights & Analytics Consumer, said she expects retailers will be more creative with their promotions, targeting their best customers or card holders, for example, to avoid blowout 60 percent sales they can't afford.

"They have fully planned on keeping their inventories in line where they think they should be and not overstocking their shelves," Grannis said.

Retailers of course are also facing uncertainties regarding the impact of the government shutdown, the outcome of Obamacare legislation and an unstable job market.

(Read more: Consumer confidence lower than expected in October)

Even typical budget relievers do not appear to be easing consumers' concerns. Wal-Mart CEO Mike Duke told CNBC on Tuesday that lower gas prices have not had their usual impact on Walmart shoppers.

"This year's a little bit different, I think, with the tailwind of gas prices offset by the headwinds of the payroll tax change at the beginning of the year [and] the upcoming SNAP [Supplemental Nutrition Assistance Program] benefits changes on the first of November. I think it remains to be seen how it's going to impact our sales," he said.

According to Customer Growth Partners, less than 48 percent of the working age population has a full-time job, which is "the lowest it's ever been since women entered the workforce in large [numbers] two generations ago," Johnson said. He sees part-time workers as less likely to make discretionary purchases, which could hurt sales for apparel retailers, department stores, home furnishings and consumer electronics.

It could also put a strain on self-gifting, which the National Retail Federation predicted will dip slightly this year to $129.62 from a survey high of $140.43 last year.

"There's still a chance for the 'one for you, one for me' mindset, but people might be a little more cautious," Grannis said.

—By CNBC's Krystina Gustafson.