US consumers a hard sell for traditional retail
If there was one lesson from this year's holiday shopping season, it is that many traditional retailers are having to work a lot harder to persuade Americans to open their pocketbooks.
A lot of stores had to discount heavily to eke out a modest increase in sales, likely squeezing profit margins in the process.
(Read more: Why a Target gift card may be worthless)
Some improvement in the U.S. economy and declines in the jobless rate, plus gains in stock and home prices, are failing to resonate with many Americans whose incomes are struggling to catch up to where they were before the financial crisis.
But to many retail experts and economists there are other less cyclical factors at play. Consumers are spending more. Government figures show monthly personal consumption has risen for seven straight months, with November's outlay marking the fastest increase in five months. But they just are not spending in the shopping malls like they used to.
And that means that, even if the economy picks up significantly, retailers of many products could still struggle.
"We are in a something of an evolutionary process, said Bill Martin, founder of data firm ShopperTrak, which monitors foot traffic in about 60,000 retail stores. Americans are spending more online and becoming more careful about what they buy, he said.
Some of this has been unfolding over a long period, although the changes might be picking up pace.
For example, department stores have found themselves on the wrong end of trends for some time. According to data compiled by Reuters, they now capture just $3.37 of every $100 of U.S. retail spending, the lowest since records began in 1992, when the number was nearly $9.
Some of that is explained by the rise of Wal-Mart Stores Inc and other big box discount retailers. But the pace of decline has picked up, with department stores losing about 0.28 percentage points of market share at an annualized rate between 2002 and 2011, compared with 0.22 in the prior 10 years.
(Read more: Where to snag post-holiday deals)
The problem is two-fold. The middle class consumers to whom the likes of JC Penney Co Inc and Kohl's Corp cater have struggled with stagnant wages and a payroll tax rise, prompting them to reduce spending on apparel, said Scott Tuhy, a retail analyst at Moody's Investors Service in New York.
People have also gravitated toward spending on services such as travel - airline ticket prices and hotel room rates are up - as well as movie downloads and other content for their TVs, smart phones and tablets. Prices to attend live sports events, theme parks, movies and rock concerts have also been rising.
In addition, increasing healthcare costs have been eating up discretionary income, with many employers seeking higher contributions from their staff.
According to the Commerce Department, spending on services hit an annual rate of $7.1 trillion in November, by far the biggest slice of overall consumption
"There was a day you bought your TV, refrigerator, furniture, everything in a department store, whereas today, it's really just apparel and maybe jewelry," said Stuart Hoffman, an economist at PNC Financial Services Group in Pittsburgh. "But as incomes rise over time, people spend more on services - travel, entertainment."
As data from MasterCard showed last week, it took deep discounts and hefty promotions to spur a 2.3 percent rise in holiday sales between November 1 and December 24 compared with a year earlier. The figures include apparel, jewelry, electronics, luxury goods and home furnishings.
"Given how promotional the season turned out to be, profits are likely to be flat because retailers had to provide quite a lot of discounts to get those sales," said Moody's Tuhy.
And it's not as if people aren't doing some serious shopping.
U.S. sales of big-ticket items such as autos and home-related goods such as washing machines, as well as purchases in home-improvement stores, surged in 2013, boosting overall retail sales. Homes sales also increased pretty steadily from mid-2012, although a summer spike in mortgage rates cooled things off a bit this fall.
Some of the gains reflected a long-anticipated release of pent-up demand as the economic recovery has gained momentum, but it might also be partly a reflection of changing attitudes, with the focus on more practical purchases.
According to the National Association of Realtors, more than half of home buyers between July 2012 and June 2013 made some sacrifices, such as reducing spending on luxury items, entertainment and clothing.
"Pent-up demand has helped on housing and autos, but consumers are still cautious. Things are getting better, yes, but even if you have a job, things are still tight," said Sam Bullard, an economist at Wells Fargo in Charlotte.
In its 2014 retail industry outlook, Moody's said it expects the auto and home improvement sectors to outperform again in 2014, good news for Home Depot Inc and General Motors Co, whose stocks are up 32 percent and 41 percent, respectively, this year.
It's been a different story with more ordinary purchases.
An Ipsos/Reuters poll released just ahead of the Christmas holiday found consumers plan to spend about a third less this year than last year on items such as jewelry, toys and electronics.
(Read more: A pricier 2014? You can thank Amazon Prime)
Sluggish sales of toys and packaged foods pushed down sales at Wal-Mart's U.S. stores in the third quarter and prompted the company to forecast disappointing holiday sales results, while Target Corp blamed "constrained" consumer spending for a tepid rise in quarterly sales.
One factor is that the giant TV is much cheaper than it was, and tablets and many of today's laptop and desktop PCs are cheaper than their predecessors of a few years ago, again not helping store sales. Shoppers are also increasingly likely to buy such items online from the likes of Amazon.com Inc and Apple Inc.
Participants in Deloitte's 2013 holiday shopping survey for the first time named the Internet as their number one shopping venue, with 68 percent of smart phone users and 63 percent of tablet users planning to use their devices to help them.
As people check out goods online before heading to the store
impulse buying can take a hit, said Martin. ShopperTrak data shows mall shoppers visited an average of 3 to 3-1/2 stores this year, down from 4-1/2 to 5 in 2007.
The improvement in the economy and gains in asset prices have also clearly not trickled down to large segments of the broader population.
Hiring has picked up, helping to push the jobless rate to a five-year low of 7 percent in November, but some 11 million Americans are still unemployed, and many are earning less than before the recession. When adjusted for inflation, average weekly wages have barely budged since late 2008.
That has made Americans, particularly those in middle- and lower-income brackets, far more discerning when it comes to their spending.
"There's been psychological scarring for people from this recession, much like how some people who lived through the Depression said they were scarred," said Hoffman. "There are still a lot of people who can't afford to do much, and those who can are holding back."
Follow us on Twitter: @CNBCWorld