The lockstep movement of virtually everything in the market, be it up or down, is making it tough for investors to maintain diversified portfolios.
Whether it's sectors within the stock market, US equities compared to their foreign counterparts, or between stocks and commodities, the parallels are stunning. In some cases, correlation is at 80 percent or better, compared to an ideal that's closer to 40 percent.
Investors seek noncorrelated assets because they help balance a portfolio. If one industry—say food and beverage—is lagging, then it would be expected that an unrelated one—such as semiconductors—should be rising.
But in the past decade that reliability has slowly broken down, and it's pretty much collapsed since the beginning of the financial crisis in 2007.
"What we find in equities is less an oscillation between risky markets and less risky ones, but rather a strong—and continuous—rise in correlations over the past 10 years," analysts at HSBC Global Research wrote in a recent study.
The firm cited "structural shifts" like the rapid growth of the $1 trillion exchange-traded funds industry over the past three years, as well as "geo-macro factors" that have made the global investing world more interdependent and thus pushing prices in the same direction.
The correlation phenomenon is finding its way increasingly into analyst discussions lately, with the primary goal of devising ways around the problem. Some suggestions include taking advantage of the few areas where correlation has broken down, such as emerging market debt against Treasurys; following gold's meteoric rise; and playing emerging and frontier markets that are differing from global trends.
There's even an increasingly popular index that allows investors to buy options on correlation—the CBOE Correlation Index, which is 2 years old and gauges correlation by looking at contracts on the entire S&P 500 index against those on individual contracts. HSBC says the index may soon overtake the CBOE's Volatility Index in popularity as a market indicator. The index had been near all-time highs in September though it has eased a bit in October.
But the problem has no easy solution, while a multitude of factors makes putting together a truly diversified portfolio an arduous task and betrays the popular notion that this has become a stock-picker's market.
A Changing Marketplace
In a broad stocks vs. bonds sense, making a portfolio broadly diversified isn't that tricky: A 60-40 or so allocation of stocks to bonds at least cushions risk that way, so long as the two classes continue their traditional inverse relationship.
But investors also need diversification within stock categories. Should the broader market start tanking, other sectors within it should withstand the storm.
Standard & Poor's 500 stocks have been correlated, or moved in the same direction up or down, 75 percent of the time since January 2007. That compares to a historical average of 39 percent and makes traditional safety areas like consumer staples and health care harder to rely on.
"Investors are getting buffeted by so many headwinds. Every other week we go positive or negative," says Sam Stovall, S&P's chief equity strategist. "It's like pulling petals off a daisy—the market loves me, the market loves me not."
ETFs are but one cause cited for high correlations. They are composed like mutual funds but trade like stocks and either track indices or are composed of a basket of stocks that can be reliably expected to move like an index—either a broad one like the S&P, Dow or Nasdaq, or narrower ones like the Dow transports and other industry-specific indices. (More on ETFs: Why ETFs May Be An Investor's New Best Friend)
High-frequency trading, with its computerized programs and rapid-fire movements, has taken over 56 percent of market volume so it also has exerted major influence as retail investors flee the equity markets.
And the globalization of markets also has played a role, hampering the ability of investors to use emerging markets as a hedge against the US.
"Volatility in the entire portfolio has just gone up because the correlations are much higher than they used to be," says Brian Nick, investment strategist at Barclays Wealth in New York. "As a matter of course, a 50-50 portfolio of stocks and bonds was more volatile in the last three years than it has been in the five years before that."
Looking Around the World
Looking for 'Decoupling'
In analyst parlance, "decoupling" means when investment classes that normally would move together—think stocks and yields on Treasurys—start to break apart.
In a world of high correlation, investors will take diversification any way they can get it, including through decoupling.
Bank of America-Merrill Lynch analysts, in a series of research notes, examined both decoupling and correlations as the market looks for effective hedging techniques.
"One of the most interesting statistical themes in (emerging market) debt over the past year has been correlation breakdowns," firm analysts wrote in September. "These breakdowns are noteworthy to mention because they offer attractive diversification benefits."
The firm has found, for instance, that emerging market debt and US Treasurys have seen a correlation breakdown, providing a diversification opportunity. In addition, emerging market currencies have seen correlation slip compared to Latin American and Asian denominations.
At the same time, Nick says Barclays is moving investors into the long end of the Treasury curve—such as 10-year notes and 30-year bonds—to diversify.
He and others also say using various long-short strategies still can be effective. But the market remains at the mercy of months like May, when 70 percent of the trading days saw at least 85 percent of all stocks move in the same direction.
"There's value in being able to go longs and shorts, it's just as not as glaring as it was earlier," Nick says. "Because you're seeing these landslides, these are very tough days to be a stock-picker. You're at the mercy of whether we had good data."
Investors have also tried going global with their portfolios, but that also has proven to be problematic.
The HSBC study looked at the MSCI All Country World Indexand found its components to be correlated strongly among US, European and many emerging market nations.
It did find that Japan along with smaller markets like Egypt, Morocco, Pakistan, Sri Lanka and Jordan have lower correlation to other world markets.
Still, investors continue to look to foreign markets for diversification hopes.
"You can't just be in domestic stocks. You certainly have to have an international flavor or exposure in your portfolio," says Paul Simon, CIO at Tactical Allocation Group in Birmingham, Mich. "You truly want to look at asset classes that have the strongest fundamentals, that are in a secular trend. Even in the short term you may have some noise and they may become correlated, but as you move further out on the time scale ultimately fundamentals reassert themselves."
Simon also believes gold, which continues to scale to record highs and has done so regardless of equity market performance, is an effective diversification tool as much as it is a flight to safety and a hedge against inflation.
In the past year, according to the HSBC study, the lowest correlation among industries has been 65 percent (autos and household products) and 69 percent for pharma and retailers, while the unlikely pairing of food and beverage with semiconductors has provided a 79 percent correlation.
Autos and household products have manged to avoid high correlation ratios.
A Matter of Mindset
Ultimately, though, diversification may be hard to achieve simply because of market sentiment. Despite low VIX readings, market volume, sentiment surveys and fund flows show there remains plenty of fear—or at least uncertainty—in the market.
And with any dose of significant news, particularly when it comes to the economy, the few retail investors still in the market join with the dominance of high-frequency traders to pull the market strongly in one direction or the other.
"Investors are like fainting goats. If they get startled, their limbs freeze and they fall over."
"Investors are like fainting goats. If they get startled, their limbs freeze and they fall over," says S&P's Stovall. "Sneak up behind them and say 'bear market' and they freeze up and move into a more defensive area. If they feel danger is gone they move back into cyclical areas."
The main antidote to excessive correlation, Stovall says, is an economy that returns to normalcy which in turn will lead to a market where defensive sectors can withstand market downturns.
"What it really shows is that despite having a large number of investments in your portfolio, unless you've got Treasurys in there typically you're not diversified," he says. "If you've got large-cap, mid-cap, small-cap, developed markets and even to some extent REITs (real estate investment trusts) and commodities, if the market is going up they're all going up.
"All would benefit from an improvement in economic growth, which translates into an improvement in corporate earnings which translates to a rise in asset prices."
(Editor's Note: This story has been updated since its original publication in September.)