Fund flows flashing that there's danger ahead

Traders on the floor of the New York Stock Exchange.
Brendan McDermid | Reuters
Traders on the floor of the New York Stock Exchange.

Investors are still betting on a rising stock market, but they've been doing so with less long-term conviction.

A compelling story is developing from how market participants are allocating cash in 2014. On net they are continuing to push into U.S. equities, but they're doing so now with short-term bets on exchange-traded funds rather than historically longer-term commitments to mutual funds.

Read MoreCashin: What could be behind the selloff

About $35 billion has flown into domestic ETFs, boosting the burgeoning industry's assets to nearly $1.9 trillion, according to the Investment Company Institute, and likely past $2 trillion by year's end.

At the same time, ICI figures show that nearly $28 billion from equity funds has exited the $12.9 trillion (excluding money market) mutual fund space. After an early year surge in inflows, investors have pulled money from stock-based mutual funds for five months running.

Bears out in force; Dow off 250 points
Bears out in force; Dow off 250 points   

ETFs and mutual funds differ in terms of liquidity—the former trade like stocks and price in real time, while mutual funds only price at the end of the day and can't be traded during market hours. The difference may not seem dramatic on the surface, but in a market that moves at light speed, the trend and what it means to investors' mentality is getting noticed.

Read MoreJulian Robertson sees TWO bubbles brewing

"It's all about risk," said Todd Schoenberger, president of J. Streicher Asset Management. "Traders and investors are keeping their fingers on the sell button."

It's been a bumpy road higher for the market, particularly over the past few months during which the S&P 500 has risen less than 1 percent, though it remained 6.7 percent in the green year to date despite Thursday's retreat.

Investors ought to prepare for a long road of volatility ahead now that the Federal Reserve is planning on normalizing interest rates and ceasing its monthly bond-buying program that has helped provide trillions in liquidity, said Michael Hartnett, chief investment strategist at Bank of America Merrill Lynch.

"The end of excess liquidity means the end of excess returns and the end of excessively low volatility," Hartnett said in a note to clients. "We think 2015 looks (like) a good year for bulls of volatility. That is not a prediction of a market crash, although we believe the risk of a (greater than 10 percent) correction in the market rises substantially as ZIRP (the Fed's zero interest rate policy) ends."

Read MoreHere's the first 'correction' call...for 2015

The market seems to agree.

In addition to the flight from equity mutual funds, investors have been buying downside protection based on the CBOE Volatility Index, a popular market fear gauge that has surged 38 percent in the third quarter. The VIX remains at a tame level by historical standards but helps explain the market mood at least in relative terms.

As stock-based mutual funds have taken a hit, bond funds have been raking in cash to the tune of more than $61 billion this year, without a single losing month, according to ICI.

To be sure, there are likely additional factors at play in the fund flow trend.

Active management, which is much more often the case in mutual funds than ETFs, has suffered this year with its worst performance against benchmarks in more than 15 years. ETFs also tend to be more popular among younger investors, with those approaching retirement using mutual funds to cycle out of stocks and into bonds.

Read More8 out of 10 active managers striking out this year

Also, hedge funds have been using ETFs increasingly to hedge portfolios as well as chase performance in another year of largely failing to meet their benchmarks.

"This is a tough market for active, absolutely," said Nick Colas, chief market strategist at ConvergEx.

But the overall tenor of the market suggests that investors are nervous and looking for ways to protect their portfolios.

With a changing Fed rate regime and increasing levels of geopolitical turmoil, the need to be nimble only will escalate.

Read MoreThe market's next Fed fear: The exit strategy

"Nobody wants to be trapped, especially when you have everybody beating the drums that we're due for this significant selloff," Schoenberger, of J. Streicher, said. "People are still in, they want to participate. But they want to know if there's trouble, they have a way out."