Will bear market push investors to active management?

If a bear market doesn't improve the outlook for active investment management, probably nothing will.

Over the last decade, assets have been flowing out of actively managed investment funds that try to beat a market benchmark and into low-cost passively managed ones that simply track the benchmark. But active fund managers are hopeful that they'll have a better shot at outperforming in a bad market than in a good one.

"There's a cyclicality to when active and passive strategies perform better," said Rob Almeida, an institutional portfolio manager with MFS Investment Management. "My hope is that we've hit an inflection point."

Since 2010, roughly 1 in 5 active large-cap equity and blended-fund managers have outperformed the , according to data from independent investment research firm Morningstar. That ratio may not be as bad in other asset categories using other benchmarks, but the numbers suggest that trying to beat the average return of an investment market is not worth the cost and effort.

"About 25 percent or less of active managers beat their benchmarks in the long term and that's always been the case," said Tim Strauts, markets research manager at Morningstar. "They can't overcome the extra fees and trading costs they have, and there's a broader awareness of that."

Investors are following the performance.

The migration to low-cost passively managed funds has been massive in the last 10 years. The annual inflows into passive funds surpassed those into active for the first time in 2004, and the trend has accelerated since the financial crisis. There is currently $3.8 trillion of assets in actively managed funds and $2.5 trillion in passive.

If fund flows continue at the current pace, passive fund assets could exceed active in as little as five years, suggested Strauts. "There will always be active management, but it's pretty clear that active will be a smaller part of the market than passive at some point in the near future," he said.

Read MoreAre you overwhelmed by fund choices?

The biggest factor in the outperformance of passive indexing is not the failure of active strategies as much as the high costs that come with them.

"Cost may be the best predictor of long-term performance," said Jim Rowley, a senior investment analyst at Vanguard Group, a pioneer of low-cost indexing. "It's not that active strategies can't beat passive ones, but they can't overcome the cost advantage of indexing."

The cost of the Vanguard S&P 500 exchange-traded fund, for example, is 0.05 percent of assets compared to 0.8 percent for the average actively managed large-cap U.S. equity fund. That might not seem like a big difference, but compounded over the long term, it has proved to be an insurmountable hurdle for active managers.

A big part of the growth in passive assets is being driven by the popularity of target-date funds in 401(k) retirement plans. In the past, most companies provided a choice of actively managed funds for employees and used money market funds as the default option.

Read MoreMoney manager growth good for investors

An increasing number of plan sponsors are now automatically enrolling employees in the plan and using target-date funds as the default option. Those funds are actively managed to the extent that they rebalance a diversified portfolio of stocks and bonds based on the investor's age, but they typically invest in low-cost indexed funds rather than actively managed ones.

Beyond the cost advantage of index mutual funds and exchange-traded funds, the market has now evolved to the point where all investors can get exposure to a huge range of asset classes and markets at very low costs.

From emerging markets small-cap stocks to U.S. high-yield bonds and commodity funds, the investing spectrum has expanded exponentially and enabled what Rowley calls active indexing strategies.

"Investors can now find low-cost index funds in most investment spaces," he said. "It enables them to use passive products to build a broadly diversified portfolio and to overweight some exposures if they choose."

Registered investment advisors—the fastest-growing segment of the financial advisory industry—have become enthusiastic users of low-cost ETFs for exactly that purpose.

Don't count active fund management out, however. The desire to beat the market remains a powerful motive for many investors, and actively managed mutual funds still account for about 60 percent of total assets in the industry.

Firms that offer active management are hoping the more volatile markets this year will give their managers a better chance to beat their benchmarks and possibly reverse some of the asset flows into passive funds.

"Active managers need more dispersion of returns on companies and securities for them to outperform the market," said Almeida at MFS Investment Management. Since the spike in volatility at the end of last year, that opportunity appears to be improving.

The proportion of active managers who are beating their benchmarks so far this year has risen above 50 percent, according to Almeida's reading of Morningstar data.

The relatively short period of outperformance, however, has yet to impress investors. Virtually all the outflows from U.S. equity funds this year ($79 billion through July) have been from actively managed funds, said Strauts of Morningstar.

Read MoreShould you go active or passive when investing?

Even in smaller, less-efficient markets where investors might arguably want active managers to navigate the landscape, they are still choosing passive funds. More than 80 percent of the $217 billion in inflows to international equity funds over the last 12 months have been to passive funds.

An admitted enthusiast of passive investing, Strauts doesn't expect a bear market will drive investors back to active management.

"There will always be periods when active strategies outperform passive, but they tend to be short and unpredictable," he said. "There is no data that says active management does better in bear markets."

—By Andrew Osterland, special to CNBC.com