What even Vanguard and Fidelity can't figure out

Income in retirement is the single-biggest issue for older Americans who are not affluent and who lack a legacy defined benefit plan such as a public sector worker's pension. Yet efforts by the largest mutual fund companies to create income solutions have struck out. While the big mutual fund companies, led by Vanguard Group and Fidelity Investments, have had tremendous success with target date funds that get an investor to retirement, it's been a different story when it comes to managing income in retirement.

Vanguard announced recently that it was merging a number of its Managed Payout Funds into one portfolio. After five years of trying to sell its Managed Payout Funds family, Vanguard stated that, "to help make it simpler for you to use the investment to supplement your retirement income, we plan to merge the current three portfolios into a single fund, which will be named Vanguard Managed Payout Fund."

The merger is expected to be completed in early January. "We remain committed to an all-in-one option for retirees and believe a single fund will simplify our managed payout offering and better serve clients' needs going forward," Vanguard CEO Bill McNabb said in a statement.

(Read more: The pros and cons of target date funds)

In the mutual fund industry, a fund merger is synonymous with failure. The fact is that if Vanguard had raised enough in these funds, it wouldn't have merged them. "These guys don't get out of bed for anything less than $10 billion," said Jeff Tjornehoj, director of research at Lipper. "To be out there for five years and get $1.5 billion, it's a disappointing figure."

Fidelity's Income Replacement Fund series has fared no better—actually, it's fared worse, with a total of $83 million in assets, according to Lipper. And a look at the assets raised in similar portfolios from other big fund companies shows the same "crickets playing" when it comes to the attempts by fund companies to package income for retirees. Pimco: $37 million; Putnam Investments: $53 million; Schwab: $194 million. In fact, Vanguard has been the big winner in a loser's game.

A Vanguard spokesman said, "We remain committed to the managed funds concept. We did this for the simplification that comes in a single fund focused on payout. We felt it was better to make it easier for investors to understand."

The income issue is a huge one, and a growing one, with the boomer generation just hitting retirement. To some extent, the fault is not the fund companies' own, other than to have launched these income funds at the exact wrong time. Vanguard's launch of the Managed Payout funds coincided with the financial crisis and a period of record low interest rates coinciding with central bank intervention in the markets.

"Boomers are saying they want a 5 percent interest rate with little risk and there is nothing like that out there. Any product that can deliver good income may not last," Tjornehoj said. In fact, the recent high-profile launches in the fixed-income market—given expectations of an end in the 30-year bond market run—have ratcheted up the risk profile of bond managers. "Even with high inflation, 5 percent is a magic number," Tjornehoj said.

It's notable—in demonstrating the difficulty traditional bond fund managers face—that on Monday Vanguard's Total Stock Market Index Fund surpassed Pimco's Total Return Fund as the largest mutual fund of all, after a record six straight months of outflows for the Pimco fund, according to Morningstar data.

(Read more: ETF model portfolios: Staying on the retirement track)

"What is the income strategy going forward? There is no easy answer," said Kate Reichart, Morningstar analyst. "Lots of funds companies haven't thought about it as much as they should." Especially in the context of the target date funds with which they have been successful and where many participants in the funds are moving into retirement. The fund companies want to keep those assets for the rest of an investor's life.

"There is an immense conversation among fund providers and with advisors on how to position products for the drawdown of these assets and provide income that the baby boomers are expecting and need," Tjornehoj said. "Try to figure that out in a low rate environment where there is no advantage built into the market on how to build a more substantial income stream."

Tim Maurer, financial planner with the Financial Consulate, is one of the advisors in the thick of this conversation. Maurer said that one problem for the fund companies is that the income market is one in which insurance companies have long dominated with annuity products. "Let's face it, insurance companies have made their bread and butter here. They [the funds] are struggling to compete."

(Read more: Barbarians at the bond fund gates)

Maurer said the interest rate environment is also a huge hurdle. "To provide an offering that is anything other than a hedge against fear is difficult," he said. And that is the issue the fund companies had to their advantage: fear. "Vanguard and Fidelity are such monsters that one would think if they applied all their muscle and energy, the funds would catch on. The fear is there," Maurer said. "Clients are concerned about outliving their money."

The current generation of retirees do not have the levels of "true" fixed-income coming from Social Security and pensions that previous generations had, Maurer said. And it could not be any worse for current retirees, other than a repeat of 2008.

The reality of their investment situation as they head into the income phase of life is treading water, and the fund companies' response has not arrived like a rescue boat. "These products are attempting to be an answer to that fear that insurance companies have been answering for years. I think in many cases, these products are more sizzle than steak," Maurer said. "They may be solving a fear, but they are not solving an investment problem in the best way."

A significant problem with managed payout funds from the outset was that the funds were designed to provide regular payouts—no matter what. They were equipped to adjust for variations in equity performance, but in the event of a major drawdown, where the fund value declines precipitously, that could wreak havoc on the underlying algorithm, and the result depended on the design of the fund—some would deplete way too soon and some would start making de minimus payments, or both, explained Tamiko Toland, managing director, retirement income consulting at New York City-based mutual fund research and consulting firm Strategic Insight.

"There is no guarantee in the design of those things, and that's just how they work," Toland said. "These funds were always vulnerable to significant drawdown events and it just so happened that 2008 came around the corner not long after these funds incepted. I always felt that they were a useful niche product but no replacement for solutions that offered more of an income guarantee."

—By Eric Rosenbaum, CNBC.com