For many Americans, retirement won't be golden

The blissful vision of retirement may die hard, but the financial contours of most Americans' post-career years are changing rapidly and radically.

Born of the optimism during the post-World War II economic boom, when private and public pension systems expanded, "retirement" for decades became a distinct period when people could expect to spend more time with grandchildren, indulge in their hobbies and generally enjoy the fruits of their labor.

But demographic, economic and political factors have altered that landscape.

Although the needs of longer-lived retirees are increasing, most notably from a health-care perspective, traditional sources of income are dwindling.

Gone are defined-benefit pension plans and retiree health benefits, and going are Social Security benefits. Retirement costs are rising, while expected returns on personal savings are falling.

Read MoreThe 4% retirement rule is broken

The upshot is that Americans of all ages and most income levels will have to alter their plans. They will have to work longer and change their lifestyles both before and during retirement.

And in many cases, they will have to rein in their expectations about their "golden" years.

"Retirement was a 20th-century idea that won't exist in the same way in this century," said Ric Edelman, chairman and CEO of Edelman Financial Services, which has about 25,000 clients.

For millions of Americans, the retirement outlook is bleak. The latest annual study of the retirement readiness of baby boomers and Generation X by the Employee Benefits Research Institute found that 38 percent of Americans in the lowest-income quartile of those generations will likely run out of money during their first year of retirement.

Read MorePlanning for retirement health care

By year 10, 64 percent of them will be penniless and 11 percent of those in the next-lowest quartile will be in the same predicament. When long-term-care costs—the biggest wild card—is factored in, the numbers get worse.

"It's astounding how fast retirees in the lowest-income quartile will run out of money," said Jack VanDerhei, research director at EBRI. "An abundance of other people are going to become indigent when they need nursing care, and no one knows who is going to pay for it."

But those at virtually all income levels face major challenges planning for their retirement.

For starters, most economists expect that Social Security benefits are more likely to be cut than expanded due to the system's long-term deficit.

Read MoreRetirees turn to encore careers

And with DB pension plans virtually a thing of the past, the responsibility for retirement saving rests solely on the individual. Yet just half of working Americans participate in a 401(k) retirement plan, and most aren't contributing enough to adequately fund their retirements.

"Half of American households won't be able to maintain their standard of living in retirement, and it gets worse for younger people because of Social Security," said Alicia Munnell, director of the Center for Retirement Research at Boston College.

The best way to ease the financial crunch of retirement is to not retire.

"It's a running joke with my younger clients that they're never going to retire," said Lazetta Braxton, a certified financial planner who is founder and chief executive of Financial Fountains, which serves primarily middle-income Americans.

Read MoreIRS ups 401(k) limits

The average retirement age increased to 62 from 57 during the past 10 years, and the age at which Americans expect to retire increased to 66 from 60, according to Gallup polls.

Another survey of 2,100 American adults by Northwestern Mutual Life Insurance found that 4 of 10 respondents over 60 said that they expect to work until at least 75 before retiring.

"The longer you can defer drawing on your nest egg, the longer it can sustain you," Braxton said.

A longer career can also lead to much higher Social Security benefits. Those who wait until 70 to tap the system get a monthly benefit that is 76 percent higher than those who file at the earliest eligible age of 62.

Read More 4 reasons to not roll over 401(k) cash

The big question is how many Americans will be able to work later in life.

"Fifty percent of the retirees we interview every year say they retired earlier than they intended to because of health reasons or because their job skills weren't marketable anymore," EBRI's VanDerhei said.

In addition, Americans aren't savers. The personal savings rate in the U.S. increased briefly after the financial crisis, but it is at 5.3 percent of income, according to the latest data from the Bureau of Economic Analysis.

That isn't nearly enough. For the most part, average Americans have next to no savings when they retire, other than what is in their 401(k) plans and individual retirement accounts.

Read MoreAre you ready to retire yet?

Those retirement accounts aren't very robust, either. The average married American couple between 55 and 64 has a combined balance of $111,000, according to the Center for Retirement Research, with the median balance far lower.

"It's alarming how little money there is in 401(k)s," said the Center's Munnell.

One of the most positive trends has been the automatic enrollment of employees into corporate 401(k) plans, EBRI's VanDerhei said.

Employees can opt out, but they are automatically enrolled at a default contribution rate of 3 percent of income.

Read MorePicked a retirement spot? Hold on!

Meanwhile, in terms of spending, the key is self-discipline.

"My job is to help people pay for their needs and have enough left over to enjoy some of their wants," Edelman said.

Studies show that those who use financial advisors generally do a better job of that than those without one.

For those on an unsustainable financial path, Edelman advises that they review all their expenses.

Read MoreRetirement planning at the 11th hour

And a second job or a return to college to upgrade skills may also help.

"Sometimes the best way to advance is to take a step backward so you can avoid more extreme measures that could be forced upon you later," Edelman said.

—By Andrew Osterland, special to CNBC.com