For most states and cities, pension obligations are the biggest single expense, and the costs are increasing. Unless jurisdictions find ways to adequately fund such costs, the strain on budgets will reach a breaking point, Carter said.
That is a view shared by Warren Buffett, the chairman of Berkshire Hathaway Inc, who last week warned that the crisis in public pensions will intensify. The main reason, he wrote in a letter to shareholders, is public entities have promised pensions they cannot afford.
"During the next decade you will read a lot of news—bad news—about public pension plans," the legendary investor wrote.
Buffett and others believe the recent bankruptcies of Detroit and other, smaller U.S. cities are just the beginning of a trend that will soon envelop municipalities around the country.
(Read more: Battle overpolice pensions in US cities takes ugly turn)
Pension obligations were a big factor in Detroit's bankruptcy. The same is true for the bankruptcies of Stockton and San Bernardino in California.
A major issue in Detroit is whether the city can slash pension benefits promised to current and retired workers. If the city prevails in court, its victory might encourage other municipalities to declare bankruptcy to deal with their pension shortfalls. That's not an option for U.S. states, which are barred from bankruptcy.
Almost all state constitutions protect pension benefits. Some, most notably California's, require paying promised benefits to retired workers before any other debts.
Since the 2008 financial crash, which resulted in huge investment losses for public pension funds and major strains on budgets generally, police and firefighting services have been slashed in many cities, along with other basic services such as libraries, street maintenance and schools.
To be sure, others are less pessimistic than Buffett, saying a more robust investment outlook and other economic factors have improved enough for cities and states to rest easier.
(Read more: Would Scots lose out on pensions if they vote for split?)
Indeed, there is evidence that the overall health of pension systems is brightening. A report released by the Federal Reserve last week said liquidity of pension funds has improved. Pension assets amounted to $3.88 trillion in the final quarter of 2013, more than a third higher than the $2.83 trillion reached in 2009.
"I think we're doing fabulously well," said Hank Kim, executive director of the National Conference on Public Employee Retirement Systems. "Certainly the stock market and the rise in equities over the five years since the Great Recession helped tremendously."
Robust returns over the past two years may have stabilized shortfalls for many systems, said Rachel Barkley, a municipal credit analyst at Morningstar, but some pension plans will likely struggle to narrow funding gaps. That's because what is paid into the funds each year has consistently fallen short of what is required.
"In general, one or two years of good returns is not going to solve their problems," Barkley said of state funds that were already in weak shape.
By contrast pension systems that were already well funded have been able to use investment windfalls to bolster their positions even more, Barkley said.
Colorado vs. Oregon
Speaking in general, public pension systems are badly underfunded in states, such as Colorado, that give elected politicians in the legislature the power to set funding levels. Oregon and other states that are mandated by law to meet annual funding requirements are in much better shape.
Over the past five years, only nine states have made the full required contributions to their pension plans, according to the non-partisan Pew Center on the States.
Colorado's public pension funds showed overall investment returns of 12.9 percent in 2012, the most recent figure that's available. It was well above its projected rate of 8 percent.
Yet the funding ratio, the measure of assets against liabilities, has remained relatively static, increasing from 61.2 percent in 2011 to 63.1 percent at the end of 2012.
Most economists view a funding ratio of less than 75 percent as unhealthy. According to Morningstar, two-thirds of U.S. states currently fall into that category.
Although Colorado is still absorbing losses from 2009, the main reason its funding gap is yawning is the state's failure to make the contributions recommended each year by its own budget experts, Barkley said.
Between 2008 and 2012, Colorado lawmakers short-changed the state pension fund by nearly $1.4 billion, and by $3.5 billion over the past decade.
Likewise, in Illinois, with the biggest funding gaps of any U.S. state, a pension fund for teachers showed healthy returns of 12.8 percent in 2013. Yet its funding gap worsened between 2012 and 2013.
By contrast, Oregon's retirement system, covering about 95 percent of the workforce, is required by law to meet the annual funding recommendation of its own accountants. The mandated recommendation, known as the "ARC," or "annual contribution rate," takes the issue out of the hands of policy-makers wrestling over budget choices and competing constituencies.
The ARC is calculated based on myriad factors, including investment returns the fund can expect in future years.
In 2010, despite big investment losses, Oregon's system was 87 percent funded. Today it is 96 percent funded.
Even if public pensions realize their projected investment returns on average over coming years, the failure by many plans "to pay less than the full ARC ... will produce less than full funding over the next 30 years," according to a recent report by the Center for Retirement Research (CRR).
Buffett and others have made dire warnings about public pensions before, notably in 2010. Some analysts predicted at the time a domino effect of municipal pension-related bankruptcies, something that has yet to materialize. And the $3.7 trillion municipal bond market has shrugged off the latest warnings.
Chris Mier, managing director of analytical services at Loop Capital, said Buffett may not have taken into account the full impact of reforms instituted by about 40 states.
Mier conceded that the reforms, which have cut benefits and increased contribution rates for workers—mostly for new hires—will take several more years to translate into improved funding levels for pensions.
Meanwhile, the cumulative budget shortfall of U.S. state public pensions has surpassed $1 trillion and is still growing, according to Pew.
"Public debt is continuing to grow, and despite reforms, the question is if states can manage another downturn or another recession," said Greg Mennis, director of Pew's public pension project.
Jean-Pierre Aubry, assistant director of state and local research at the CRR, said most funds would likely show improvement in shortfalls beginning in the next fiscal next year, as the end of a five-year "smoothing" accounting period will finally push the deep losses of 2009 off the books.
Even so, many funds "are still going to be grossly underfunded," Aubry said. Improved returns will only give them some short-term relief, he added.