Women, in particular, tend to suffer more compared to men because they typically take time out of the workforce to care for their children, parents and spouses.
"Because women have such discontinuous work histories, they have lower income, they have lower private pensions and they have fewer savings of their own," Carr said.
Women are more likely to outlive their husbands. With the rise of "gray divorce" among more mature couples, they are also more likely to wind up single in their later years.
Minorities such as African Americans or Latinos also often come up short in retirement, Carr said, but for different reasons. They are more likely to work in jobs that require more physical labor and often have fewer opportunities to climb the corporate ladder.
More from Personal Finance:
Earn $400K-plus? You may be in for a Social Security shock
A look at the swag some Oscar nominees get
Broke? You probably haven't taken these savings steps
Because these forces are mostly beyond an individual's control, there is really only one way to improve them, according to Carr.
"What we need is to really think collectively about public policies that strengthen the social safety net," Carr said.
That includes proposals such as the Social Security 2100 Act that was recently proposed in the House of Representatives. That legislation, if enacted, could help boost the minimum benefit so that it's well above the federal poverty line, Carr said.
And while sweeping changes included in Medicare for All proposals might be tough to get through Congress, lowering the Medicare eligibility age to 50 could help those individuals who have to leave the work force early, Carr said.
WATCH: Withdrawal tax rules need to change