That being said, some are more prepared than others, and the wealthiest Americans have significantly more in savings.
According to the Economic Policy Institute (EPI), which looked at the state of American retirement in a 2016 report, the top one percent of families had $1.08 million or more stashed away in 2013. That's 216 times the median working-age family (50th percentile), which had only $5,000 saved in 2013.
The 90th percentile family had $274,000 saved, which is 55 times the total of the median American family. The 80th percentile family had $116,000, or 23 times the median family.
"Participation in retirement savings plans is highly unequal across income groups," the EPI reports. "In 2013, nearly nine in 10 families in the top income fifth had retirement account savings, compared with fewer than one in 10 families in the bottom income fifth."
While retirement inequality is growing, the EPI notes, the good news is, you don't need a lot of money to start investing and building your nest egg.
One of the easiest ways to fund your future without feeling strapped is to have a portion of your paycheck sent directly to a retirement account, such as a 401(k) or IRA. That way, you'll never even see the money and learn to live without it.
And the sooner you start making automatic contributions, the less you'll have to save each month to reach your goals. In fact, if you start in your 20s, you may not have to set aside as much as you may think to retire with a million-dollar portfolio.
Like this story? Like CNBC Make It on Facebook.