Baby boomers and seniors said that their biggest financial challenge was planning for retirement, while younger people were most concerned with paying for college and sticking to a budget.
When asked about their financial fears, Americans said they are most worried about being in debt forever and living paycheck to paycheck their entire lives. Those concerns were, unsurprisingly, much more common among millennials, whose wealth accumulation was stymied by the recent financial crisis.
Read MoreIt will take Texans a while to pay off their debt
Even if we only include millennials who are living independently (excluding the full third of young Americans who are still living with mom and dad), the median young adult's net worth was about 30 percent lower in 2013 than in 1989, according to a recent study by the St. Louis Fed.
As you might expect, millennials are the least likely to be afraid of losing all their money in the stock market: Only 3 percent report that. That makes sense because only a quarter of the age group owns stocks to begin with. Older respondents were more worried about being too poor to retire or having their identities stolen.