Unexpected outlays for health emergencies were cited as the second-highest worry with regard to retiring early, with 46 percent of respondents.
That was topped only by the prospect of outliving their money, with 53 percent. Other top concerns include an economic downturn, at 35 percent; Social Security changes, 33 percent; and inflation, 29 percent.
Americans would be wise to take action and make contingency plans for the unexpected, said Matt Sadowsky, director of retirement and annuities at TD Ameritrade.
"You might think that you're OK and you have your healthcare planned for, but are you planning for the average health-care costs?" Sadowsky said. "Because you might not be average."
That includes thinking of how you will handle unexpected bills tied to your health and long-term care, as well as that care for your loved ones, he said.
More from Fixed Income Strategies:
Social Security strategies for bigger checks in retirement
Now may be the time to use bonds in portfolios
Social Security can buy luxury living in five overseas spots
Most respondents — 76 percent — said they hope to cover their retirement health-care costs with Medicare, although many also said they are not confident Medicare will be there for them in their later years.
"Especially for those who have not yet retired, confidence in Medicare is lacking," Sadowsky said.
Other popular ways to cover health-care costs in retirement, according to survey respondents, include supplemental health care insurance and Social Security.