Wealthy Americans feel more confident about the economy and the stock market, but even millionaires worry about the impact of high health care costs in old age, and see it as one of the biggest risks to their financial security.
"It's the gorilla in the room," said Chris Heilmann, U.S. Trust's chief fiduciary executive. "There's an understanding of the increasing expense of health care, and there's an understanding of longevity."
For high net worth baby boomers, there is also real life experience. Sixty percent have had to step in to provide financial assistance to a relative, according to new U.S. Trust survey of individuals with more than $3 million in investable assets.
"While the survey doesn't specifically say it's necessarily health-related… we believe it is," Heilmann said.
Nearly two-thirds of those surveyed said they haven't done financial planning to cover the long-term costs of caring for their parents, and in some cases it has begun to impact their finances.
Read MoreHow Obamacare could hurt you at tax time
"About 37 percent are saying that they are doing it at the expense of perhaps obtaining their own financial personal goals," Heilmann said.
A medical crisis ranks among the top three things the wealthy worry will impact their financial well-being. Even for those with high net worth, long-term health care needs can quickly be burden.
Read MoreMost Obamacare enrollees lacked insurance
For an elderly person with dementia or Alzheimer's disease who can no longer care for themselves, the median cost of an assisted living and nursing home facility can run well over $200 a day, depending on where they live, according to a 2014 study by Genworth.
Yet, it's not just a problem in old age. In a younger person, a catastrophic illness such as cancer, which results in high costs and loss of income, can also impact a family's financial security.
Long-term care and long-term disability insurance plans can offer some protection. But Heilmann said planning for potential health costs can be more complicated for the high net worth individuals and families, whose wealth may be tied up in their business, and whose estate plans often include inheritance for children, and charitable giving.
Read MoreFor patients, costly hepatitis C cure is priceless
"To provide the financial resources for the eventual need from a health care standpoint, or a living longer standpoint—it can't be a one-strong guitar. It needs to be looked at on a tailored basis," he said.
Not having a plan could well put their financial health at risk.
—CNBC's Bertha Coombs