Goyer has met hundreds of grandparents who are either raising their grandchildren or helping support them. "When you talk to these grandparents, some of them are so stressed out that they immediately start talking about the challenges they face."
Read More6 steps to a bulletproof retirement portfolio
They may be struggling with all the costs of clothing, food, housing, education, child care and health care, she says. And they may have a grandchild who was born with a drug or alcohol addiction, have an attachment disorder, chronic asthma or other health or emotional issues and need expensive medical and mental health help.
She says grandparents often tend to focus more on taking care of the grandchildren than themselves. If the children need money for new shoes, tutoring, counseling or extracurricular activities, grandparents may skip having a prescription refilled or a doctor's appointment, Goyer says.
But even grandparents who are overwhelmed by the challenges often point out that their grandchildren are the "light of their lives," she says. They say things like, "My grandchildren are the biggest blessing in my life. I don't know what I'd do without them. They give me a reason to get out of bed every day."
Certified financial planner Brian Power, one of the owners of Gateway Advisory in Westfield, N.J., has multiple clients with adult children who have run into financial difficulties and moved back in with their parents, bringing the grandchildren with them.
Read MoreHow to not run out of retirement cash
In most cases, it's a very stressful situation, because the grandparents are withdrawing money from their retirement accounts faster than they had planned, he says. "Sometimes it puts a complete kibosh on their overall financial plan and their retirement. It usually delays the clients' retirement, sometimes indefinitely."
People either can't retire when they want, or they actually go back to work, Power says. He has one client, a 70-year-old attorney, who is continuing to work to support his divorced daughter and her two children. All three live with him and his wife.
"The bottom line is the grandfather is doing what he has to do. He'd be retired if he could be, but he doesn't feel like he can because of the outflow of money to his daughter and the grandkids."