Few things are more disturbing than stories of elder abuse. But while physical abuse and neglect of seniors gets plenty of attention, financial abuse of the elderly is less visible.
Still, about one in eight of the elder abuse cases reported every year relate to financial abuse, according to the National Center on Elder Abuse. And the perps are not who you might think.
In a new study of the state of elder financial fraud, 58 percent of the people reporting financial abuse said the wrongdoer was a relative, most often an adult child.
"It's happening in the house. Somebody is borrowing money or helping themselves to things. The older adult knows it's happening, but it really doesn't stop," said Janey Peterson, an assistant professor of clinical epidemiology at Weill Cornell Medical College and the lead author of the study.
Financial abuse of the elderly in general has been growing. A 2012 survey of experts working with the elderly by the Investor Protection Trust found that 84 percent believe the problem of elder financial abuse has been growing.
Absent changes in the rate of financial abuse of the elderly, the situation is likely to worsen further as the population ages. The Census Bureau estimates that the number of people over age 65 will increase from 40 million as of 2010 to 82 million by 2040.
The Weill Cornell study found that 4.7 percent of respondents had experienced financial abuse in their lifetimes. So if the current incidence of financial fraud by family members continues, by 2040, more than two million seniors will have been financially abused by people they love.
Sometimes, the victims of elder financial abuse by family members are wealthy. In one notable case, Brooke Astor, the late New York philanthropist and socialite, was swindled out of millions by her son Anthony Marshall while she suffered from dementia before her death at the age of 105.
Erika Safran, a financial planner in New York, recalls going over a retired couple's finances and discovering $300,000 in taxable income in a prior year.
"One of the children inspired or forced them to liquidate an annuity," she said. "That money's gone poof so I don't know where it is."
Read MoreThe 'grandpa scam' that's costing seniors a bundle
Safran said she spoke to another adult child in the family, and eventually someone obtained an order of protection against the abusive adult child.
But abuse like that is not the most common kind, the Weill Cornell researchers found.
Instead, they concluded that low-income seniors are at greatest risk, particularly those in poor health, needing help with day-to-day activities, and living with one or more non-spousal relatives. If just one or two relatives live in the house, the incidence of elder financial abuse grew to 8.2 percent, and three or more non-spousal relatives took it to 13.7 percent. The people most at risk, Peterson said, are "vulnerable adults who may have people living with them because they need help."
She added that her "scientific guess" is that seniors who are cognitively impaired are also at greater risk. (They were not in the survey because participants had to be able to give informed consent.)
The comments from interviews of older adults in the Weill Cornell study demonstrate the severity of the problem. "My adult daughter changed my lease and tried to take over my apartment and stole money from me," said one respondent. "My son did not pay rent and I had to apply for welfare," reported another.
Government agencies are stepping up efforts to track financial abuse of the elderly.
Read MorePrivacy doesn't trump financial abuse of elderly
But in the meantime, there are a few red flags that can indicate a problem. If you have a relative in a living situation that fits what the Cornell Weill study described as risky, keep an eye on the resident senior's finances and the household spending.
Piles of paperwork are a red flag, Safran said. "You have no idea whether there is any money there, or they stopped opening it because they didn't like the numbers. What that really means is that someone's not taking care."
It's a good idea, she said, to ask to help an elderly relative with bill paying and other financial tasks, just to get a handle on what is coming in and going out.
Read MoreWhat to do when parents can't manage money
Peterson is hopeful that as awareness of elder financial fraud increases, reporting may pick up as well. "Older adults really need to speak up to somebody they trust," she said. But that is easier said than done. "Older adults are aware of it, and it's really difficult for them. I think they feel as parents responsible for their children, no matter how old they are. It's difficult for them to say no."
—By CNBC's Kelley Holland