"Typically the general rule [in retirement] is not to treat home ownership as an investment," said Pete Lang, a retirement, investment and tax advisor and founder and president of Lang Capital in Fort Mill, South Carolina. "It's better viewed as a living expense."
That means it's important to look at the monthly net cost of housing rather than size alone.
"For instance, you may have a couple who live in a rural area with a five-bedroom home who decides to downsize to a two-bedroom condo in the city. It is very possible that it would cost more, ... and may have additional costs such as condo fees," said DeDe M. Jones, a managing director of Innovative Financial in Lakewood, Colorado.
The high costs of housing in markets like California or Connecticut consume as much of 25 percent of retirees' income, said Jonathan Smoke, chief economist for Realtor.com, which analyzed searches of listed properties by 65- to 74-year-olds.
Downsizing provides less opportunity in low-cost areas, especially in the Midwest, South and many areas of the Northeast. In those areas home prices, taxes and insurance tend to be lower, so retirees won't see any financial gains from moving, Smoke said.
For others, there are considerations about estate planning and capital gains.
"A large capital gain might negate the benefit of a downsize," said Jim Ciprich, a Morristown, New Jersey-based wealth advisor for Regent Atlantic.
For example, a couple who bought their home in 1970 for $80,000 that is now worth $800,000 could be subject to a big capital gains tax on the proceeds. The IRS excludes capital gains of $250,000 for an individual or $500,000 if married and filing jointly. Anything above that amount is subject to a 15 percent capital gains tax. If a couple already has over $466,950 in ordinary income in the year they sold their home, they are subject to a 20 percent tax.
For that reason, if a retiree plans to pass down the property, it may be better to have the gain stepped up at death depending on the age of the retiree.