Ready for retirement? Whether Americans have a retirement savings plan plays a big role in their opinion about having an affordable retirement, according to the Employee Benefit Research Institute's 2015 annual retirement confidence survey.

But financial advisors say there are many other factors potential retirees should consider before leaving the workforce, including the timing of Social Security payments, planning for leisure time and budgeting based on unrealistic rates of return. Some retirees, they say, get these wrong and live to regret it.

Read MoreBest retirement spots for your interests

Take travel. "I've had dozens of clients that put off traveling, waiting until 'the time was right' only to let illness and other life issues prevent them from embarking on all the trips they put off for years," said Jeff Rose, a certified financial planer at Alliance Wealth Management in Carbondale, Illinois.

Avoid these six common retirement regrets.

—By Lucy Maher, special to CNBC.com
Posted 7 Oct. 2015

"Between one-third and one-half of all people retire involuntarily in any given year, usually for reasons of health — theirs or a loved one's," said Thomas Murphy, a certified financial planner at Murphy & Sylvest in Dallas. "Of those who retire voluntarily, many do so with no real understanding of how much it will cost to live in retirement or when or how the retirement money will come."

What's more, it's difficult for older Americans to return to the workforce should they want to after retirement.

A 2013 survey by The Associated Press-NORC Center for Public Affairs Research found that 69 percent of adults age 50 and older who have searched for a job in the last five years say "they have experienced a lack of available jobs, 63 percent had trouble finding jobs offering an adequate salary, and 53 percent said there were not enough jobs offering adequate benefits. Forty-five percent have experienced feeling too old for the available jobs, 43 percent have encountered employers being concerned about their age, and 32 percent were told they were overqualified."

Dipping too deep into your nest egg immediately upon leaving your job could make or break your retirement dreams.

Read MoreHow much do you really need for retirement?

"Sometimes they don't see that they've gone too far with spending until much later … and then it is too late to do anything about it," said Jason Flurry, a certified financial planner at Legacy Partners Financial Group in Woodstock, Ga. "They think they will just 'cut back' later if needed, but that's a sad way to plan for your golden years."

Instead, many advisors caution their clients to perfect the balancing act of enjoying their second act while being cognizant of the fact that they are not pulling in the same income as they did when employed.

Travel is much more enjoyable when you are active and mobile than when reliant on friends and family to get around.

"There are two times in retirement: when you are healthy and when you are not," Murphy said. "Plan accordingly. Travel and see whatever your heart desires. Plan to do so in the first five years of retirement. If you are healthy enough to travel after that, consider yourself lucky. Just make certain you don't go too wild. Budget carefully to make certain you will not run out of money."

Of course, having a plan in key, but being sensible about its ability to fund your retirement is just as important.

Read MoreHow to draw down your retirement savings

"People plan using rates of return that are not realistic," Flurry said. "Back in the 1990s, everybody was expecting 10 to 12 percent returns, conservatively, while borrowing money and hoping for even more. Obviously, that isn't sustainable."

If you think you'll be happy reading the paper and playing golf all day, you're probably wrong.

"You need a reason to get out of bed every morning, get dressed and go do something," Murphy said. "Otherwise you are likely to be miserable. Happiness in retirement is closely correlated with the number and strength of relationships you have. Men, in particular, often lose the bulk of their relationships when they stop working because most of their friends are at work. This is by far the most important area to plan so as not to 'flunk retirement.'"

A good plan of action? Have a hobby or two, a regular volunteering gig or even a part-time job to keep your mind active and yourself engaged in your community.

Most financial advisors agree that waiting as long as you can to claim Social Security benefits is the best plan for retirees that can do so.

Read MoreWhen you should file for Social Security benefits

"From my perspective, most people are better served waiting until closer to 70 to activate Social Security benefits," said Flurry. "The difference in payout even from age 68 to 70 is often as much as $1,000 per month. So, with life expectancies continuing to expand and the option to take half of the benefits paid each month as a surviving spouse, getting the most money for the longest period of time seems to make the most sense for someone in reasonably good health."