FA Playbook

So your pet project took off. Time to ditch the day job?

Coeli Carr, Special to CNBC.com
Delayed retirement dilemma

Keith Fowler's pumpkin wine has been a big hit with the three dozen or so people who've enjoyed samplings.

"My testers have told me they'd love to have it through the fall months and would pay a good price for it," said Fowler, who lives in Westfield, New York. His eight-acre property, which he bought in 2009, is home to 500 vines producing Traminette and St. Vincent grapes, as well as a bunch of blueberry, raspberry and, yes, pumpkin patches.

Tegra Stone Nuess | Getty Images

The twist? Fowler's tasters, who serve as informal market-research subjects, are also his work colleagues at Bush Industries, a furniture manufacturing company, where the burgeoning vintner is director of information technologies.

That's right: Fowler still has his day job, which he loves, but he concedes it's keeping him away from his viticulture.

Read MoreBoomers face employment woes

As more people in their 60s come to regard retirement as nothing more than a stepping stone to new professions and money-making ventures, a unique dilemma has arisen.

What happens when a "side endeavor"—already set in motion while one is working full-time—takes off faster than expected? Should you stick it out at a bread-and-butter job until the last possible moment or toss 9-to-5 security to the wind?

Delaying gratification

Fowler, 63, would like nothing better than to put all his time and energy into wine-making, which he and his wife, Heidi, a homemaker, see as their entrepreneurial focus after he retires from his position in about two years.

But because he has relied on using disposable income from his salary to buy vines—which he expects to more than double in number this year—trellises, equipment and supplies, he's understandably reluctant to give up his funding source.

Read MoreFinding your encore career

As part of governmental compliance requirements that Fowler must fulfill to sell product—his company, which produced about 100 gallons of wine this year and does not yet have an official name—he'll have to invest about $30,000 in new infrastructure, including converting his basement and barn into part of a bonded warehouse.

"I'm really in an 'if-only-I-could' situation, but I can't leave full-time employment right now," said Fowler, who's also a U.S. Marine Corps veteran and has another entrepreneurial effort going: military memorabilia company WhisperingHope.net, which was launched in 2006 but does not generate enough income to provide a livelihood. "I'll be more ready to retire when I have my safety net in place—my Social Security, other pensions and the ability to take IRA distributions—but not until then," he added.

Searching for a job at 50+

Experts say that older individuals' unwillingness to leave lucrative day jobs, even as their start-ups take flight, can be complicated situations that would benefit from the input of both psychologists and financial professionals.

"It's a big decision involving future income and wealth creation, so there are long-term financial, as well as emotional, consequences," said Neal Roese, a psychologist and professor of marketing at Kellogg School of Management at Northwestern University, whose specialty is the process of decision-making.

Read MoreFacing retirement health costs

Roese, author of self-help guide "If Only: How to Turn Regret into Opportunity," believes there's always the chance that, by not leaving a full-time job when the entrepreneurial iron is hot, a person could truly miss out on an opportunity. "The market could change," he said. "If your new business introduces a brilliant idea, other companies now have the ability to capitalize on it and get a foothold.

"Suddenly, when you're ready to jump in, it's become a crowded space."

The planner perspective

However, sometimes even jumping in under the best of conditions can lead to mayhem.

"We have seen clients who, after they go entrepreneurial while they're still at a full-time job, merge their personal and business money by, say, taking $5,000 out of their retirement fund because of a sudden business expense," said Karen Altfest, certified financial planner and executive vice president of Altfest Personal Wealth Management. "They don't keep income records for their enterprise and instead try to keep a running account of those numbers in their head."

Read MoreAdvisors like emerging markets

Some of this behavior, she said, is motivated by the person's lack of certainty—not knowing whether their new venture is part-time or how long it will last. "Proper bookkeeping—and a business plan—keeps it real, and some entrepreneurs don't want to put themselves out on a limb," added Altfest.

"It's not a good idea for older side-entrepreneurs to quit their full-time jobs until they're aware of how heavily their employer is subsidizing a good portion of their expenses, such as health insurance, Social Security and other benefits," said Jerry Barbalatt, a CFP and president of independent financial planning and insurance/investment consulting firm Parker Allen & Company. "It takes effort, understanding and the discipline of good bookkeeping to help replicate the financial regularity one has gotten used to."

What it invariably comes down to is your risk tolerance, said CFP and psychologist Bradley Klontz, a partner at Occidental Asset Management and co-founder of the Klontz Consulting Group's Your Mental Wealth and Financial Psychology Institute initiatives. "When people start an entrepreneurial venture while working at a steady job, it tells me they're somewhat risk-averse and hedging their bets,in case the new business fails," he said.

For Klontz, the culprit is often a person's "money script," a set of beliefs based on childhood experiences around money that shape the way we think about and interact with money as adults.

Read MoreDon't 'robo' your retirement

"Once you've made these deep-seated money scripts conscious, which allows you to understand your fears about leaving your full-time job, you are in a position to let a financial professional help you look at your situation objectively," he said.

Klontz said he believes it's worthwhile sharing psychological issues with a financial planner, accountant or coach. "You can say, 'I'm aware I have an exaggerated fear of failure and a lot of anxiety around not having enough money because I grew up poor, but I believe this is a really good opportunity and I'd like another set of eyes to help me make a wise decision,'" he said. "The goal is to see if your fears are legitimate."

Once you've made these deep-seated money scripts conscious ... you are in a position to let a financial professional help you look at your situation objectively.
Bradley Klontz
partner at Occidental Asset Management

Judith Sills, a clinical psychologist in private practice in Philadelphia and author of "The Comfort Trap: Or, What If You're Riding a Dead Horse?" has her own way of addressing these safety-net issues.

"This is a quantifiable decision, which makes it non-emotional, and so my client and I literally sit down with a paper and pencil and do the math," she said. "We talk about how they could slow down to a 40-hour work week and apply that extra time to the entrepreneurial effort."

Read MoreHow to not outlive a nest egg

James Gottfurcht, clinical psychologist and founder and president of Psychology of Money Consultants, said he has seen a steady increase in questions at his workshops about when to leave a job.

"It's a dilemma, and there is no one formula to solve it," said Gottfurcht, who advises clients to ask themselves what it would mean to them personally if their new venture succeeded or failed and how they'd feel about it. "A person's response to this self-query often provides valuable insights toward resolving the dilemma."

—By Coeli Carr, special to CNBC.com