Here's how a family of 3 gets by in San Francisco without anyone working a job

Sam Dogen and his son play in Golden Gate Park, San Fancisco.
The Financial Samurai
Sam Dogen and his son play in Golden Gate Park, San Fancisco.

If the amount of money you're saving each month doesn't hurt, you're not saving enough.

I've told myself this saying every month since I was a first-year analyst working in finance in 1999. The saying came to me one evening at 9 pm while I was still at the office, waiting for a Hong Kong-based research analyst to get back to me with some data for a client. I had already been at the office since 5:30 am. Unfortunately, the analyst never got back to me.

Because the hours were so brutal, working in New York City toughened me up. I knew there was no way I could continue working 70+ hour weeks for decades, so I had to figure out a way to eventually escape. The only solution I could come up with was to save as much as possible and reinvest the proceeds in income-producing assets.

Here's my story on how my wife and I finally escaped the grind at the age of 34 for good. Today, I'm 41 and she's 38, and we now have a 16-month-old baby boy who we both take care of full-time because we live entirely off our passive income.

Essentially, I escaped full-time work, for good, at age 34, by following these six steps.

Step 1: I planned ahead

Sooner or later, most people will become miserable doing whatever it is they do for a living. The key is to foresee your misery so that by the time you are actually sick and tired of your job, you'll already have the financial cushion to make that change.

For me, the biggest upside to working long and stressful hours straight out of college was recognizing very quickly I could not last in such an environment for decades. Therefore, I cut costs to a minimum, saved every other paycheck and 100 percent of each year-end bonus, and invested in as many passive income assets as possible for 13 straight years.

Here are some specific things I did in my 20s and early 30s:

  • Lived in a studio after college with another guy for two years
  • Regularly stayed at work until after 7:30 pm to take advantage of the free cafeteria food
  • Lived in one of the sketchiest parts of SF with two other people in a two bedroom, one bath, apartment, despite a raise and a promotion
  • Vacationed locally, instead of internationally, in my 20s because work would send me to Asia for conferences two to four times a year
  • Drove beater cars worth less than 10 percent of my gross income
  • Aggressively invested all my money in San Francisco real estate because I needed to get my living costs down

Step 2: I negotiated a severance

13 years after I started my career in finance, I left my job by negotiating a severance equal to roughly six years worth of living expenses. At the time of my departure in 2012, I was a 34-year-old executive director who had accumulated four years worth of deferred compensation in the form of cash and stock.

As part of my severance, I was able to keep all my deferred cash and stock compensation, get paid three months of federally mandated WARN Act pay, receive six months of fully paid healthcare, and receive a six-figure lump sum severance check.

Without a severance, I wouldn't have had the guts to leave my job so early. But the severance effectively bought me six years worth of my life back. Given that time is priceless, I figured why not take the leap of faith. If money started getting tight, I could always get a job again.

Step 3: I made sure I had enough passive income to survive

When I left my job, not only did I have regular income coming in as part of my deferred compensation, I also had passive income that would help tide me over should my employer decide to renege on my severance. After all, my severance was to be disbursed over a period of five years and, during this period, the company could change its mind or go bankrupt.

In 2012, my passive income consisted of CD interest income, dividend income from stocks, and rental property income from a two bedroom condominium I bought in 2003. I had a large amount of CD income because long-term CDs back then were paying 4-4.125 percent.

By saving 50-75 percent of my after-tax income for 13 years, I was able to invest the proceeds and amass roughly $78,000 in passive income. There were times when I questioned the wisdom of sharing a studio with a friend and driving beater cars worth less than 1/10th my gross income for a decade. But once I was able to experience freedom, it made all these sacrifices worth it.

With the 10-year bond yield at ~3 percent, know that every $1,000 you save today has the potential to generate at least $30 a year in risk-free income tomorrow. If you take on a little more risk, it's conceivable that your $1,000 could end up generating $40-$60 a year. Start your passive income journey as soon as possible because it takes a long time to build something significant.

Step 4: I used my free time wisely

If you're able to retire early, please don't waste all your free time slacking off. Sure, go travel around the world for several months, play tennis in the middle of the day and take a siesta after a particularly hearty meal. Get it all out of your system.

Eventually, you'll want to get back to work doing something you love to do. If you have a severance and some passive income, you can afford to take lower paying jobs that may pique your curiosity. For me, after my first year of freedom, I decided to consult part-time at several financial technology startups in the San Francisco Bay Area for about 10-20 hours a week at each start-up over different time periods.

There I built online marketing know-how and new connections for my true love: connecting with people online through my personal finance site, Financial Samurai. Ever since I was 12, I have been writing to pen-pals from across the world.

I started Financial Samurai during the depths of the financial crisis in 2009 after I had lost about 40 percent of my net worth in a matter of six short months. It was cathartic to connect with others around the internet who had also been devastated by the crisis.

Once I left work, using my newly acquired free time, I began to regularly publish new articles three times a week. The topics ranged from investing in real estate to discussing international equities to highlighting other severance negotiation case studies in order to retire early.

It turns out that if you regularly work on something for years, good things happen. After almost 10 years, Financial Samurai not only continues to be my favorite hobby, it also generates some supplemental income to help us continue building more passive income in order to remain free.

Step 5: My wife opted out too

A couple of years after I engineered my layoff, I suggested my wife do the same. She is three years younger. We agreed that if everything worked out on my end, by the time she turned 34, she too, could leave her day job. Life is so much better spending it with someone you love.

After nine years of working at the same old job, she was getting tired of all the office politics and the occasional difficult client. She had recently been passed over unfairly for a raise and promotion. As a result, she no longer had any motivation to continue. She wanted out. Fortunately, six months later she got her promotion and began taking steps to figure out her exit.

For our combined sense of security, it was nice to have my wife's steady income. Getting on her company's health insurance plan was also a great benefit. But after two years, we had built up more passive income and felt pretty certain that both of us wouldn't need to work if we negotiated a severance for her as well. As for health insurance, we would just buy our own to the tune of $1,300 a month through our online business. Painful, I know.

In the end, my wife was able to get a really unique severance package that allowed her to work two days a week for the same pay for five months during her transition out. The more valuable you are as an employee, often the easier it is to negotiate something good.

By 2015, our passive income was beginning to generate roughly $150,000 gross a year, enough for my wife and me to live a decent life in San Francisco.

Step 6: We took advantage of our freedom

Once my wife engineered her layoff, we traveled for eight weeks a year across the world. We hit many of our bucket list stops such as Angkor Wat in Cambodia and St. Petersburg in Russia. Our goal was to travel as much as possible until we no longer wanted to travel for years! Why? Because we finally decided we were ready to start a family.

After several years of trying, our son finally arrived in the spring of 2017. Between the time my wife left her job and our son was born, we continued to save at least 50 percent of our after-tax income from our online business by living super frugally. We bought a smaller house in a less expensive neighborhood in 2014 and rented out our larger, more expensive house that same year, thereby lowering our expenses by roughly 40 percent.

In 2017 I also got my final after-tax severance check from the employer I left in 2012. It almost felt as though my employer had given us a gift for our son since the check arrived just days before he was born.

Being a landlord in San Francisco had been painful. We could only find 4-5 single guys interested in renting our house. This inevitably led to a lot of damage and tenant turnover. After we realized a child would require a tremendous amount of personal attention to care for, we decided to sell our rental house in 2017.

We had tried to sell the house in 2012, the year I left my job, for $1,700,000 and gotten no takers. This time, we were thrilled to be able to find a buyer who paid $2,740,000, or 30 times our annual gross rent!

We reinvested $550,000 of the proceeds in real estate crowdfunding to take advantage of cheaper valuations with higher net rental yields in the heartland, $550,000 in bonds, and $700,000 in various stocks and index funds. We went from what had been a lot of hassle to earn $60,000 a year in net rental income to completely hassle-free passive income with less risk exposure.

The sacrifice is worth it

In order to live off $213,000 a year in gross passive income in expensive San Francisco, we own a relatively humble 1,920 square foot, three bedroom, two bathroom home and drive a car worth less than 1/10th our gross income. We never buy any new clothes and we take full advantage of the free things the city has to offer during the weekdays.

$213,000 a year might sound like a lot to you, but the median home price in San Francisco is roughly $1,600,000, or almost eight times our annual passive income. For a family of three in 2018, the Department of Housing and Urban Development [HUD] declared income of $105,700 or below as "low income." Therefore, I consider us firmly in the middle class.

We will continue to save and invest in more passive income generating investments because we have a potentially huge expense coming up if we don't win the San Francisco public school lottery. That's right: Even if you pay $20,000 a year in property tax for a decade, your child has no guarantee of going to your neighborhood's public school.

Private school tuition in the city costs around $15,000 - $35,000 from K - 8th grade, and $20,000 - $45,000 for high school in today's dollars. As a result, we've got four years before he's eligible for kindergarten to try and earn another $30,000+ a year in passive income.

Although we forewent many luxuries after we graduated from college, there is not a day that goes by where we aren't thankful for being able to leave our jobs for good at 34.

To us, the sacrifice was well worth it! And I strongly believe the sacrifice will be worth it to you as well.

Sam Dogen started in 2009 as a way to cope with the financial depression. In 2012, he negotiated a severance after 13 years in finance in order to be free. He now spends most of his time taking care of his baby boy, writing about personal finance and playing tennis.

Like this story? Subscribe to CNBC Make It on YouTube!

Don't miss: Young people want to retire by age 60—but experts warn the 'new retirement age' is 70