I quit my job at 34 with $3 million—here are my 5 biggest regrets about early retirement

Why is it that the younger we are, the more we're in a rush to get things done? We want to graduate faster, get a job faster, get rich faster and retire faster. Up until my early 30s, that was always my mindset.

And in 2012, I finally retired at 34. By the time I quit my job, I had amassed a net worth of about $3 million that generated roughly $80,000 in investment income per year.

Don't miss: The best credit cards for building credit

So far, it's been an enjoyable experience. The freedom to wake up whenever I want and do whatever I want is priceless. I no longer have to deal with long commutes to work, 60-hour workweeks and office politics. I get to focus on my hobbies and spend more time with my wife (who also retired early) and son.

But no situation is perfect. If I could go back in time and retire all over again, here are five things I'd do differently:

At 42, I now realize how absurdly young I was when I retired. Several people even commented on how irresponsible and reckless my decision was, especially because I was just entering my peak earning years.

But I had been with my firm for an 11 consecutive years; I was tired, bored and wanted to do new things.

Looking back, I could have stayed for at least another year and found a new role within the firm in a different office. I had always wanted to work overseas — someplace in Hong Kong, Taiwan, Beijing or London. Maybe it would have rejuvenated my interests and convinced me to work a few more years.

I would also put 100% of the extra money earned into various risk assets like stocks and bonds. Assuming a 4% annual return, I could have generated an additional $20,000 or more in passive income per year.

Instead, I went straight to negotiating a severance — and even burned valuable vacation days before my departure!

My wife and I delayed our decision to become parents because we were so focused on our careers. I also didn't think I was mentally prepared to be a dad without first having an enormous financial buffer.

I was 37 when we decided we were ready. My wife, who was 34 at the time, had just negotiated her severance for early retirement. After two years of trying, I finally became a dad at 39.

In retrospect, I would have preferred to be a first-time parent in my early 30s. You could argue that I'd have less time to spend with my son because I'd be in the thick of work. But the way I see it, getting to spend a greater percentage of my overall life with him would be priceless. (Now I just hope I can live long enough to watch him grow up.)

By waiting so long, I couldn't take advantage of my firm's incredibly generous — even by international standards — parental leave benefits: 20 weeks of paid leave (plus paid nannies for those who have to travel for work).

As a result, we missed out on tens of thousands of dollars and had to pay more than $1,700 a month in healthcare premiums on our own.

In 2006, I came up with the idea to start a personal finance website called Financial Samurai.

Unfortunately, it didn't happen right away; I was too busy trying to climb the corporate ladder for bigger titles and higher salaries. (How could I not? The correlation between effort and reward was so strong.)

But after the financial crisis in 2008, it became clear that no matter how well I performed, I'd no longer get paid or promoted at my previous pace. Instead, my career would remain stagnant for a while and I'd actually get paid less.

So in 2009, I finally launched Financial Samurai. At first, it was mostly a way for me to make sense of the financial chaos. But over time, writing about real estate, investing, career strategies, family finances and retirement became a hobby.

Since launching, millions of readers have visited the website to learn and share insights on how to reach financial independence.

Today, Financial Samurai is an established website, and its rapid growth continues to produce a decent amount of income. Still, I know that if I started it sooner, in 2006, it would have been much larger and more successful today.

I was so fixated on planning my retirement in 2012 that the idea of leveraging up to buy more property and stocks never crossed my mind. (We even tried to sell our primary residence that year. Luckily, it didn't happen because the market was so soft).

If I worked a few extra years before retiring, I would have had the financial confidence to buy more real estate in 2012, right before prices began to take off. (A rental property in San Francisco that cost $900,000 in 2012 would be worth roughly $1.6 million today.)

(Credit: Bay Area Market Reports, Compass)

Had I continued working through the bull market, I probably could have made more than $250,000 in returns, based on my investment cadence at the time.

Retiring early made me excessively risk-averse — and I paid the price.

After 13 years of working in finance, I no longer found it interesting — and so I saw retirement as my way out.

Looking back, I wish I took the time to explore different industries. I often toyed with the idea of joining an innovative tech startup with explosive growth. After all, I was living in San Francisco, the tech capital of the world.

But because I was rushing to negotiate my severance, I didn't even think to apply to companies like Uber, Airbnb and Pinterest.

I also could have leveraged my interests in real estate and technology to start a real estate crowdfunding company — or, at the very least, join one. I still believe that real estate is one of the most straightforward ways most Americans can build wealth over the long term.

Things could always change in the future, but right now, I feel too old and exhausted as a stay-at-home dad to pursue any of those dreams.

No matter how much you plan for your future, you'll never get it perfectly right. But the more time you spend thinking about your options, the better off you'll be.

So if you're set on retiring early, I suggest waiting it out until you turn 40 (or at least for another year). Maybe take a sabbatical or extended vacation to recharge if you're feeling burned out.

And once you do decide to make the jump, make sure you have enough investment income to cover your desired living expenses (plus a 20% buffer in case things change).

Lastly, you should know beforehand what you'll be doing with your newfound time. Otherwise, you might find yourself bored and unsatisfied in retirement.

Writing on Financial Samurai has kept me busy and happy. I also started a podcast to keep things interesting. I figure my boy might get a kick out of listening to his old man once I'm gone.

Sam Dogen worked in investing banking for 13 years before starting Financial Samurai, a personal finance website. He received a B.A. in Economics from The College of William & Mary his MBA from the University of California in Berkeley. Sam has been featured in Forbes, The Wall Street Journal, The Chicago Tribune and The L.A.Times.

Like this story? Subscribe to CNBC Make It on YouTube!

Don't miss:

• I retired at 34 with $3 million—here are 5 downsides of early retirement that no one tells you
• 61-year-old self-made millionaire: 4 things I regret wasting money and time on in my 20s
• What to do in your 20s, 30s and 40s if you want to retire wealthy, says money expert