Tesla may be facing growth challenges now, but to hear Musk tell the story of his earliest days with the company, Tesla was always a bit of a long shot.
"I was actually always fairly pessimistic about the outcome of Tesla, all the way from the beginning," Musk said on Sunday's "Ride the Lightning Tesla Motors Unofficial Podcast."
In those very early days, "I thought, we have a maybe 10% chance of success," Musk told host Ryan McCaffrey. "And we came very close to failing many times, so I never really thought before it actually happened that it would be that successful."
Investing in Tesla was a bet Musk made with some of the $180 million he netted selling PayPal in 2002. He invested in the electric vehicle company in 2004 and became its chairman. (He also used Paypal money to found SpaceX in 2002.)
In those early days, skepticism about Tesla was widespread and persistent, Musk reports.
"These days people don't remember just how foolish it seemed to make an electric car. We were just relentlessly denigrated and said that, you know, we are obviously going to fail and the fact that we were doing an electric car was just stupidity squared," Musk said on the podcast.
Back then, even seeing a Tesla being driven by a stranger was a thrill for Musk.
"It was quite trippy to see someone in a Model S who … I didn't know — that means they actually bought the car and I didn't know who they are," Musk said on the podcast.
"It's a great honor that somebody would part with their hard-earned money to buy our car instead of somebody else's, some other car."
Now, "the negativity continues," Musk said on the podcast, "but now that there are lots of electric cars, people don't realize that this was not the case at all back in 2008, when we went into production with the Roadster, or certainly 2012 when we went into production with the Model S."
Recently, Tesla has had tough times.
In April, the electric vehicle maker reported it had a rough fiscal quarter: Tesla reported a larger-than-expected loss and less revenue than was expected. In May, CEO Elon Musk wrote an email to all Tesla employees requesting "hardcore" cost-cutting measures.
In the summer of 2018, Musk said he was working as many as 120 hours in a week to ramp up production of the Model 3 Tesla. "The other option would have been, Tesla dies," Musk told tech journalist Kara Swisher about the long hours in October.
And for Musk, that was not an option: "Tesla cannot die," he told Swisher. "Tesla is incredibly important for the future of sustainable transport and energy generation. The fundamental purpose, the fundamental good that Tesla provides is accelerating the advent of sustainable transport and energy production."
But it certainly was a close call. Tesla came within a few weeks of going out of business as it worked to reach Model 3 production goals, according to Musk.
"It hurts my brain and my heart," Musk said in an interview with Axios on HBO in November. "It hurts. [Such an intense work schedule] is not recommended for anyone. I just did it because if I didn't do it ... there was a good chance Tesla would die."
Currently, Musk says Tesla's viability depends on its ability to spend its most recent investment of $2.4 billion carefully. Musk said in en email to staff he will "personally review and sign" every tenth page of expenses.
"This is hardcore, but it is the only way for Tesla to become financially sustainable and succeed in our goal of helping make the world environmentally sustainable," Musk wrote to Tesla employees in May.
Like this story? Subscribe to CNBC Make It on YouTube!