But to hear Ma tell it, his success has nothing to do with technical or business skills.
"I know nothing about technology, I know nothing about marketing, I know nothing about [the legal] stuff," Ma said at the Viva Tech conference in Paris on Thursday.
So what does he know that's made him successful? "I only know about people."
"Everything we do as a founder: Make your customer happy. Get your team. Make your team happy. And you will be happy," Ma said.
Indeed, Ma, who originally planned to be a teacher, backed into the idea of starting a tech company after he first encountered the internet in 1995 on his first trip to the United States.
"[My friend] said, 'Jack, this is Internet.' I asked, 'What is Internet.' He said, 'Search whatever you want.'"
At first, "I don't want to touch because computer is so expensive in China. If I destroy it, I cannot pay," Ma said. But Ma "searched the first word 'beer.' I don't know. It's easy to spell," Ma told Rose. Ma then searched the word "China." There was "no data about China," said Ma.
"So I talked to my friend: Why not I make something about China? So we made a small very ugly looking page called 'China,'" Ma told Rose. It was the genesis for setting up Alibaba, which started as an "e-marketplace for information," Ma said, and developed from there.
But despite his lack of technical skills, as the company grew, Ma said he did know one thing: "I know people. I was trained to be a teacher," Ma said at Viva Tech. "As a teacher, you want the students better than you are. Your best product is your students."
That skill helped Ma inspire and encourage his employees, and that has lead to huge success for his company.
"You have to listen to them… You have to do all the things to making sure that you can empower them, you can make them achieve, team work, you can then collaborate. So, this is how we do," Ma said
"I found that the more you care about your team, the team will care about you and the team will care about their customers," Ma said.
And the customer is the bottom line: "I talk to all of my people, early days: 'Don't make me happy, don't love me. Make your customer happy, make your customer love you, and I will love you any minute.'"
Like this story? Subscribe to CNBC Make It on YouTube!