Here's why Warren Buffett says that he and Charlie Munger are successful

Warren Buffett is the third richest person in the world, worth an estimated $85.9 billion, and his business partner Charlie Munger is worth an estimated $1.71 billion. Buffett and Munger got to where they are today in part thanks to hard work and dedication to their craft. But Buffett also acknowledges that they also got a major head start because they, as he puts it, won the "ovarian lottery."

In a clip found using CNBC's Warren Buffett Archive, Buffett explained the concept of the "ovarian lottery" at Berkshire Hathaway's Annual Shareholders Meeting in 1997 to mean that many of the advantages you get in life, such as your nationality or your health, are determined by chance.

The topic surfaced in a roundabout way after Patrick Byrne, the CEO of a New Hampshire-based manufacturing company, asked the duo their thoughts on the rate at which capital gains should be taxed. Buffett replied that they should be taxed heavily and that those with the means to pay aren't going to be "turned off from using their talents by a 28 percent capital gains tax," which is what the rate was in 1997.

To justify his answer, Buffett laid out a hypothetical scenario: It's 24 hours before you're born and you've been granted the ability to decide all of the economic rules for the society you're able to enter. The catch is that you don't know any details about what your place in the world will be. Your gender, race, nationality, health, intelligence level and all other defining characteristics are left up to chance.

"You're going to get one ball drawn out of a barrel that probably contains 5.7 billion balls now, and that's you," Buffett says.

With your own circumstances unknown, "what kind of a society are you going to construct?" Buffett asks.

In many way, Buffett says, he and Munger won the real-life version of the ovarian lottery and that helped launch them into their immense success. "I mean, Charlie — when we were born the odds were over 30-to-1 against being born in the United States, you know? Just winning that portion of the lottery, enormous plus," he says.

He also points out that he and Munger were both born male and white, which gave them further advantages from the start.

He even believes it's luck that blessed them with a sharp eye for business. "We won it in another way by being wired in a certain way, which we had nothing to do with, that happens to enable us to be good at valuing businesses," he says. "And, you know, is that the greatest talent in the world? No. It just happens to be something that pays off like crazy in this system."

Buffett borrows this concept from John Rawls, a philosopher at Harvard who laid out similar ideas in "A Theory of Justice." In this work, Rawls explains that a truly just society can only be created if it's based on principles that everyone agrees to before knowing where they'll land within it.

Buffett agrees: "The trick, it seems to me, is to have some balance that causes the people who have the talents that can produce goods that people want in a market society, to turn them out in great quantity, and to keep wanting to do it all their lives, and at the same time takes the people that lost the lottery and makes sure that just because they, you know, on that one moment in time they got the wrong ticket, don't live a life that's dramatically worse than the people that were luckier."

Nearly 10 years after giving this speech, Buffett still holds many of the same beliefs. "I don't think I need a tax cut," he told CNBC earlier this year when the Republican tax plan was released.

He specifically spoke out about the estate tax, which is levied on assets transferred from one person to another at the time of death. He said eliminating it would be a "terrible mistake" because the current system in America vastly favors the rich.

"I sure don't think it's good for a society where there's a ton of inequality to start with. I think that's a terrible mistake," he added.

Don't miss:

• A 36-year-old who learned to invest like Warren Buffett explains Buffett's No. 1 rule
• Warren Buffett just won a $1 million bet—and highlighted one of the best ways to grow wealth
• A 36-year-old who learned to invest like Warren Buffett explains how saving can actually cost you money

Like this story? Like CNBC Make It on Facebook!