This is the metric Warren Buffett uses to measure success (Hint: It's not money)

Warren Buffett
David A. Grogan | CNBC

Warren Buffett isn't just regular rich, he's also rich in friendship.

Whether he's eating lunch at McDonald's with his pal and fellow billionaire Bill Gates, spending time with or working out business problems with his right-hand man Charlie Munger, relationships are how Buffett defines success, according to Gates.

"Warren Buffett has always said the measure [of success] is whether the people close to you are happy and love you," Gates writes in a Reddit Ask Me Anything post.

For Buffett, dollars aren't the ultimate prize.

"Being given unconditional love is the greatest benefit you can ever get," Buffett tells MBA students in a 2008 talk.

"The incredible thing about love is that you can't get rid of it. If you try to give it away, you end up with twice as much, but if you try to hold onto it, it disappears. It is an extraordinary situation, where the people who just absolutely push it out, get it back tenfold."

And that is one of Buffett's secrets to happiness.

"I have never found anyone who has dozens of people who love them ... who is an unhappy person," Buffett explains.

In his own experience, that adage has held true.

"I get to do what I like to do with people that I love. That is happiness. I am happy day after day after day. How could I be any happier?"

Even financial success, no matter how great, is hollow without love.

"I have seen all kinds of people that they are miserable," he continues. "They have what the rest of the world may think is important, but they don't have anybody who gives a damn about them.

"If you have a family that is totally alien, and I see that all the time, often in rich families, I don't see how you can be very happy," he adds.

If that seems easy to say, coming from a billionaire, Buffett explains he could live on much less.

"I'm already happy. I would be happy with, you know — certainly with $100,000 a year," Buffett said on PBS Newshour in June.

Buffett has lived in the same house since 1958 and says, "if I could spend $100 million on a house that would make me a lot happier, I would do it. But, for me, that's the happiest house In the world. And it's because it's got memories, and people come back, and all that sort of thing."

Don't miss: Tony Robbins and Warren Buffett agree this is the biggest risk you can take with your money

Like this story? Like CNBC Make It on Facebook!

Why the secret to your success is who you marry