Keep your good friends close and avoid the troublemakers, Munger and Buffett advised.
"You particularly want to avoid evil, or seriously irrational people, particularly if they are attractive members of the opposite sex," Munger said with a laugh. "That can lead to a lot of trouble."
Buffett added that one thing he has found very easy to do is to "hang out with people better than you."
"If you're picking associates, pick out those whose behavior is somewhat better than yours, and you'll drift in that direction," Buffett said. "Similarly, if you hang out with a bad bunch, you're very likely to find your own behavior worse over time."
Buffett recommended doing a quick check-in with yourself to see if you actually like who you spend time with. Ask yourself, "What qualities do they have that you can have if you want to?"
Likewise, "look at the people that you can't stand to be around. What qualities do you have that they have? Can you get rid of them?"
"You can do all of that a young age. It gets harder as you go along. It's not very complicated," Buffett said.
"If this gives you a little temporary unpopularity in your peer group, the hell with them," Munger added.
Like this story? Like CNBC Make It on Facebook!