Warren Buffett describes a pivotal moment when he was 20 years old that changed the course of his career

Warren Buffett bought his first stock at 11

It's no secret that Berkshire Hathaway CEO Warren Buffett has a knack for investing. After all, he's been in the market for 76 years.

But it was a trip Buffett took to Washington, D.C. one Saturday afternoon as a 20-year-old Columbia University grad student that changed the course of his investing career.

Buffett's early passion for reading led him to read through every book the Omaha Public Library had on investing by age 11 and through the ones in his father's office, he tells Forbes magazine.

At 19, Buffett accidentally bought "The Intelligent Investor" by investor Benjamin Graham, who would later become Buffet's hero and mentor. Buffett re-read Graham's book about half a dozen times, he says, inspiring Buffett's investment philosophy and strategy to this very day.

These billionaires still drive these cheap cars

"My favorite investment, one that embodies this philosophy, is Geico, which I learned about when I was 20 years old," Buffett tells Forbes.

At Columbia Business School, where Buffett became a student of Graham's, he visited Columbia library on Saturday in 1951. There, he learned that Graham was chairman of Government Employees Insurance Company, or GEICO, he tells The Wall Street Journal.

"A week earlier, I knew nothing about the company," Buffett says. Still, he wasted no time locating Geico's headquarters. He got on a train and left New York City.

"When I arrived in Washington on that Saturday morning, I went to Geico's office and was distressed to find out that the doors were locked," Buffett says. "But I pounded on the door and finally a janitor let me in."

Warren Buffett's partner Charlie Munger shares his advice for living a rich and successful life

"Unlike the Omaha of the 1950s, Saturday wasn't a workday in Washington," he adds.

According to Buffett, the janitor directed him to Lorimer "Davy" Davidson — the only other person working in the office that day — who would later become CEO of Geico.

"Davy had no reason to talk to me, but when I told him I was a student of Graham's, he then spent four or so hours answering unending questions about insurance in general and Geico specifically," Buffett says.

"He answered my questions, taught me the insurance business and explained to me the competitive advantage that Geico had," Buffett says. "That afternoon changed my life."

Like this story? Like CNBC Make It on Facebook.

Don't miss:
Elon Musk's mom Maye Musk just made history by scoring this modeling gig

How Warren Buffett's optimism has helped him succeed, according to psychology

Warren Buffett is worth $75 billion but says he would be 'very happy' with way less