This tech CEO started out as a high school janitor — here's the No. 1 thing learned about getting ahead

Today, Mark Testoni is the CEO of SAP National Security Services, a cyber security firm where he manages a team with over 500 employees. He's responsible for the company's sales, products and strategy, and coordinates with the U.S. government about matters of national security.

But when he was just starting out his career, he felt lost.

"I was kind of an indifferent high school student, largely because I didn't have a lot of confidence in my own abilities," Testoni, 62, tells CNBC Make It. "Just wasn't a very confident guy."

He headed to college in the mid-70s at Towson University outside of Baltimore, Maryland, but couldn't quite find his place there either. He dropped out, tried going to community college, and dropped out there too. Then his dad sat him down for a talk. It was time to get a job.

"My dad was a principal in the public school system and he said, 'I'm going to work on finding you a job,'" Testoni remembers. "My dad, being sly as a fox, found me a job: as a janitor in a high school."

As a 20-year-old, Testoni found himself mopping floors, scrubbing bathrooms and taking out the garbage for students who weren't much younger than he was.

"When you clean high school students' restrooms, it gives you a lot of perspective on life," Testoni says. "Although I took pride in my work, it wasn't something that I wanted to spend the rest of my life [doing]."

However, the experience taught him an important skill that would lead to success later in life: discipline.

"I got to learn how to show up to work every day, punch a card, and if I was two minutes late, they docked my pay. I started to learn the principles of personal responsibility and accountability," Testoni explains. "It was actually what was the beginning of a renaissance for me personally."

That new-found motivation led Testoni to the military, where he enlisted in the U.S. Air Force in 1977. "I knew that was a way I could go to college," he says, and get another shot at pursuing higher education. "I was looking for opportunities."

The structure of the military was exactly what Testoni needed to succeed. After serving 20 years in the Air Force and earning an undergraduate degree from the University of Tampa along with a M.B.A from Southern Illinois University, Testoni spent nine years as an executive at Oracle, helping the company partner with the U.S. Department of Defense. Then he became the CEO of SAP National Security Services in 2011.

Over his decades long career as a technology executive, Testoni has seen again and again how important discipline is for success.

In his experience, the people who get ahead have one trait in common: "When they say they're going to do something, they do it," he says. "Or they may not accomplish it, but they try like hell to do it. You have to make commitments and show up and meet them."

And discipline is something he focuses on in his own daily routine. "I still fold my underwear the way that I was taught in basic training 41 years ago," Testoni says with a laugh. (He folds the left side in, then the right side and rolls them up.) "It's little things like that. If you build discipline into your life you'll build discipline into your work."

For another example, he takes time every morning to list out his priorities for the company that day. "I take stock in the morning, and I often do it as I'm getting ready," he says, asking himself, "What are those three or four things that I have to accomplish today so at the end of the day we are a better company?"

That habit started with his work as a janitor. At that time, he would ask, "What are the three things, between what the boss is telling me and other things I've committed to, what are the three or four things I'm going to get done today while I'm pushing the broom down the hall?"

Now as CEO, Testoni has an eye out for a new generation of young people driven to succeed: "The successful ones, they have that focus and discipline."

Don't miss: How this Ohio janitor built himself a multimillion-dollar oil empire

Like this story? Like CNBC Make It on Facebook