Call them the almost-grads.
This week, Wal-Mart Stores Vice President David Tovar had to very publicly acknowledge that the reason he was leaving the retailer was that the company uncovered he hadn't finished all the courses he needed for the degree listed on his resume.
Thousands of other "college graduates" are probably in the same boat and don't even know it, according to the findings of one nonprofit that spent four years finding ex-students who were just a few credits shy of getting a degree.
In some cases, students just hadn't filled out the last bit of paperwork or paid a library or parking fine, said Michelle Asha Cooper, president of the Institute for Higher Education Policy. Meanwhile, they were out there living their lives with a potential career killer in their past.
Read MoreGuess which industry fudges its résumés the most...
"Several of them just hadn't paid their $20 parking ticket," she said.
In Tovar's case, he said years ago he walked in the University of Delaware's graduation ceremony, then moved to New York and started a job. Months later he learned he was actually a few credits short. "I got a job and never looked back," he told CNBC earlier this week.
But now that more companies use outside firms to fact-check all resumes, little white lies can become deal breakers. Tovar's was checked when he was being reviewed for a promotion.
Read MoreHackers next big target: Your kids' college
"It's not surprising to me that this kind of thing can happen," Cooper said. "Many of the schools don't have strong data systems."
One man in Atlanta has taken his college to court, accusing it of costing him a job. Terry Boyd thought he was about to get a $150,000 job offer when he got a call from his potential employer. " 'We checked with your school. They said you didn't graduate,' " he was told, according to his attorney Marsha Mignott.
That was news to Boyd, who actually had the Morehouse College degree hanging on his wall for three decades.
The college said its records indicate the degree was awarded in error because he received a D, not a C in an organic chemistry class—a claim Boyd disputes, and noted he never could have advanced to Chemistry 2 if he had indeed gotten the D.
"Sloppy record-keeping" by the college is likely the cause, Mignott told CNBC. The Morehouse case is still in court and the college denies it's at fault.
Cooper, whose organization spent four years successfully helping 4,500 former students who were nine credits or less away from graduating through its Project Win-Win, said many colleges hobble students by requiring them to opt-in to get the diploma. The extra step—usually paperwork specifically asking for the degree, along with a check—sometimes gets lost in the shuffle and many students assume this step happens automatically.
When her team contacted ex-students saying the Win-Win program wanted to help them finish their degree, some thought it was a scam because they thought they already had their degree.
A full 17 percent of the U.S. population has some college but no degree, Cooper said.
Read MoreThink twice about faking your resume
Financially, the difference between some college and an actual degree adds up over a lifetime, said Jeff Strohl, the director of research at the Georgetown University Center on Education. On average, over a lifetime, the person with an associate's degree will make $200,000 more than someone with "some college." A person with a bachelor's will make $700,000 more over a lifetime than the person who didn't finish a degree, Strohl said.
Strohl also wasn't surprised that Tovar found out too late that he didn't have all the required credits. But it's hard to determine how many other people are in that same boat, compared with the number of people who intended to drop out of college, often due to financial or academic problems. Roughly, he estimated it could be between 4,000 and 20,000 people a year who have not actually graduated from college due to an "oops factor."
Many of those may never know it until a company such as the National Student Clearinghouse, a nonprofit, tries to verify information. On its website, the company claims that "41 percent of applicants lie about education."
Of the 31 million students who enrolled in college within the past two decades but didn't get a degree, 4 million made at least two years of progress, according to a National Student Clearinghouse Research Center report.
"If you go out and help (the almost-grads) during a recessionary economy, those people go from barely making it, to better off," Strohl said. "It makes a difference."
—By CNBC's Amy Langfield
(CORRECTION: National Student Clearinghouse is a nonprofit. It has not urged colleges to track down their students who are just shy of graduation, as was stated in an earlier version of this story.)