Duration matters: The longer someone was out of work, the more likely it is that they've quit looking.
Of the total, 55 percent who were unemployed for more than two years fell into the category; 32 percent of those idle for 13 to 24 months and 34 percent out for seven to 12 months had quit as well. Just 21 percent out for three months or less had stopped looking.
Read MoreHow to protect your career from the next crash
Overall, nearly 1 in 5 (19 percent) said they spent no time looking for work in the week previous to the survey. Just 10 percent said they spent more than 31 hours looking.
Unemployment compensation also matters.
Federal guidelines allow for 26 weeks of unemployment compensation, though extended benefits are available in some circumstances.
Nearly 9 of out 10 respondents (89 percent) said they would "search harder and wider" for work if their benefits ran out. Moreover, in a series of statements about benefits, the one that garnered the most agreement, with 69 percent, was that benefits were "giving me a cushion so that I can take my time in searching for a job," while 59 percent said compensation "has allowed me to take time for myself," 36 percent agreed that it "has allowed me to turn down positions that weren't right for me" and 40 percent agreed "I haven't had to look for work as hard knowing I have some income to rely on."
Of those out of work and not receiving benefits—those who have quit looking are not eligible—22 percent said their benefits had run out and 32 percent said they weren't eligible.
Read MoreRise in higher-paying jobs lighting US economy
The decline in labor force participation, in fact, has been a key to the drop of the unemployment rate in the post-recession economy. The jobless rate has slid from a high of 10 percent in October 2009 to its current 5.4 percent, the lowest level since May 2008. However, the participation rate has fallen from 66.1 percent to 62.8 percent during the same period.
Benefit programs have expanded as well, even as unemployment compensation dropped from the 99 weeks of eligibility during when the jobless rate was much higher.
The Supplemental Nutrition Assistance Program—food stamps—now serves 45.7 million Americans, down from nearly 48 million in 2012.
"Over the last year, we have seen the unemployment rate go down, but we too easily forget that there are people still hurting, still wanting to work, but on the verge of giving up," Express Employment's Funk said. "I believe everyone who wants to work should have a job, so we must not overlook those who have been left behind and left out of the job market."