"You see a little more chitchat, potlucks, people exchanging gifts and it slows everything down," said Andy Decker, a senior regional president at Robert Half.
On the upside, taking the time to have lunch with a co-worker sort of counts, said Amy Glaser, senior vice president of Adecco Staffing, a separate staffing firm. "It's always productive to build relationships with peers and colleagues at work."
Workers in the 55-plus age group were significantly more likely to report they'll be "very productive" in the last week of the year compared to Gen X employees and millennials — 45 percent versus 24 percent, respectively, according to Robert Half's report.
Read MoreShould you expect a holiday bonus?
"That has to do with where you are in your career," Decker said. "It may not matter if it's a holiday. If you are a senior executive, president or business owner, the more likely you are to be engaged."
But it would be a mistake, Decker said, to spend a lot of time at work not getting much done.
"If you are here, be here," he advised. Otherwise, "go home and enjoy time with your family."
For those looking for work, it would also be a mistake to blow the week off, Glaser added. "A lot of organizations are trying to wrap up and fill positions before the end of the year," she said, particularly in industries that are busiest during the fourth quarter, including human resources, retail and finance and accounting.
For job hunters in those fields, "it's a great time to look for opportunities," she said.