Others, though, argue that the right major can make a real difference—in the short run, anyway.
"There is a general relationship that exists between your college major and your chances for employment right after college," said Edwin Koc, director of research, public policy, and legislative affairs at the National Association of Colleges and Employers. For example, he said, engineering majors for the past few decades have had a relatively easy time getting hired right out of school.
Read MoreThese college majors will land you a job
But then there is the matter of a college graduate's second job, and the ones after that. For those positions, employers' focus changes.
"At the early stage, employers are using a shortcut in terms of having a skill set that they want to be immediately available, and that shortcut is the academic major," Koc said. The critical thinking and the writing skills–those may become more relevant as you progress in your career."
Read MoreWhy Johnny can't write, and why employers are mad
Koc also warned against choosing an overly narrow major because there is a greater risk that it will become irrelevant. A major in cybersecurity may boost your chances in the job market right after college, he said, but the market can easily change.
"You can enhance your immediate prospects by really focusing in, but that's really all you're enhancing." A general major in business or computer science may have more lasting relevance, he said.