There's a difference between having a boss you don't really like and having one that makes you want to quit. And in fact, a good amount of workers actually do quit their jobs because of their boss.
A 2015 Gallup survey of more than 7,000 people found that 50 percent had left a job to "get away from their manager."
"At any one time, 10 to 15 percent of us have a coworker, or actually usually a boss, that's bullying us," Stanford University professor and bestselling author Robert Sutton tells CNBC Make It.
But how do you know it's time to actually quit because of your manager?
Sutton, author of "The A-----e Survival Guide, " recommends trying to improve your situation before quitting. However, he says, there are multiple warning signs that indicate it may not be worth staying.
Here are the top three red flags you may need to quit over a bad boss:
"If you're having trouble sleeping," Sutton says, "that's an especially bad sign."
Not being able to get a good night's rest could mean your boss is making you very stressed, according to Sutton, who teaches organizational psychology.
Chronic fatigue, on top of stress, can be damaging to your health, resulting in anything from an inability to concentrate to an increased risk of heart disease. Of course, you should see a doctor to make sure your work environment is truly the culprit. But if it is, take it seriously.
"The second sign is you can't compartmentalize," Sutton says, "You're so upset, you come up and you start taking it out on your family, your friends, whatever."
At some point, nearly everyone experiences some spillover of stress from work to their personal life. But if your boss's behavior leaves you constantly aggravated, stressed or sad and it's impacting your relationship with your family and friends, it might be time to make a change.
Whether it's being more distracted or feeling less motivated, having a bad boss can impact your work. And turning in poorly-executed work, unfortunately, can make your boss even angrier.
"We have hundreds of studies that show if you feel disrespected and demeaned everyday you will work less hard," Sutton says. "You'll be less creative."
"If you're having those three problems, you're sick, you're making other people sick, and your work sucks," he says, "to me that's a sign that you better get out, as long as you can find a job to pay the mortgage and take care of the other expenses in your life."
But the psychologist also recommends you consider your other options before you jump ship.
First, make sure you're not exaggerating the situation. Ask yourself, "Am I being too sensitive?" Sutton suggests. If you have a trusted coworker who shares the same boss, consider asking them how they feel, without talking poorly of your manager.
Second, figure out how if you can change the way you interact with your manager. Maybe having a quick check in where you discuss how to improve your workflow could clear the air. Or perhaps, you just need to try to avoid the person or learn how to speak up for yourself, Sutton says. Remember, if you ever feel in danger around your boss or fear you're being harassed, speak up and tell your HR contact.
Last, explore whether or not you might be able to move departments within your company, Sutton suggests.
"There's a lot of evidence that suggests," he says, "it's just as good to move to another part of the organization as it is to leave."
Look at your company's job postings and talk to your HR representative or a trusted contact at work, ideally someone who is superior to you and could help you make the transition. If you're not comfortable speaking about your boss in a critical way, say you're interested in working in a new area of the company or acquiring new skills.
If all these options fail, then start actively applying to other jobs on the nights and weekends. Go through your list of professional contacts and friends to see who you could reach out to about finding your next job. While you won't want to seem desperate, you could start networking with them and learning about their work and the company they work for.
But be sure to line up another gig before you quit, Suttons says. That way, you adhere to his top rule.
"Quit," he says, "but don't be stupid about it."
Like this story? Like CNBC Make It on Facebook.