Get smart! Your phone habits are annoying at work

Purestock | Getty Images

Your colleagues probably do not want to overhear you talking on your cellphone about your cat's digestive problems.

Also, they generally don't like to see your gaze slip to your iPhone while you are supposed to be having an in-person conversation. Eyes up here, everyone!

Americans may love their smartphones—but they aren't necessarily so enamored with how their colleagues are using them in the workplace.

A new survey of 1,070 employed Americans, conducted by Harris Interactive on behalf of mobile business platform Jive Software, finds that almost everyone is annoyed by some type of bad phone etiquette at work.

The top offender, cited by 65 percent of those surveyed: Having loud or private conversations in public areas.

(Read more: What NOT to buy your boss for the holidays)

Even in open floor plan offices, experts say this is something that can be easily avoided.

"Your mobile phone is mobile. Stand up, walk to somewhere that's a little bit more private to have those private conversations," said Sydney Sloan, Jive's social media expert.

It's not just your mobile phone conversations that are annoying your co-workers—it's also all those beeps and buzzes your phone itself is making. About 59 percent of those surveyed said they were annoyed by co-workers who fail to silence or turn off their mobile phones when it's appropriate.

If you are working in a cubicle farm, you should generally always have your phone on vibrate or silent mode, said Jodi R.R. Smith, owner of the etiquette consulting firm Mannersmith.

But even that may not be enough, she said, because the vibration your phone makes on your desk also can be grating to your co-workers.

(Read more: For women, asking for a raise can backfire)

If you absolutely have to have your phone ringer on, make sure that the ring tone itself is work-appropriate.

"Some are more professional than others," she noted.

Everyone who has a smartphone knows the allure of that little screen and its attendant alerts. That's probably why 52 percent of those surveyed said another big annoyance is when people check their phones while having an in-person conversation.

Smith said it's perfectly fine to check your phone periodically while you are chatting with a colleague about their weekend plans but also waiting for a big report to come through, or awaiting a call from your boss

But if you are meeting with an important client, she said, the phone should be neither seen nor heard.

"It should not make a noise. It shouldn't even be buzzing. I should have it turned to airplane mode," she said.

(Read more: Mommy penalty: The cost for women who have it all)

That doesn't mean you have to completely ignore people who are trying to call or e-mail you. After all, that might annoy those colleagues and clients in an age when we expect everyone to always be connected.

Smith said one strategy is to schedule time between meetings to check your phone, so your office knows you will get any message within about an hour. Another is to be aggressive about using voice mail messages and e-mail "out of office" replies to tell people where you will be during the day and when you will be checking your phone.

Interestingly, only 38 percent of workers surveyed on behalf of Jive were annoyed by people checking their phones during meetings.

You still may want to slip your phone in your pocket during that meeting. Jive's Sloan noted that productivity could improve considerably if everyone put down their devices and paid attention to the discussion for once.

"Maybe, actually, the meeting would go quicker because everybody would be more focused," she said.

—By CNBC's Allison Linn. Follow her on Twitter @allisondlinn and Google or send her an email.