Of those that did consider it an issue, 40 percent said bad publicity was the biggest problem it could cause. Meanwhile, 34 percent said the biggest problem caused was brand damage, while 33 percent thought it would create recruitment problems.
The online survey, conducted by 3Gem on behalf of NGA Human Resources, also found that 35 percent of men did not see the pay gap as an issue, compared to 22 percent of women.
"It is cause for concern that a significant proportion of business leaders still do not take the gender pay gap seriously. Whilst compulsory reporting is imminent, progress towards closing the gap will only be made if firms are prepared to put in place meaningful programmes," said Geoff Pearce, Managing Consultant – Reward at NGA Human Resources, said in a press release.
When asked what caused the gender pay gap, most survey respondents believed it was due to women's personal career decisions: 49 percent said the main factor was that women are more likely to take career breaks. Other reasons included the lack of women in the overall workforce (22 percent of respondents) or fewer women in senior management roles (27 percent).
New regulations coming into force on 6 April in Britain will require employers with more than 250 staff to report the gap between what male and female staff are paid and the amount of bonuses received.
"To meaningfully close the gap, organisations need to develop a culture of equality at work," Pearce told CNBC via email.
"Shared parental leave and childcare support that enable women and men the same chance to balance a career and home life would be a major step forward. It is now up to organisations to take responsibility for closing the gap, and not just wait for further government regulations to force change."
Follow CNBC International on Twitter and Facebook.