That plays into how likely a cheater is to get forgiven: 76 percent of the women would forgive their partner for a sexual affair, while only 35 percent of the men would, the survey found. This contrasts sharply with the mere 30 percent of women and 80 percent of men who would forgive emotional cheating, the survey found.
"What kind of cheating is more painful? It totally depends on the individual and maybe on gender as well," Sigurd Vedal, Victoria Milan's CEO, said in a statement.
(Read more: Why employers should watch unfaithful spouses)
How common are these types of affairs? A 1983 survey of infidelity among U.S. married couples found 31 percent of men and 16 percent of women had a sexual affair with no emotional involvement, while 13 percent of men and 21 percent of women had been romantically but not sexually involved and 20 percent of both sexes had engaged in affairs including both sexual and emotional factors.
To be sure, the Victoria Milan survey only looks at the attitudes of users of the website, which describes itself as an "extramarital dating site," but it does suggest cheaters won't necessarily forgive their spouse for doing unto others.
(Read more: Cheating App: Dangerous Liaisons Get Less Risky)
In addition, divorce statistics suggest the forgiveness theory may not work in practice.
About 23 percent of men and 19 percent of women cheat on their spouses, according to a 2011 online survey of around 900 heterosexuals by Indiana University.
The percentages aren't too far off from the reasons couples give for seeking a divorce: An AARP survey found about 27 percent of divorces around mid-life and beyond were due to cheating; Australian government statistics found around 20 percent of divorces were attributed to infidelity.