The New Hampshire Retirement System voted on Tuesday to end its $239 million relationship with Fisher Investments, bringing the total divested from the money manager in recent weeks to more than $2 billion.
The plan's investment committee voted 5-0 in favor of firing Camas, Washington-based Fisher, citing lewd comments that Ken Fisher, the billionaire founder of the firm, made at an investment conference on Oct. 8, according to Marty Karlon, a spokesman for the New Hampshire Retirement System.
In all, six institutional clients have left Fisher in the last two weeks, bringing total asset losses to just more than $2 billion.
On Monday, Fidelity announced it also would remove its money from Fisher. The firm had managed $500 million for Fidelity's Strategic Advisers Small-Mid Cap Fund.
"The recent statements made by Ken Fisher, the founder and chairman of Fisher Investments, are not only offensive and inappropriate, they are incompatible with the values of the retirement system and bring into question Mr. Fisher's judgment," the New Hampshire Retirement System said in a written statement.
Money that Fisher previously managed for the $9.2 billion retirement plan will now be spread among four other money managers.
NEPC, a Boston-based investment consultant for the New Hampshire plan, had also recommended that its clients end its relationship with Fisher.
"In 2018, NEPC created an internal group called the Unfavorable News Committee to respond to non-investment negative news events connected to our investment managers," the consultancy said in a report obtained by CNBC. "Mr. Fisher's comments clearly fit that mandate."
Four government pensions had previously pulled close to $1.3 billion from Fisher Investments..
Also on Tuesday, the Public Employees' Retirement System of Mississippi said it would put Fisher Investments on a watch list due to "organizational concerns," according to Ray Higgins, executive director of the plan.
Mississippi PERS currently has $558 million invested with Fisher. In total, the pension plan has about $28 billion in assets.
CNBC obtained an audio recording of Fisher's comments at the Tiburon CEO Summit, as well as audio of him speaking at a previous conference.
Clips from both were featured on CNBC's "Power Lunch." Combined, they show that the money manager made flippant remarks about sex.
In the audio obtained by CNBC, Fisher said at the Tiburon conference, "Money, sex, those are the two most private things for most people," so when trying to win new clients you need to be careful.
He said, "It's like going up to a girl in a bar ... [inaudible] ... going up to a woman in a bar and saying, hey, I want to talk about what's in your pants."
Further, when Fisher was a speaker at the Evidence-Based Investing conference in 2018 he compared marketing mutual funds to propositioning a woman for sex at a bar.
"I mean the, the most stupid thing you can do, which is what every mutual fund firm in the world always did, was to brag about performance, uh, in, in a direct mail piece, which is a little bit like walking into a bar if you're a single guy and you want to get laid and walking up to some girl and saying, 'Hey, you want to have sex?'" Fisher said, according to audio obtained by CNBC.
The billionaire has since apologized for his comments.
"Some of the words and phrases I used during a recent conference to make certain points were clearly wrong and I shouldn't have made them," Fisher said in a statement. "I realize this kind of language has no place in our company or industry. I sincerely apologize."
Organizers of both conferences subsequently banned him from speaking again in the future.