Goldman Sachs has joined the flight of institutional investors from Fisher Investments.
The giant investment bank is pulling $234 million from Camas, Washington-based Fisher, according to a source close to the matter.
However, the end tally could be even greater, a source told CNBC.
Goldman confirmed Fisher would no longer be an underlying manager for the Goldman Sachs Multi-Manager Global Equity Fund in an Oct. 25 filing with the Securities and Exchange Commission.
Fisher Investments declined to comment.
In all, institutional investors, along with Goldman, are withdrawing more than $2.7 billion from Fisher Investments in light of Ken Fisher's lewd comments made at a conference on Oct. 8.
Meanwhile on Thursday, the Los Angeles fire and police pension plan voted to fire Fisher, where it held $522 million.
The Los Angeles pension has $24 billion in total assets.
The board of commissioners said they had invited Fisher himself to speak at the meeting, which was webcast live, but he did not attend.
"The only explanation is that Mr. Fisher was unable to attend and had business in the office," Ray Ciranna, general manager of the Los Angeles Fire and Police Pension System, wrote in an email to CNBC.
In total, Fisher Investments has lost more than $2.7 billion in recent weeks as eight institutional clients — six of which were government pensions — parted ways with the firm. Fisher had $94 billion in assets under management as of Dec. 31, 2018, according to their SEC filing.
That figure reached $112 billion as of Sept. 30, 2019, according to the firm.
On Monday, Fidelity said it would remove its money from Fisher. The firm managed $500 million for Fidelity's Strategic Advisers Small-Mid Cap Fund.
Additionally, the New Hampshire Retirement System voted on Tuesday to terminate its $239 million relationship with the firm as well. That same day, the Public Employees Retirement System of Mississippi, which has $558 million invested with Fisher, said it would put the firm on a watch list due to "organizational concerns," according to Ray Higgins, executive director of the plan.
CNBC obtained an audio recording of Fisher's comments at the Tiburon CEO Summit, as well as audio of him speaking at a previous conference.
Clips from both were featured on CNBC's "Power Lunch." Combined, they show that the money manager made flippant remarks about sex.
In the audio obtained by CNBC, Fisher, 68, said at the Tiburon conference, "Money, sex, those are the two most private things for most people," so when trying to win new clients you need to be careful.
He said, "It's like going up to a girl in a bar ... [inaudible] ... going up to a woman in a bar and saying, hey, I want to talk about what's in your pants."
Further, when Fisher was a speaker at the Evidence-Based Investing conference in 2018 he compared marketing mutual funds to propositioning a woman for sex at a bar.
"I mean the, the most stupid thing you can do, which is what every mutual fund firm in the world always did, was to brag about performance, uh, in, in a direct mail piece, which is a little bit like walking into a bar if you're a single guy and you want to get laid and walking up to some girl and saying, 'Hey, you want to have sex?'" Fisher said, according to audio obtained by CNBC.
The billionaire has since apologized for his comments.
"Some of the words and phrases I used during a recent conference to make certain points were clearly wrong and I shouldn't have made them," Fisher said in a statement. "I realize this kind of language has no place in our company or industry. I sincerely apologize."
Organizers of both conferences subsequently banned him from speaking again in the future.
More from Personal Finance:
Here's how much income tax you're paying to your state
A parent's guide to helping their kid get into college
Michael Avenatti allegedly failed to file tax returns. That's a bad idea