The conflict at the heart of the entire industry's problem, Bogle argued, is that most mutual fund companies serve two masters: They are owned by public companies and so have a duty to maximize profits, which means charging investors more. But they also have a duty to individual investors, and the evidence is steadily rising that the clearest path to good returns is low fees.
Read MoreCNBC 25: Jack Bogle
"There's a dollar-for-dollar conflict between the two," said Bogle.
That conflict is mirrored throughout the industry—such as with stockbrokers who earn commissions for higher-priced products or the administrator of your retirement plan at work who might pay a lower plan cost in exchange for including higher-priced funds among your fund selections.
Investors in some ways have enabled the industry. As long as they're making money, they're happy. "In my experience, investors are much more interested in absolute returns than relative returns," said Bogle. But that means they're giving up some returns, and in down or even markets, it can mean that they lose money when they could have made it because of the high fees.
"If you're a management company, what do you want to do when your prime line of business is not doing well? You start a new fund and put a higher price on it and sell it," Bogle said.
Read MoreHow much hidden fund fees cost you
What do you do in response? The first step is to arm yourself with information. If you don't like what you hear or you can't get an answer, vote with your feet by shifting money to companies and funds in which you have more confidence. Then, Bogle said, it is in the public interest—and yours—to become a complainer.
"If you had a crowd of investors who said, 'Look, this is just wrong,' directors would have to listen, whether they want to know or not," he said.
Here are some key questions to help you stay in control of your investing life so you can become the confident complainer.