Bogle blasts finance industry's failing ethics

Want a window into the size of the problem in financial services? Listen to Jack Bogle rattle off the following three numbers: $56 trillion, $1.6 trillion, and 1.15 percent. They illustrate the professional and ethical decay out of control in the financial services industry that the 84-year-old rails against, present tense:

"It's not time for me to slow down," the Vanguard Group founder told a crowd of about 600 people who came to the Bryn Mawr Presbyterian Church near his home on Philadelphia's Main Line on April 28 to hear his analysis of rising costs and ethical problems in the financial system. (Bogle began the first index fund in 1974—passive strategies, such as index funds, now account for about a third of equity assets. Malvern, Pa.-based Vanguard Group has $2.5 trillion in assets.)

Read MoreWhy Bogle made the CNBC First 25 list

Bogle's mantra is the idea of buying and holding stocks and bonds for the real value underneath—the value of the goods and services produced by the companies. He used the three numbers to draw a contrast between this value-generating part of the economy and the speculators in the financial services sector who try to make money off the movements in the market.

Why $56 trillion?

The amount of trading in the U.S. stock market reached $56 trillion last year, an all-time high.

"Short-term trading represents 99.5 percent of the market's activity; long-term capital formation 0.5 percent. Yet only capital formation adds value to our society. Trading, by definition, subtracts value. Indeed, casino may be too kind a term to describe the Wall Street of today's marketplace," Bogle said.

And as for $1.6 trillion?

The financial industry now accounts for $1.6 trillion—almost 10 percent of our gross domestic product (GDP) vs. 5 percent in 1980, 6 percent in 1990 and 7.5 percent in 2000.

Finally, how about that 1.15 percent?

Despite the mutual fund industry's huge growth, expense ratios of major U.S. equity mutual funds (he excluded Vanguard, which has lowered expense ratios on an asset-weighted average to 0.17 percent) have risen from an average 0.62 percent in 1951 to 1.15 percent in 2013. Yet there are vast sums of assets in these funds today after decades of asset-gathering growth by fund marketers and brokerage companies.

"The huge economies of scale available in the management of other people's money has been arrogated by the managers to themselves, rather than being enjoyed by the shareholders," Bogle said.

Read MoreThe CNBC First 25 list

Bogle, who routinely takes on Wall Street, isn't on the side of Michael Lewis when it comes to the financial journalist's new book, "Flash Boys: A Wall Street Revolt" and the issue of high-frequency trading, which allows some traders to make very thin margins on many trades by working ahead of moves they expect in the market.

"Despite Flash Boy's scathing (and partially accurate) criticisms, high-frequency trading (HFT) of stocks is not going away," he said. "The book's remarkable success reaffirms that well-written polemics by proven authors fly off the shelves, while balanced studies of controversial subjects rarely sell books."

Bogle, who has often criticized ETFs when they are used as trading vehicles, continued that theme on Monday, linking ETFs to Flash Boys and high-frequency trading. Because ETFs can be traded easily, they've helped to enable strategies, including high-frequency trading.

"The huge risk of a technology breakdown is out there," Bogle warned. Yet he said that he recognizes that high-frequency trading is here to stay and that it has brought some benefits, including lower trading costs. He suggested that regulators should expand insider-trading regulations to control the practices.

Read MoreBuffett: This is huge for mom-and-pop investors

Along with traders, whom he called the croupiers of Wall Street, he particularly singled out advisors for criticism—both advisors to institutional investors, like pension funds, and advisors to individuals.

"Each year, these advisors ... are paid staggering sums of money—perhaps $300 billion or more—for their presumed ability to add value for their investors. But in the stock market, the average money manager earns, yes, average returns before all of their costs. What else is new? But after their advisory fees, trading commissions, tax inefficiency and all their marketing expenses and operating costs, the 'zero-sum game' they play becomes a 'loser's game'—a game that, in the aggregate, inevitably subtracts value from their clients' wealth. There is no mathematical way around that."

Bogle also called on the mutual fund industry to take a larger role in corporate governance. The evolution of the industry toward passive investment strategies means that finance no longer creates an incentive for companies to be governed well.

Today the dominance of index funds belies the old Wall Street rule: "If you don't like the management, sell the stock." A new index-fund rule is emerging, Bogle said. Since index funds can't sell the stock (if it's in the index, it stays in the fund, no matter what), the new mantra must become, "If you don't like the management, fix the management."

"This truism—yet to be honored—will sooner or later alter profoundly the relationship between financial America and corporate America," Bogle said.

He pointed out that the median pay of CEOs of major corporations in 2013 was $13.9 million, a 9 percent increase over 2012, or what he called a "nice living."

Bogle called on the audience to become activists themselves.

"If you own mutual funds, get out your pen and paper and write to their CEOs and their independent directors, demanding that they step up to the plate on corporate governance issues," he said.

Bogle—speaking at times in a shaky voice—took questions from the audience after his speech and from a perch on the altar lobbed a criticism at JPMorgan Chase CEO Jamie Dimon. Referring to the fact that Dimon kept his job and got a pay raise even after taking responsibility for the $6 billion London Whale trading loss, Bogle said, "I thought the captain went down with his ship. But I am old-fashioned."