Indeed, a decade after passing the Sarbanes-Oxley Act that some now cite as a culprit in creating this divide, Congress in 2012 passed the Jumpstart Our Business Startups (JOBS) Act that has tried to make it easier for smaller, newer companies to go public.
Nearly 85 percent of the companies which tapped the new "emerging growth company" category to go public after the JOBS Act was passed had less than $250 million in annual revenues, the Securities and Exchange Commission noted in its 2013 review.
That hasn't alleviated suspicions, however, that public markets have become a dumping ground for Silicon Valley.
"We're not there yet, but that's my concern," Colas said. "It used to be that companies went public to access larger pools of capital, and that they still had a huge runway of growth potential ahead of them."
Read MoreSidecar, Uber, Lyft: Are ride-sharing start-ups in a bubble?
That, he said, is no longer. Now, "they can do anything as a private company that they can do as a public one."
There is also, for private companies, the added advantage of staying out of the crosshairs of activist investors, and of public shareholders in general who are criticized for being too short-term oriented today.
Clayton Christensen of Harvard Business School has christened this "The Capitalist's Dilemma" and urged for fixes to make public markets more long-term value-oriented. (Mr. Christensen, notably, is also the author of "The Innovator's Dilemma," a 1997 book observing that most companies miss disruptive innovations by following the rules of management school, which left a deep impact on Silicon Valley.) He largely blames the rules of finance themselves for generating this behavior.
The pressure on public companies to meet short-term results as opposed to investing heavily in their future is said to be one of the reasons Facebook dragged its feet so long before going public. Even those start-ups who do eventually list, like Google and Alibaba, today often adopt protective structures that leave more control in the hands of executives than has traditionally been the case.
Regulators are aware that the appeal for companies of participating in public markets has dimmed—to the detriment, perhaps, of those very markets.
"Many growing businesses have consciously avoided the public markets over the past decade because of the regulatory baggage that accompanies the offering regime," said SEC Commissioner Dan Gallagher in a January speech.
Mr. Gallagher is calling for the SEC to finalize its outstanding JOBS Act Amendments, home in on secondary markets and brokers where some privately held corporate shares now change hands, and support "venture exchanges" not dissimilar from some of today's popular crowd-funding platforms.
The irony is that the lack of access to today's generation of fast-growing start-ups is now pushing some "mom and pop" mutual funds to gain access anyway.
Fidelity, T. Rowe Price, and Blackrock are among the big ones buying and holding shares of private Silicon Valley start-ups, as the New York Times recently reported.
Read More Main Street Portfolios Are Investing in Unicorns
There are several problems with this approach: first, the financials of the companies remain opaque. Second, some of the funds, especially pension funds, are investing through a venture-capital or private-equity middle man that charges hefty fees, making it harder to generate solid returns.
Third: Where's the liquidity? In public markets, like the New York Stock Exchange or the Nasdaq, information about the shares of each listed company-—namely, its price, volume, and number outstanding—is constantly available. Private markets offer nothing like that, nor do investors always realize that their holdings of common stock may be wiped out by the holders of preferred stock ahead of them in line should anything happen to the company.
This is what has Mark Cuban, the outspoken tech billionaire cum owner of the Dallas Mavericks basketball team, so upset.
"If we thought it was stupid to invest in public internet websites that had no chance of succeeding back then, it's worse today," he ranted on his blog in March.
"[T]he only thing worse than a market with collapsing valuations is a market with no valuations and no liquidity," he said. Referring to the proliferation of start-ups and apps today, he said, "there is ZERO liquidity for any of those investments. None. Zero. Zip."