Few of the speakers think the market is heading for outright calamity. Even Nouriel Roubini said that he thought equities would continue to climb on the back of monetary accommodation for another two years. Wilbur Ross warned of a "time bomb" in bonds—but predicted the time bomb wouldn't explode until at least 2018.
"We like what we see in the U.S. but we would call it a low growth, low risk environment. We're the tallest midget," said Jonathan Nelson, founder and CEO of the private equity firm Providence Equity Partners. Nelson said he was looking for assets not correlated with the broader economy, such as professional sports.
"It's just kind of euphoria out in the equity land," Leonard Green & Partners managing director Jonathan Sokoloff said.
Josh Harris, the co-founder and chief investment officer of Apollo Global Management, argued that value could still be found in investments but that the search had become more difficult. Investors need to look outside of many traditional asset classes and areas to find good opportunities he said.
The same thought was echoed by Tony Ressler, the founding partner of Ares Management.
"Investing requires a wider fairway. It includes adding distressed, adding ABS, adding structured credit," Ressler said.
One of the great areas of hoped for opportunities—buying assets from deleveraging European banks—hasn't materialized in anywhere near the scale many regular attendees at the Milken Institute hoped for. Wilbur Ross explained that this was because the European authorities have been hesitant to force their banks to take losses, which means many assets sit on their books at inflated values. Actually selling the assets would require a realization of their market value.
For the most part, the private equity guys seemed to agree that stocks and bonds have run way ahead of the the performance of actual businesses, including those in their portfolios.
"There's a complete disconnect between performance on Main Street and performance on Wall Street. We don't know what it is. It's not alarming but it is definitely a trend throughout our portfolio," Sokoloff said.
(Read More: The Fed Is a Jobs Destroyer: Ken Griffin)
Ken Griffin offered one explanation for this. Central bank-induced financial asset inflation has "reduced participation by market professionals," he said. That allows prices to rise on waves of optimism untempered by more skeptical hedge fund investors.
Perhaps the most negative view of the entire conference came from venture capitalist Peter Thiel, who said that he worries that the current situation could be worse than the Great Depression because the science and technology sector are innovating less than they did in the 1930s.
Thiel said, "I worry that we've lost the future as an animating idea."
(For more from the Milken Conference, see Real Estate Moguls Seek Riches Overseas and Surfing the Markets with Mohamed El-Erian)
—Follow John on Twitter @twitter.com/Carney