The sooner the Federal Reserve winds down its massive quantitative easing bond-buying program the better, private equity titan Barry Sternlicht told CNBC on Wednesday. "This baby has to stand on its own legs now," he added.
"I think people were nervous with the [Fed] taper the rates would shoot up," the chairman and CEO of Starwood Capital Group said in a "Squawk Box " interview. "Obviously they didn't. They went the other way. I think a lot of people are set up on the wrong side of the trade."
The yield on the 10-year Treasury dipped below 2.70 percent in early trading Wednesday. "They expected the 10 year to go to 3.5 percent. … People thought we were subsidizing rates by 100 basis points with the Fed," he said. "I look at the bond market as the purveyor of truth."
The reason rates are not rising is because of the economy—growth is "not that strong" and won't meet the 2.8 percent to 3 percent targets set by the Fed for 2014, he said.
Investors will be looking for clues on how the central bank views the economy Wednesday afternoon, when the Fed releases minutes from its March meeting.
Last month, policymakers scaled back QE for the third-straight meeting by $10 billion. The Fed is now buying bonds and mortgage backed securities at the pace of $55 billion a month.
In a slow growth environment, stocks will probably have a "zero to 10 percent" return this year, Sternlicht predicted.
Without a pickup in the economy, the stock market will have a hard time moving strongly higher, he said. "The total equity value of the stock market is 115 percent of GDP or something like that. Everytime it gets here, it stalls."
"You can't take 115 to 132 percent of GDP," he continued. "You're not going to have another 20 percent year."
Outside controlling $33 billion in client money with Starwood Capital, Sternlicht said he invests—not trades—stocks with his personal money. He said he had a good year in 2013 with a gain of 32 percent, which was slightly higher than the increase in the of 29.6 percent.
—By CNBC's Matt Belvedere.