"I am convinced that at some point later this year we'll see the first 25 basis point rise. It'll rip through the infrastructure of utilities, and REITS and real estate," said O'Leary, one of the stars of the reality show "Shark Tank."
For example, if a couple took a $200,000 loan for a $250,000 house, that house would have to go up 6 percent a year to cover the debt, transaction fees and taxes, he said. However, after five years, O'Leary doesn't think that home will be worth more than the original sale price of $250,000.
Read More Low rates do nothing for mortgages
"Don't buy any real estate. Don't do it," O'Leary said.
Instead, he thinks investors should rent their homes and spend their money in the market—50 percent stocks, 50 percent bonds—for a better return and more liquidity.
Mortgage rates fell last week, as did applications for refinances and home purchase loans. Pending home sales were just up 0.4 percent in April, according to the National Association of Realtors, and the S&P/Case-Shiller composite index showed home prices jumping in March.
—By CNBC's Michelle Fox