"I'm surprised interest rates haven't risen more," Kovacevich told CNBC's "Power Lunch" on Wednesday. They've "been manipulated by the Fed for far too long."
In a robust economy and historically low interest rates many people are anticipating an interest rate hike in 2018. Kovacevich said abnormally low interest rates do impact the number of people refinancing loans.
He pointed out that while people are more willing to take on new loans with lower interest rates, when mortgage rates that are lower than they should be, mortgage prices are higher. U.S. home prices rose 6.2 percent annually, according to Tuesday's S&P CoreLogic Case-Shiller Home Price Index.
"The risk is when you manipulate the rates and all of the sudden they go back to normal the way they should be," Kovacevich said. "You could burst the bubble. I don't think we have a big bubble here, but you certainly have elevated asset prices because of manipulation of the interest rates."