Some market watchers expect the Federal Reserve to begin raising interest rates in September.
Read MoreHousing values: Why the divide isn't narrowing
The last time there was a mortgage rate spike, in late 2013, demand declined about 10 to 20 percent, Kelman said.
"When rates go up precipitously, when the market has a bit of a spike, that's when you really see that pullback. We don't see people saying 'well, I'll pay whatever it takes.' We see people saying, 'while rates are low I'm going to pay this price.'"
He thinks there needs to be less volatility in the housing market.
"What you want is more durable growth, not this up and down."
Pending home sales are now running at the highest levels since June 2013, and 11.1 percent above where they were a year ago, according to the National Association of Realtors.
Read MoreIn hot housing, how to win bid war
The hottest markets right now are San Francisco and Denver, Kelman said.
"They are both crazy. There are not enough homes for sales in Denver; everybody in California is fleeing high prices."