"The buyers are frustrated. There is a lot of multiple bidding. I had one recently where they accepted a deal sight unseen," said Michele Anderson, a real estate agent with Progressive Urban Real Estate.
Read MoreDenver housing: Rocky Mountain high and HOT
Sales of existing Cleveland homes rose just more than 10 percent in April from a year ago, even as the supply of homes for sale dropped nearly 40 percent, according to the Northern Ohio Regional Multiple Listing Service (NORMLS). Foreclosure inventory is down 31 percent from a year ago, according to RealtyTrac, even though banks are repossessing homes now at a far faster pace than they had been.
Tight supply, as is the case in much of the nation, is pushing home prices higher, but dramatically higher in Cleveland, where the median price is well below the national average.
Read MoreWater, millennials drive Portland, Oregon, housing
The median price of a Cleveland city home sold in April was $55,000, up 80 percent from a year ago, according to NORMLS. In the broader metropolitan Cleveland area, the median price was $109,700, a 175 percent increase from a year ago.
"Prices have recovered from being really low during the recession. We don't have as many distressed properties on the market, so bank-owned homes that are there are going for more money because there's more competition," said Anderson.
While Cleveland is not really a target for young millennials, and new companies are not heading there as they are to other urban markets, the downtown is seeing some rebirth. The Republican National Convention is being held there in 2016, which will draw national attention to the area. Anderson says people seem more positive about the city, and there is something of a downtown resurgence, although all of the new construction is in multifamily rentals.
Read MoreNew listings not helping Twin Cities' housing rebound
Cleveland may be improving, but it still has a long way to go. The city's median household income is just more than $26,000, about half the national average, according to the U.S. Census, and the unemployment rate is twice the national average, at 12 percent.