If you're looking for that mansion in the Hamptons this summer, your choices may be dwindling.
The inventory of luxury homes in the locale—which make up the top 10 percent of sales—is down 42 percent in the first quarter compared to the same period a year earlier, according to a report from real estate broker Douglas Elliman and appraiser Miller Samuel. There were 259 luxury homes on the market, about a 14-month supply, in the first quarter.
During the quarter, the average sale price in the Hamptons hit $1.76 million, up 3 percent from 2014.
Read MoreRupert Murdoch's penthouse on sale for $72 million
All those numbers mean that this summer in the Hamptons will be especially competitive for wealthy real estate buyers.
"For buyers, it's going to be a little tough this year," said Jonathan Miller. "It's not going to be frantic. But it's going to be more challenging than last year. Supply is tighter and demand is, if anything, a little stronger."
Of course, a strong first quarter doesn't guarantee a strong summer. But Miller said, "We had a robust first quarter and I expect more of the same in the second."
The first-quarter median sales price of $920,500 was the highest first-quarter number in seven years. It also marked the highest first-quarter sales volume in eight years, suggesting that the bad weather did little to deter wealthy Hamptons buyers.
Read MoreInvesting in wine? Taste the Bordeaux
Miller said that because the Hamptons is "joined at the hip" with the Manhattan market, the strength in Wall Street salaries and other high-level jobs in the city is driving demand. Yet, most of the growth isn't in the $20 million megahomes, but more in the $1 million to $5 million cottages and more modest getaways.
"The 1 million to 5 million is the bread and butter of the market," he said.