"If they don't cut the rent, I will leave," said the head of a consumer goods chain that also has a shop in Causeway Bay. Revenues have fallen 30 percent over the past year as the number of mainland visitors fell by half, he said.
"We can't bear the costs," he added, declining to be named as he did not want to highlight his company's financial situation.
Hong Kong retail sales in January fell to their lowest level since 2003, a factor property agents said prompted more retailers to negotiate lower leases, or just move out.
Read MoreRich Chinese spending less at home, more in Europe
The decline in sales also coincides with plans by several luxury retailers, including Chanel and Compagnie Financiere Richemont's Cartier, to cut prices in Asia to counter the sharp decline in the euro.
"Their businesses aren't doing so well, so they decided to essentially hand the keys back to landlord," said Tom Gaffney, head of retail at property consultancy Jones Lang LaSalle.
Property consultancy Savills says average prime street shop rentals fell 8.5 percent year-on-year in 2014, as the number of Chinese tourists began to fall.
This year, a luxury goods retailer in Causeway Bay managed to negotiate a 15 percent discount on its lease while another retailer renewed their contract at the same terms, a property agency involved in the deals told Reuters.
The slowdown is forcing many retailers to adapt.
Paul Tang, the owner of a tiny shop in the heart of Causeway Bay, said sales of his eclectic mix of banned Chinese books and milk powder fell 25 percent this year, forcing him to courier goods to clients in China. "It's no use sitting in my shop and crying," he said.
A possible restriction by the authorities on multiple daily visits to Hong Kong could further add to the pressure on landlords more exposed to the retail market, such as Hysan Development, Wharf Holdings and Sun Hung Kai Properties.
Day-trippers accounted for about 60 percent of the 47 million mainland visitors to Hong Kong last year often buying daily necessities such as milk powder. But some landlords are worried that protests by activists accusing mainland visitors of crowding public transport and causing shortages of goods could make things worse.
"The current political and social sentiment may diminish mainlanders' enthusiasm to visit and shop in Hong Kong," Roger Hao, chief financial officer of market leader Hysan, told Reuters. "But Hysan has a diverse yet balanced retail portfolio ... and that should minimize the impact."