Paul Leaf is a lawyer being fitted for a custom-made suit that will cost him $1,000. He does this a few times a year. "Recently it's been every two months," he said. "My credit card bill has been higher."
Leaf met to be measured by tailors in a suite in downtown Los Angeles recently, along with other customers like Paul Ruszat. "This is the only place I buy my suits and clothes," said Ruszat.
The tailors measuring the two men are Anthony Asaf and his son, Mark, who have flown to L.A. on one of their semi-annual trips to the United States from Hong Kong.
"We see more demand coming here," said Mark Asaf. "We find ourselves extremely busy here, more busy than Hong Kong."
That's both a good thing and a bad thing.
Read MoreWhat not to wear: The most unfashionable fashion stocks
Hong Kong is renowned for shops turning out quality, custom suits at a fraction of the cost for bespoke suits from the UK or continental Europe. However, prices are going up, and it's taking longer to make the suits, because there is a shortage of quality tailors.
"Making a suit, the hand labor, is just not as popular with the younger crowd," said Mark Asaf. He said the number of premium tailoring shops in Hong Kong has collapsed by more than half over the last 10 years, and the average age of a tailor is 60.
"I think people would rather be doing this," he said, imitating typing on a computer keyboard, "than this," he said, moving his hands to mimic sewing.
Asaf's father, who founded the company in 1983, agreed: "Young people think differently."
The result is that it is taking longer for suits to get made. Mark Asaf said it used to take five weeks after visiting the U.S. to deliver finished suits to customers. Now it's taking twice as long. It's also taking longer for customers who go directly to Empire's shop in Hong Kong. That's far different than when Richard Ruszat first visited more than a decade ago.
"We went to Hong Kong, and I chose 12 shirts. I got to pick the material, I got to pick the collar and the cuff," he said. "The next morning, all 12 shirts were ready for pickup."
Read MoreMillennials redefine luxury—and the stakes are high
Still, the good news is that orders remain strong as business formal has made a comeback because of shows such as "Mad Men," "Boardwalk Empire" and, well, "Suits." Some custom suit work has already moved out of Hong Kong to mainland China or Vietnam, but most agree the quality is not the same.
If demand remains strong, and the supply of tailors continues to weaken, Hong Kong prices will rise to lure in new tailors. Yet if prices rise too much, the city will lose some of its price advantage over Savile Row in London or custom tailors in Milan. Price has been one thing that attracted California lawyer Shareef Farag to Empire, and now he's hooked.
"Once I started having to wear a suit a lot for work, I really began to appreciate the difference in the quality, as well as the fit, that you get from a tailored suit," he said.