The billionaire chairman of luxury giant Compagnie Financiere Richemont said changes in technology will create a massive underclass and growing inequality that could spark "envy and hatred" of the wealthy.
In a speech Monday at the Financial Times Business of Luxury Summit in Monaco, Johann Rupert said the most important issue for the luxury industry and economies globally is the structural unemployment caused by robots, artificial intelligence and the new machine age.
Inequality will accelerate in the coming years due to the growth in structural unemployment coupled with a "new abundance" for the global winners, he said. That will cause "envy, hatred and social warfare" against the winners in the new economy.
Read More Hedge-fund billionaire: Inequality could be 'disastrous'
"We can't have the point 1 percent of the point 1 percent taking all the spoils," he said. "Now folks those are our clients. But it's unfair and it's not sustainable."
"I don't know what new social pact we'll have," he said. "But we'd better find one. Otherwise our clients will be targets. They'll be hated, despised."
Inequality is a far bigger threat for the luxury goods business than discussions over e-commerce and "clicks and bricks," he said.
Read MoreChristie: Wealth gap worse under Obama
"This is really what keeps me up at night," he said. "Because people with money will not wish to show it. If your child's best friend's parents go unemployed, you don't want to buy a car or anything showy."
Rupert isn't the first billionaire to warn of the consequences of the growing wealth gap. Billionaires Paul Tudor Jones, Warren Buffett, Jeff Greene and Stan Druckenmiller have all warned of a top-heavy economy where a small elite is reaping most of the income and wealth gains. Yet Rupert's comments—from a luxury chief speaking to other retailers to the rich in Monaco—suggest that inequality is becoming a concern even to those who have benefited from the spending of the wealthy.
"We are destroying the middle class," he said. "It will affect us, and it's unfair."
Rupert said Richemont—which owns Net-a-Porter, Alfred Dunhill and jeweler Van Cleef & Arpels, along with luxury watch brands Cartier, Piaget, Vacheron Constantin and IWC—will continue to focus on desirable products with brand equity.
He didn't offer specifics on how Richemont will adapt to a new world of stealth wealth and 1-percenter resentment. But he minimized the threat from smartwatches and other technology, saying Apple will sell "a ton" of Apple watches, yet it won't replace Richemont brands. While gadgets likes smartwatches and smartphones are often disposable, Cartier watches are kept for a lifetime or even generations.
Read MoreWealth gap widens in developed world
"If your daughter turns turns 18 or 21, you want to give her something she will remember the rest of her life," he said.
When asked about whether the corruption crackdown in China will hurt the luxury business, Rupert said he is still bullish on China's long-term prospects.
"If you have to make a law that says you're not allowed to have military number plates on Ferraris, Bentleys and Rolls-Royces do you smell a bit of corruption?" he said. The anti-corruption drive is a "wise thing do to," he said, adding, "I do think it's going to continue."
Yet he said the gender imbalance in China, with so many more men than women, will benefit the luxury business and the purchase of luxury gifts.
"You have more men than women, so how are (men) going to get lucky?" he said. "They're going to have to be very generous to the women."