"We're preparing for a 25 percent tariff on our business and that's on top of 20 percent already on running shoes," Weber said in an interview on CNBC's "Squawk Box." "It's really going to be upsetting for us."
The U.S. and China are locked in a trade war that's seen each side imposing tariffs on each other's products. Most recently, the U.S. levied duties on $200 billion worth of goods from China, prompting Beijing to put tariffs on $60 billion worth of U.S. goods.
A number of industrial companies in recent weeks, including paint and coating maker PPG and supply distributor Fastenal, have expressed concern about the U.S.-China trade war damaging their businesses.
Weber told CNBC that if Brooks moved operations it would likely need to be permanent. "We don't do musical chairs in our supply chain."
Brooks' former parent company, Russell Athletic, was sold to Omaha, Nebraska-based Berkshire in 2006. Six years later, Berkshire made Brooks a separate business unit. Weber reports to Greg Abel, a vice chairman at Berkshire.
Correction: An earlier version of this story incorrectly stated Weber reports directly to Buffett.