"Tariffs are taxes," the Texas Republican said in a "Squawk Box" interview. "They impede economic growth. They pick winners and losers. We do worry about that."
However, Brady emphasized that Texas' economic growth has been strong enough to overcome tariffs, but they are having an effect. Texas is a major soybean producer, which have been hit with retaliatory tariffs.
Some GOPsenators on Tuesday condemned the Trump administration's plans to offer billions in aid to farmers hit by tariffs.
Trump is attacking what he sees as unfair trade on a number of fronts, including tariffs on steel and aluminum imports. His moves have been met with retaliatory measures from the European Union, Canada, Mexico and China.
The Trump administration has also slapped tariffs on $34 billion of Chinese products and threatened to raise tariffs to 20 percent on European cars.
Trump rebuked attacks by the senators, saying in a tweet Wednesday: “Every time I see a weak politician asking to stop Trade talks or the use of Tariffs to counter unfair Tariffs, I wonder, what can they be thinking?”
Trump on Wednesday is expected to meet with the president of the European Commission, Jean-Claude Juncker, in an attempt to overcome their differences, specifically on car tariffs.
Brady, who helped put through the tax law passed in December, is working on a "phase two" of the Republican tax plan.
When tariffs on steel and aluminum were announced in March, the House Ways and Means Committee released a letter signed by 107 Republican U.S. representatives urging Trump to trim down his proposals.