Those benefits, he said, include "patent protection, research and development tax credits, national security and more." He added, "They shouldn't be allowed to shift their tax burden onto others."
Though laws have changed over the years to make inversions more difficult, they remain relatively easy to accomplish when done as part of an acquisition of a foreign company.
"It's unfair to criticize companies that do inversion when we live in this very complicated world," said a lawyer at a top law firm who has advised on inversions but spoke on the condition of anonymity because he was not authorized to comment publicly. "All companies should pay their fair share, but I'm not sure that companies that are inverted don't do so."
A number of pharmaceutical companies have set up in countries with low corporate tax rates, particularly Ireland and the Netherlands. Technology companies, industrial manufacturers and the banana producer Chiquita have also announced deals that would let them move abroad, substantially cutting their United States tax bills.
Warren E. Buffett, the chairman and chief executive of Berkshire Hathaway, said on Monday that his company was unlikely to ever move abroad, despite sometimes feeling at a disadvantage because of competitors' lower tax rates.
"We do not feel that we are unduly burdened by federal income taxes," Mr. Buffett said on CNBC. "But it does get a little annoying to us when we see other people paying far lower tax rates while engaging in the same sort of business that we engage in."
Read MoreAstraZeneca's new attempt to foil Pfizer's plans
Yet in the wake of Pfizer's efforts, Mr. Buffett said he thought legislation would be coming soon.
"My guess is that when you get to companies of this size, this prominence, and with this speedup of momentum, my guess is that Congress one way or another addresses this," he said. "Whether they just try to work on this little aspect of the code that allows this, which is not insignificant, or whether that forces them to rethink sort of all corporate taxes, we'll find out. But I do think it's going to get attention."
In addition to Mr. Levin, other lawmakers have called for efforts to curb inversions in recent months. Senator Max Baucus, Democrat of Montana and a former chairman of the Senate Finance Committee, introduced legislation that would have limited inversions before he became ambassador to China. And the budget President Obama submitted to Congress included language intended to end them. Neither effort has gotten far.
Pfizer said on Thursday that it was open to changes in the corporate tax code but wanted to make sure United States companies remained competitive.
"We expect the comprehensive tax-reform debate to continue, and we are actively engaged in the debate," it said. "Pfizer supports comprehensive tax reform that enhances the global competitiveness of U.S.-based companies operating internationally."
Follow us on Twitter: @CNBCWorld