"Precision regulation is what I think is needed because ... we've got to compete in this world against every country," Rometty said in a "Squawk Box" interview from the World Economic Forum in Davos, Switzerland. "You want to have innovation flourish and you've got to balance that with security."
Rometty called on lawmakers to regulate "how the technology is used," but not necessarily the technology itself. She pointed to facial recognition systems as an example of how such regulation could work.
"When you go through an airport to get through safely, to find a criminal, those are good uses of facial recognition," she said. "To violate your civil liberties, it's not."
Beyond civil liberties issues, other concerns around A.I. involve bias in algorithms. A.I. systems use large amounts of data, which could itself be biased, and the people writing the programs can have their own biases.
Rometty also weighed in on broader concerns around digital privacy, arguing the questions about regulation are more about "values" than the technology being used.
"We already have conventional rules such as what a priest knows is protected, what your doctor knows is protected. We need to do it for the digital age, and then courts can uphold that," she said.
Rometty's "Squawk Box" appearance comes one day after IBM reported fourth-quarter results that surpassed analysts' earnings and revenue estimates.
The earnings report also contained positive signs around Big Blue's $34 billion acquisition of enterprise software maker Red Hat, which was finalized in July.
Revenue from Red Hat was up 24% in the quarter, an improvement on 19% in the last period.
"We returned the company to growth under any metric you looked at," she said. "This demonstrated the acquisition we closed of Red Hat, IBM and Red Hat are better together."
Shares of the Armonk, New York-based company were up more than 4% on Wednesday morning. IBM is one of the 30 companies that make up the Dow Jones Industrial Average, which was also higher Wednesday.