Artificial intelligence will replace some jobs, but most humans will be working with AI systems, Rometty said. IBM has expanded cognitive computer Watson over the past few years.
"I am very pleased with the work we are doing with Watson and the scale," she said.
Under Rometty, IBM has pledged to hire about 25,000 U.S. workers in the next four years, including 6,000 hires in 2017. But Big Blue has also laid off some workers amid years of restructuring. In its latest annual report, for 2015, IBM said its workforce totaled 377,757, down by 53,455 from 2013.
(IBM told CNBC in a statement that despite the workforce decline from divestitures over the past couple of years, IBM's net employment is up 50,000 over the last decade.)
Rometty said the U.S. is entering into a new era of technology, and with any new era there are bound to be some fears.
"We've seen it in the past, whether when people come off of doing farming, they had to learn to read. The industrial area, it was mechanical skills," she said.
"If we would change the basis and align what is taught in school with what is needed with business ... that's where I came up with this idea of 'new collar.' Not blue collar or white collar," she said.
Rometty's comments come ahead of the company's earnings, expected Thursday. The pressure is on the company to reinvent itself as it has struggled to expand revenue.
"What we are continuing to do is transform the company," Rometty said. "And that means that there are parts of the business growing, and there are parts of the business — while they are really great businesses to be in — they may not be in growth markets."