After experiencing falling revenue for 14 straight quarters, the tech giant is downsizing.
At The New York Times DealBook conference Tuesday, Rometty told CNBC's Andrew Ross Sorkin the decline has been caused in part simply by being a global company. The other major factor is IBM has been divesting, a lot.
"In my tenure, I've divested $8 billion of businesses," Rometty said. "The point was, they weren't about the future of where we were going."
Rometty cited an example of a recent divestiture in hardware — making semiconductors. Now, only 10 percent of IBM is comprised of hardware.
That's an example where size does matter," she said. "The world is consolidated, we were subscale, we did it for 60 years, right? So I do R&D but now I have a partner do the manufacturing for that area."
Rometty added that the strategic divestitures come from a desire to manage for the long term.
"It is really important to have a clear view of what you are … you have to keep moving up to what we call higher value," she said.
Rometty told Sorkin that IBM is working its way to that higher value. She said the company's 50 percent gross profit margins are up 80 basis points already.
"Tech is littered with areas that you could have high growth and make no money in, and that just has never been us," Rometty added. "It's been about the enterprise and what we do, and where we bring value to the client, and where in fact it brings value to shareholders here."
Update: In a note to CNBC, IBM points out its revenue has been roughly flat this year when you strip out the impact of currency and divested business.