Rometty said that the key to her century-old company remaining an institution in this country is how many times it has been able to reinvent itself and follow the latest trends in tech.
Today, those trends are the cloud and artificial intelligence, which IBM employees refer to as "cognitive" programming.
"There's a reason we call it cognitive," Rometty told Cramer. "It's about augmenting what you and I do so we can do what we're supposed to, our best. And then that's the IBM that takes that technology and the know-how about how the world works and puts that together and actually changes business. We are the champion for business."
Rometty also stressed the distinction between AI that consumers see and use and IBM's AI area of expertise, business-oriented AI programs.
"Consumer AI in your home, it's typically speech detects to a search. That's fine. That's great," she said. "But we deal in the enterprise world, so this is training Watson. Watson is trained in industries: What does underwriting do? What does a tax preparer do? What does a doctor do? What does a customer service agent do? What does a repairperson do? And it helps them be better, and, in fact, helps them do their job."
Between IBM's dealings in cognitive programming and the internet of things, Rometty predicted that 1 billion people would interact with Watson by the end of 2017. The result would be a boon to IBM's latest transformation efforts, which Rometty said revolve around one word: data.
The CEO said that Watson will likely also play a monumental role in reforming the health care system, the first step of which would involve touching the lives of 20,000 cancer patients.
"We will be able to address, diagnose and treat 80 percent of what causes 80 percent of the cancer in the world. If that's not motivating, I don't know what is," Rometty told Cramer.
Questions for Cramer?
Call Cramer: 1-800-743-CNBC
Questions, comments, suggestions for the "Mad Money" website? firstname.lastname@example.org