"When I was young, my father left. And he left my mom with four of us and no money, no house, no food," Rometty tells Cramer. "And my mom really, without a college degree, went back to school [and] showed us, 'Hey, this is not going to end like that, and every one of you are going to have a chance.'"
Rometty says her mother got a job and put her and her siblings through school, teaching them a key lesson that has resonated with Rometty to this day.
"I learned from my mom: Do not let someone else redefine you," the CEO says.
"This is that idea: do not let someone else redefine you, like don't let someone else redefine your company," she says.
Rometty also shares her thoughts on the lack of females in executive roles, saying that making it easier for women to balance life and work is the key to closing that gap.
"You've got to keep women in the workforce, whether it's through having children, caring for elder parents, all the things," she says. "Not that husbands and spouses and partners don't do those things too, but it's often what takes them out, and then to get them back in is hard."
The CEO adds that IBM has done a number of things to keep women working, including shipping breast milk back to babies when their mothers are traveling on the job.
"You keep them in the workforce, your odds are much higher," Rometty says. And that's just one of the many things we do."
Questions for Cramer?
Call Cramer: 1-800-743-CNBC
Questions, comments, suggestions for the "Mad Money" website? firstname.lastname@example.org