The small, four-office bank based in Elizabeth, New Jersey, which has about $1.6 billion in assets, and $1.4 billion in deposits, had for decades given male workers health insurance for themselves, spouses and children at no cost.
But female workers were only given such insurance for themselves, "and explicitly denied female employees the option of obtaining health insurance coverage for their spouses or families," according to the National Women's Law Center, which represented the two tellers. That meant women who worked at the bank, including DeVito and Kusterer, were subject to a larger financial burden from having to get such coverage elsewhere, in some cases, the NWLC noted.
Kusterer told CNBC, "We knew all along, for years, that this was not right." She said that there were about 10 male employees at the bank, all of whom were given family health coverage, while there were slightly less than 40 women who were denied the ability to get that coverage for their husbands or children.
In an October 2012 article in the Newark Star-Ledger, a former bank executive, Charles Morgan, told that newspaper that the benefit structure reflected the views of Union County Savings Bank's CEO Donald Sims that men are responsible for supporting their families.
Sims did not deny Morgan's claims that women and men were offered different health insurance options in terms of their family.
"I'm not denying it. I'm not saying anything," Sims told the paper at the time. "It is, as I say, not worthy of comment."
Kusterer told CNBC on Friday, that "sometimes Mr. Sims said, 'well, you have a husband,'" when the CEO was asked by employees about the disparate treatment at the bank.
"I think he thinks that since you have a husband, or you're a woman, you're not entitled to certain things," she said. "It's the general idea that men were the breadwinners of the family."
DeVito on Friday said that during a conversation one day about Sims, "My manager said, 'He's not discriminatory, he's just old-school.' "
In 2013, DeVito and Kusterer received raises that were 50 percent less than raises given other tellers at their branch on the heels of their discussing filing a complaint, they said.
Around the same time, Union County Savings Bank for the first time ever offered female employees the option of getting family health coverage, according to to the National Women's Law Center. But, the center noted, the bank for the first time required employees—of any gender—to pay a premium for that coverage, and also changed the insurance to a plan with less generous benefits.
Read MoreObamacare small-biz toll: $23 billion, 350K jobs
The tellers soon after, in late spring 2013, filed complaints with the EEOC. That agency then investigated, and within two months had found reasonable cause to believe the tellers' claims that the bank had violated federal civil rights.
The EEOC's findings then led to the settlement with the bank, which was signed Aug. 29 by Sims and signed off on by the women and the EEOC itself this week.
That agreement includes the requirement that Sims and the bank abide by federal employment law, and not retaliate against the tellers and other women at the bank. Sims, along with the head of human relations for the bank and branch managers, must also undergo two hours of training on the laws his bank was accused of violating.
The settlement will cover not only compensatory damages, but also out-of-pocket medical costs the women incurred, as well as back pay for the two tellers.
The settlement also noted that, "It is understood that this Agreement does not constitute an admission by any Respondent of any violation of any statute administered by the EEOC."
Sims was out of the office and not available to comment to CNBC, according to the bank when contacted Friday. A request for comment from another executive was not immediately returned.
Emily Martin, vice president and general counsel of the National Women's Legal Center, praised the EEOC while criticizing the bank for its treatment of female workers.
"Health insurance is part of a pay package—what an employee earns by going to work each day," Martin said. "It is shameful that in this day and age an employer would shortchange its female employees simply because they are women. And it's even worse that Ms. DeVito and Ms. Kusterer took a further hit to their paychecks simply because they sought to challenge this blatant discrimination. We applaud the EEOC for its work in bringing about this settlement and helping to right these wrongs."