Yellen said the central bank is investigating the matter and will act should the facts warrant. The bank defended the steps it has taken to address its problems.
Warren previously had sent Yellen a letter asking for the board members' dismissal. The Massachusetts Democrat revealed that Yellen has since replied to that request, acknowledging that the Fed has the authority to act.
"How could removal of these board members not be warranted given the facts that we already know?" Warren said during Yellen's semiannual testimony before the Senate Banking Committee.
Wells Fargo already has paid millions in fines and is nearing a settlement in a class-action lawsuit over the scandal, in which employees created some 2 million fake accounts for customers without their knowledge. They did so to meet aggressive sales goals that called for employees to use cross-selling techniques to enroll customers in as many programs as possible.
More than 5,000 employees have been fired in connection with the scandal, but Warren said that's not enough. She said the bank clearly has run afoul of Fed regulations calling for proper risk management, and people at the top should pay.
"Can you explain to me how the Wells board can possibly have satisfied its obligations under the Fed's risk management regulations?" Warren said.
Yellen would not comment specifically on what actions the Fed will take, but she also was critical of the bank's actions.
"I will say that the behavior that we saw was egregious and unacceptable," she said. "We do have the power if it proves appropriate to remove directors. A number of actions already have been taken. We need to conduct a thorough investigation to look at the full record to understand the root causes of the problems. We are certainly prepared to take enforcement actions if those prove to be appropriate."
Warren, though, wasn't satisfied. She pointed out that the Fed already fined Wells Fargo in 2011 for illegally pushing mortgage borrowers into more expensive loans.
"Fines are not working with these giant financial institutions," the senator said. "If bank directors who preside over the firing of thousands of employees for creation of millions of fake accounts can keep their jobs, then I think every bank director in this country knows that they are bulletproof and that poses a danger to the rest of us every single day."
Wells Fargo representatives defended the actions the bank has taken to address the problems.
"Wells Fargo's board and management team have taken many actions in response to its retail sales practices issues, including changes in senior leadership, executive accountability actions and numerous steps to ensure we make things right with any customer affected by unacceptable sales practices. That work continues and remains a core part of our efforts to build a better Wells Fargo for the future," the bank said in a statement.