O'FALLON, Mo. — In a windowless bunker here, a wall of monitors tracked incoming attacks — 267,322 in the last 24 hours, according to one hovering dial, or about three every second — as a dozen analysts stared at screens filled with snippets of computer code.
Pacing around, overseeing the stream of warnings, was a former Delta Force soldier who fought in Iraq and Afghanistan before shifting to a new enemy: cyberthieves.
"This is not that different from terrorists and drug cartels," Matt Nyman, the command center's creator, said as he surveyed his squadron of Mastercard employees. "Fundamentally, threat networks operate in similar ways."
More from the New York Times:
Germany acts to tame Facebook, learning from its own history of hate
US suspends tariffs on China, stoking fears of a loss of leverage
Deutsche bank's problems threaten a star banker
Cybercrime is one of the world's fastest-growing and most lucrative industries. At least $445 billion was lost last year, up around 30 percent from just three years earlier, a global economic study found, and the Treasury Department recently designated cyberattacks as one of the greatest risks to the American financial sector. For banks and payment companies, the fight feels like a war — and they're responding with an increasingly militarized approach.
Former government cyberspies, soldiers and counterintelligence officials now dominate the top ranks of banks' security teams. They've brought to their new jobs the tools and techniques used for national defense: combat exercises, intelligence hubs modeled on those used in counterterrorism work and threat analysts who monitor the internet's shadowy corners.
At Mastercard, Mr. Nyman oversees the company's new fusion center, a term borrowed from the Department of Homeland Security. After the attacks of Sept. 11, the agency set up scores of fusion centers to coordinate federal, state and local intelligence-gathering. The approach spread throughout the government, with the centers used to fight disease outbreaks, wildfires and sex trafficking.
Then banks grabbed the playbook. At least a dozen of them, from giants like Citigroup and Wells Fargo to regional players such as Bank of the West, have opened fusion centers in recent years, and more are in the works. Fifth Third Bank is building one in its Cincinnati headquarters, and Visa, which created its first two years ago in Virginia, is developing two more, in Britain and Singapore. Having their own intelligence hives, the banks hope, will help them better detect patterns in all the data they amass.
The centers also have a symbolic purpose. Having a literal war room reinforces the new reality. Fending off thieves has always been a priority — it's why banks build vaults — but the arms race has escalated rapidly.
Cybersecurity has, for many financial company chiefs, become their biggest fear, eclipsing issues like regulation and the economy.
Alfred F. Kelly Jr., Visa's chief executive, is "completely paranoid" about the subject, he told investors at a conference in March. Bank of America's Brian T. Moynihan said his cybersecurity team is "the only place in the company that doesn't have a budget constraint." (The bank's chief operations and technology officer said it is spending about $600 million this year.)
The military sharpens soldiers' skills with large-scale combat drills like Jade Helm and Foal Eagle, which send troops into the field to test their tactics and weaponry. The financial sector created its own version: Quantum Dawn, a biennial simulation of a catastrophic cyberstrike.
In the latest exercise last November, 900 participants from 50 banks, regulators and law enforcement agencies role-played their response to an industrywide infestation of malicious malware that first corrupted, and then entirely blocked, all outgoing payments from the banks. Throughout the two-day test, the organizers lobbed in new threats every few hours, like denial-of-service attacks that knocked the banks' websites offline.
The first Quantum Dawn, back in 2011, was a lower-key gathering. Participants huddled in a conference room to talk through a mock attack that shut down stock trading. Now, it's a live-fire drill. Each bank spends months in advance re-creating its internal technology on an isolated test network, a so-called cyber range, so that its employees can fight with their actual tools and software. The company that runs their virtual battlefield, SimSpace, is a Defense Department contractor.
Sometimes, the tests expose important gaps.
A series of smaller cyber drills coordinated by the Treasury Department, called the Hamilton Series, raised an alarm three years ago. An attack on Sony, attributed to North Korea, had recently exposed sensitive company emails and data, and, in its wake, demolished huge swaths of Sony's internet network.
If something similar happened at a bank, especially a smaller one, regulators asked, would it be able to recover? Those in the room for the drill came away uneasy.
"There was a recognition that we needed to add an additional layer of resilience," said John Carlson, the chief of staff for the Financial Services Information Sharing and Analysis Center, the industry's main cybersecurity coordination group.
Soon after, the group began building a new fail-safe, called Sheltered Harbor, which went into operation last year. If one member of the network has its data compromised or destroyed, others can step in, retrieve its archived records and restore basic customer account access within a day or two. It has not yet been needed, but nearly 70 percent of America's deposit accounts are now covered by it.
The largest banks run dozens of their own, internal attack simulations each year, to smoke out their vulnerabilities and keep their first responders sharp.
"It's the idea of muscle memory," said Thomas J. Harrington, Citigroup's chief information security officer, who spent 28 years with the F.B.I.
Growing interest among its corporate customers in cybersecurity war games inspired IBM to build a digital range in Cambridge, Mass., where it stages data breaches for customers and prospects to practice on.
One recent morning, a fictional bank called Bane & Ox was under attack on IBM's range, and two dozen real-life executives from a variety of financial companies gathered to defend it. In the training scenario, an unidentified attacker had dumped six million customer records on Pastebin, a site often used by hackers to publish stolen data caches.
As the hours ticked by, the assault grew worse. The lost data included financial records and personally identifying details. One of the customers was Colin Powell, the former secretary of state. Phones in the room kept ringing with calls from reporters, irate executives and, eventually, regulators, wanting details about what had occurred.
When the group figured out what computer system had been used in the leak, a heated argument broke out: Should they cut off its network access immediately? Or set up surveillance and monitor any further transmissions?
At the urging of a Navy veteran who runs the cyberattack response group at a large New York bank, the group left the system connected.
"Those are the decisions you don't want to be making for the first time during a real attack," said Bob Stasio, IBM's cyber range operations manager and a former operations chief for the National Security Agency's cyber center. One financial company's executive team did such a poor job of talking to its technical team during a past IBM training drill, Mr. Stasio said, that he went home and canceled his credit card with them.
Like many cybersecurity bunkers, IBM's foxhole has deliberately theatrical touches. Whiteboards and giant monitors fill nearly every wall, with graphics that can be manipulated by touch.
"You can't have a fusion center unless you have really cool TVs," quipped Lawrence Zelvin, a former Homeland Security official who is now Citigroup's global cybersecurity head, at a recent cybercrime conference. "It's even better if they do something when you touch them. It doesn't matter what they do. Just something."
Security pros mockingly refer to such eye candy as "pew pew" maps, an onomatopoeia for the noise of laser guns in 1980s movies and video arcades. They are especially useful, executives concede, to put on display when V.I.P.s or board members stop by for a tour. Two popular "pew pew" maps are from FireEye and the defunct security vendor Norse, whose video game-like maps show laser beams zapping across the globe. Norse went out of business two years ago, and no one is sure what data the map is based on, but everyone agrees that it looks cool.
Jason Witty, the chief information security officer at U.S. Bank, admits that the blinking map he breaks out for customer briefings is mostly for show. But it serves a serious purpose, he said: making the command center's high-stakes work more tangible.
"If you show customers the scripts you're actually running, it's just digits on a screen," Mr. Witty said. A big, colorful map is easier to grasp.
What everyone in the finance industry is afraid of is a repeat — on an even larger scale — of the data breach that hit Equifax last year.
Hackers stole personal information, including Social Security numbers, of more than 146 million people. The attack cost the company's chief executive and four other top managers their jobs. Who stole the data, and what they did with it, is still not publicly known. The credit bureau has spent $243 million so far cleaning up the mess.
It is Mr. Nyman's job to make sure that doesn't happen at Mastercard. Walking around the company's fusion center, he describes the team's work using military slang. Its focus is "left of boom," he said — referring to the moments before a bomb explodes. By detecting vulnerabilities and attempted hacks, the analysts aim to head off an Equifax-like explosion.
But the attacks keep coming. As he spoke, the dial displayed over his shoulder registered another few assaults on Mastercard's systems. The total so far this year exceeds 20 million.