The Iowa caucus debacle represents one of the most stunning failures of information security ever.
This failure was delivered by the same Iowa Democratic Party officials who have said for the last four years they were "ramping up" their technology capabilities, convening seemingly endless security task forces to ensure foreign powers did not disenfranchise voters, and collaborating with federal agencies like the Department of Homeland Security to make sure everyone was in the loop on voting security.
Voters will be paying close attention to how party leaders ensure that votes going forward have clear contingency plans in place, not just to protect against hackers, but from all types of technology failures, including applications that might not work.
Iowa officials counting the results coming in Monday from the caucusing app reported irregularities that required them to switch from the app to counting votes manually. Party officials said the "underlying data" put into the app was fine, but it is unclear as of yet how they know this or even what they consider "underlying data."
"Last night, more than 1,600 precinct caucuses gathered across the state of Iowa and at satellite caucuses around the world," the Iowa Democratic Party said in a statement Tuesday. "As precinct caucus results started coming in, the IDP ran them through an accuracy and quality check. It became clear that there were inconsistencies with the reports. The underlying cause of these inconsistencies was not immediately clear, and required investigation, which took time."
The Iowa Democrats were using an application made by a partisan progressive start-up named Shadow Inc., managed by a nonprofit investment company called Acronym. In a statement, Acronym distanced itself from Shadow.
"We are reading confirmed reports of Shadow's work with the Iowa Democratic Party on Twitter and we, like everyone else, are eagerly awaiting more information ... with respect to what happened," Acronym said in a statement.
Iowa Democrats explained that backup measures for the Shadow app took "longer than expected."
"We have determined that this was due to a coding issue in the reporting system. This issue was identified and fixed. The application's reporting issue did not impact the ability of precinct chairs to report data accurately," the Iowa Democratic Party statement said. Voters will surely be asking the Iowa Democrats to prove how they know the information is accurate with so many reported irregularities.
Shadow apologized on Twitter Tuesday afternoon. "We sincerely regret the delay in the reporting of the results of last night's Iowa caucuses and the uncertainty it has caused to the candidates, their campaigns, and Democratic caucus-goers."
The Iowa Democrats and Democratic National Committee will have to answer several puzzling questions about why they chose to use the application in the first place.
First, in 2016, the Iowa caucuses used an application made by Microsoft, which worked. It's unclear why they didn't keep the same application, created by an established company instead of one from an untested start-up.
Microsoft is making sure people know it didn't make this year's app. "We had a great partnership with the Iowa political parties in 2016, but we are not part of the caucuses this year and have not been involved in building or supporting their app," a company spokesperson tweeted.
Second, in August, the Democratic National Committee recommended Iowa stop using an app altogether. The Democratic National Committee's Rules and Bylaws Committee voted to follow those recommendations. It said a security review had determined the virtual caucus did not meet standards for cybersecurity and reliability.
"We are — over the last week and continuing today and in the days ahead — continuing to look at what options might be available to us given the time frame that's left," Iowa Democratic Party Chairman Troy Price said in September, according to NPR. "We know there's not a lot of time left. There's 4.5 months between now and when Iowans head to the caucus sites."
DHS acting Secretary Chad Wolf told Fox News on Tuesday that the app "was not vetted for cybersecurity."
Now, Iowa is scrambling for answers.
Iowans are learning about the important distinction today between cybersecurity and information security.
Loosely speaking: In cybersecurity, organizations work to defend against hackers. In the broader field of information security, organizations work to be able to recover quickly whether they have been hit by a cyberattack, someone tripped over a cord in a data center or a server farm gets knocked out by a hurricane. Cybersecurity falls into the bigger bucket of infosec and resiliency planning.
In this case, it appears as though cybersecurity wasn't the issue, but the proper back-up planning, testing and vetting procedures were completely deficient or simply absent entirely. They had an app that they knew was problematic. They used it anyway without properly testing their back-up plans, each stage of which have proved to take longer than usual.
Preparing for the inevitably of a cyberattack meant the Iowa Democrats, Democratic National Committee and DHS should all have been ready to bounce back from a problem like this. The fact that they still haven't recovered is likely to be more disheartening to voters than any malicious Twitter campaign or fake Facebook ad or Russian phishing bid.
All of these organizations owe it to the electorate to never let something like this happen again. Because if they can't recover from a bad app, a hack or a hurricane could be far more devastating.