Net Net: Promoting innovation and managing change
Net Net: Promoting innovation and managing change

Chertoff on the Big Business of 'Big Data'

It's the latest catch phrase to grab a hold of Silicon Valley, "Big Data." The size of data sets is getting so large that its getting harder for companies and firms to handle, analyze and securitize the tremendous amount information that is coming in.

With this information overload, even the government is looking for technology to help them wrap their arms around the data deluge. But along with every challenge there is opportunity. And this opportunity is a big one. In fact, its in the billions of dollars. EMC is a classic example of how companies are moving into the "Big Data" space with its $3.2 billion in "Big Data" acquisitions last year and they are looking to spend just as much in acquisitions this year.

EMC is not the only company in the race for "Big Data" supremacy. Oracle , Hewlett-Packard , International Business Machines and SAP AG are all vying for the top spot.

To get the latest pulse on this new technological frontier, I sat down with former Homeland Security Secretary, and founder of the Chertoff Group, Michael Chertoff. His firm specializes in looking for the next big thing in technology that can help companies when it comes to data synthesis and security.

LL: Where should the "smart money" be investing when it comes to "Big Data" technology?

MC: I don't think there will be just one techbnology. When you analyze how companies should tackle "Big Data" there are a lot of different solutions for different parts of the problem in how all this information should be managed and monitored. Clearly many companies are investing in the Cloud in terms of real time data collection and the ability to see what's coming in as well as intercept data and block if necessary.

Another set of solutions is monitoring the traffic and seeing where the flow is, detecting anomalies, integrating that data with signature data and based on that information, determine if you have any threats going on. Technologies that will allow you to be resilient or constitute I think will be important. At the other end of the "Big Data" spectrum, I think identity authentication, investments in tokens, biometrics will also be part of the solution. All of these elements will be areas of investments.

The biggest challenge we have is the abundance of information and what are we trying to do with it. If you can't figure out what's important, how the data connects with other data, or use it, that information is rendered useless. Its a miss opportunity. That's where analytics comes in. Analytics allows you to make the decision of what is important based on what you are looking for and what you have looked for in the past. In the cyber arena, there is a lot of information you collect in real time. You need to have technology that will be able to look at the data and inspect it to see if the data that is streaming should be flagged.

LL: You have talked about the technology aspect of security but what about the people factor?

MC: People are just as important when assessing a security risk. For example, the Fort Dixx case came from a guy who was developing pictures and someone saw something in that photo and reported it. In the Times Square bombing plot someone saw something funny and reported it as well. So the human factor is always important. Its an important part of data collection. Then, you have to integrate it.

LL: Anytime when you talk technology, it is very easy to get wonky and fall into inside baseball. Is that one of the biggest challenges facing the industry? Effectively communicating the message?

MC: When you build an architure you can not build something on the assumption that people will be infallible because than it would be a failed infrastructure. People make mistakes. And I have often found that the "experts" don't connect with their audience. Technical people talk right past the policy makers and civilians because of all the jargon. The audience does not understand why the technology is important to them. One of the things I have done to communicate my message is I get technical just enough so I can actually explain what the technology means to the policy and the customers.

LL: You have talked about the need for a public, private partnership. Can the two find a way to effectively co-exist?

MC: There is an interdependence between public and private partnership. Both need each other to be effective. The government can't do without the private sector but to be honest, the private sector can not do without the government sector as well.

The government does have the capabilities and insights that if they don't share, it would handicap the private sector. The idea is to find the right configuration where the government can lend the benefit of what they are doing but without raising the specter of the government controlling the Internet. If that happened, then we start heading down the road of what is going on in other countries and we don't want to go there. Its a balance we are trying to make.

LL: You mention the need to relationships. How important are relationships with other countries when it comes to information sharing?

MC: I think a lot of the problems people have identified when there is no "information sharing" has not been malicious. It is simply that you get a piece of information you don't think is particularly useful and you don't pass it on.

In order to avoid this, you have to balance two things. You want to have analysts to have the ability to look across all the data bases and understand what is important but at the same time you don't want a Bradley Manning (the US soldier arrested in May of 2010 in Iraq on the suspicion of having passed restricted material to the website WikiLeaks).

You don't want a PFC roaming for reasons that are not appropriate. There are ways to reconcile those between access control and tracking. You have algorithms that make you ask, why is a certain person spending a lot of time on a certain data base at one in the morning? You need to figure out how to use your technology tools to best detect internal intruders as well as external.

A Senior Talent Producer at CNBC, and author of "Thriving in the New Economy:Lessons from Today's Top Business Minds."


Questions? Comments? Email us at

Follow  on Twitter @

Follow NetNet on Twitter @

Facebook us @