Mention big data to the average person on the street and you'll often get a yawn—or outrage. It's an overarching term that generally brings one of two things to mind: the NSA's PRISM project or automated suggestions at major e-commerce sites. Either way, the assumption is it's not something that affects their day-to-day lives.
But there's a lot more to big data than snooping and up-selling. And, as it becomes an increased part of business life, it's having a bigger impact on everyday occurrences than you might realize.
Companies and other entities are using the digital footprints you leave, knowingly and unknowingly, to influence a number of things you'd never expect had any ties to big data. Click ahead to see 10 examples.
By Chris Morris, Special to CNBC.com
Posted 23 Sept. 2013
While most professionals know the benefits LinkedIn offers as a professional networking service, few realize the site is actively trying to find you a better job as well. Beyond the data entered by users, it tracks the job postings you view and the companies you research. It also notes how quickly you're changing jobs, indicating your promotion path. Put together, the data help the site target and let you know about the "jobs you might be interested in."
"We want to digitally map the global economy, identifying the connections between people, jobs, skills, companies, and professional knowledge—and spot in real-time the trends pointing to economic opportunities," CEO Jeff Weiner said in a recent blog post.
In 2008, Google launched its Google Flu Trends experiment, a tracking service meant to show flu activity around the country and where it was prevalent. Today, that service is available in 29 countries. By data mining records of flu-related search terms, it models the spread of the virus and the models match the CDC's data, but are returned in real-time, letting the tech giant beat the government's information by several days. It's not perfect (last year's early outbreak confused the algorithms), but it's a generally effective way to protect yourself.
You might be able to slip past some speed traps, but you can't hide your lead foot from your smart phone. Apps that track "vibration information"— which typically encompasses most fitness apps— can extrapolate the data they receive to determine if you're walking, running, on a bike or in a car.
The flip side to this is that officials can also use cell phone data to spot problems early. New Jersey has a traffic management service in the town of Woodbridge that collects cell phone signals to spot patterns, like slow traffic. And in Boston, city officials have launched an app that collects vibration data as people drive, which is used to fix potholes throughout the city.
Allstate and Progressive both have programs that offer discounts to drivers who install a monitoring device in their vehicle that offers real-time information about their driving habits, but big data's reach in the insurance world goes far beyond that. Companies rely on long-range weather modeling services from companies like Kinetic Analysis and IBM's Deep Thunder project to help determine rates. Those services extrapolate trends from big data to make more accurate long-term forecasts.
A decade ago, the NFL schedule was determined manually. Then came an era of using computer software to determine the best matchups. In 2004, though, the league took a bigger leap and decided to incorporate big data into the process. Using a service from a predictive analytics firm FICO, it hoped to maximize its television ratings, balance travel for teams and boost fan engagement. Making the switch has resulted in higher NFL revenues from television, sponsorship and licensing.
In many ways, we make it easy for the NSA and other organizations to track us. Checking in on Foursquare and Facebook as we zip around town or go on vacation is just the tip of the iceberg, though. Photos taken with cellphones contain "geotagging" information, which show precisely where the shot was taken, (meaning that even if you don't tell everyone where you are explicitly, it's still pretty easy to figure out.)
With her meat dresses and semi-nude romping in videos, Lady Gaga knows how to turn heads, but credit her manager, Troy Carter, for some of that success. Using big data, he helps determine how to best promote Gaga and take advantage of her large fan base. Using data culled from her 40 million Twitter followers, and 59 million Facebook "Likes"— as well as data from ticket sales— he created Backplane, a micro network. It steers core Gaga fans to LittleMonsters.com, where the team can cut out the middleman, improve earnings and learn more about what fans want.
While modern games create terabytes of data per day, it was largely being ignored until recently. Today, though, developers and publishers are finding as they examine how players interact with the game, they're able to make subtle tweaks to improve the experience, something that can increase the time players spend in the game and, in some instances, help boost the number of in-game purchases they make, which gives publishers a revenue bump. That's helpful for long-leg games like Activision's "Call of Duty" and EA's "Battlefield," but it's more critical in the mobile space, where updates are made sometimes on a daily basis.
Netflix saw subscriptions rise earlier this year when it resurrected the cult classic TV show "Arrested Development. " And its "House of Cards" has won an Emmy award.
Selecting those shows was no accident. The company carefully monitors its customers' viewing habits, observing which shows are hidden treasures and what sort of programs it should design to further engage them. So far, it seems to be working. Not only was "House of Cards" a hit, Netflix's original series "Orange is the New Black" also proved popular with viewers, who talked it up loudly on social media.
Suicide among American veterans is a big problem. In February, the VA reported that while the percentage of veterans who kill themselves has decreased slightly since 1999, the estimated total number who have committed suicide has increased.
Theorizing that today's soldiers and veterans are active on social media, DARPA and Dartmouth University have launched a program to data mine the biggest social networks to search for patterns indicating suicidal behavior. Centiment, a data mining company that specializes in finance, military and health care, is leading the project, which looks for keywords, word patterns and other warning signs among veterans who volunteer to be part of the study.