While big data is fast becoming an essential tool for businesses and marketers, it can still be hard for the average consumer to grasp. Faced with dense data sets and jargon-like "Hadoop," people's attentions tend to quickly wander elsewhere.
But big data isn't all about optimizing shipping routes and streamlining customer support calls. Sometimes, it reveals details about the world we might never have suspected. As data scientists crunch more and more numbers, they're finding a few startling trends— and they've managed to conclusively prove some long-held beliefs.
—By Chris Morris, Special to CNBC.com
Posted 9 Oct. 2013
Netflix isn't the only place where orange is the new black. Kaggle, where companies post data and data miners from around the world compete to produce the best models, found that when shopping for a used car, it's best to focus on one that's an unusual color.
We had a competition for a used car dealer to see which cars bought at second-hand auction would be good deals, " said founder and CEO Anthony Goldbloom. "Somebody found that unusually colored cards were much more likely to be reliable than standard-colored cars. The intuition there is if you're the first buyer of an orange or unusually colored car, then you were probably an enthusiast and really took care of that car."
Next time you're in a traffic jam, take note of when you finally start getting back up to a steady speed. It probably won't be immediately after you pass the fender bender. In another Kaggle study, researchers found that while a traffic jam certainly extends backwards, it also propagates forward. "Not nearly to the same extent," said Goldbloom, "but certainly a little bit."
What the researchers can't yet determine, though, is why this happens.
When University of Pennsylvania researchers analyzed the Facebook status updates of 75,000 volunteers, they were looking for a different way to analyze personalities. According to Lyle Ungar, a computer science professor at Penn, one of the most surprising findings was that people who posted regularly about sports tended to be among the most emotionally stable.
Those same people also tended to talk about other social activities, including beaches, church and family time topics.
If you can avoid connecting through Denver the next time you have travel plans, it might be a good idea. In a study for GE, Kaggle looked into what contributing factors went in to flight arrival times. One important feature, the researchers discovered, was the elevation of the airport.
"The higher the airport is above sea level, the more likely it was to suffer delays," said Goldbloom. "That might be because they're more subject to things like snowstorms and exposure to bad weather."
Another good airport to avoid? Colorado's Telluride Regional Airport, which at 9,078 feet above sea level is the nation's highest commercial airport. It's 67 percent higher than Denver.
Sometimes, it's not the quality of the pictures you post that entice people to click the "like" button or post a comment on Facebook. Sometimes, it all comes down to the caption. Want to increase a pic's popularity? Kaggle says the word "tomb" tends to engage people.
The lowest scoring words? "Nome" and "San Jose." The California city, said Goldbloom, scored "really bad."
That same University of Pennsylvania study found that people whose personalities are more reclusive tend to show a much larger interest in Japanese culture. Researchers noted the words "anime," "pokemon" and "manga" were used much more frequently among introverts, as well as Japanese emoticons.
Long before big data became a buzzword, Wal-Mart was mining its customer information. In 2004, as Hurricane Frances barreled down on Florida, the company began poring over data from previous storms to predict what was needed. Alongside the usual items like batteries and generators, the company found there was an extremely high demand for Pop-Tarts—strawberry Pop-Tarts— to be precise.
Beer sales, too, would skyrocket. The company quickly loaded up trucks and sent the products to the Sunshine State, where both sold quickly, boosting Wal-Mart's revenues.
Airlines live and die by schedules, so predictive modeling is crucial for them. As it turns out, the food you eat gives them an idea of how reliable a passenger you'll be.
The Economist notes that an airline yield management study found that food was one of the best predictors of whether a passenger would make it to his or her flight on time. Those who had requested a vegetarian meal proved to be much more dependable.
Beyond looking at personality types, the Penn researchers also did a breakdown by sex, highlighting the most frequently used words by men and women on Facebook.
Among the most used by women were "shopping,"excited" and the heart emoticon "<3". Men? After No. 1 "wishes he," the next two were unprintable.
In a deep dive into psychopathy, Kaggle took an extensive look into an array of Twitter profiles. That resulted in two notable findings. First, those who used the word "but" frequently tended to exhibit more symptoms of the personality disorder. Additionally, people who showed particularly strong punctuation skills had a higher degree of psychopathy.
Certainly, knowing when to use a comma and a conjunction don't make you a candidate to become a serial killer, but we're going to keep a close eye on the copy-editing desk here just to be safe.