We've all heard the phrase "big data" so much in recent years that the words have ceased to convey much meaning. But to hear specialty retailer Moosejaw talk about its use of software from AgilOne is to understand why there's so much hype around it.
Read MorePowering the holidays: The magic behind your cybershopping
As frigid winter days rapidly approach and weekend ski trips take shape, traffic to Moosejaw's website spikes. Then there's the holiday surge on Cyber Monday, when the company, based just north of Detroit, will pull in as much revenue in 24 hours as it did in half the month of June. About 40 percent of its annual sales come during the holiday period.
Down sweaters from Patagonia, hiking boots from Asolo and North Face jackets are among the many hundred dollar plus items that Moosejaw sells at 11 physical stores and its website. There are plenty of other places to buy similar items, so Moosejaw has to win with deals, but more importantly by targeting customers with the right item on the right day of the week at the exact right time.
Loading a cart up with snow gear is a good indication that a hat or gloves may be in order. Someone who buys every new fleece product that hits the site is providing a clear sense of likes, and probably dislikes. Some consumers respond to web banner ads and others are more likely to click through if an offer comes via e-mail. Reminding consumers when they have loyalty points that are about to expire turns out to be a rather good way to get them shopping.
Read MorePowering the holidays, part 1: Lost seconds can cost retailers millions
AgilOne collects all the data that flows through from customers so that Moosejaw can maximize its marketing spending and be as certain as possible that it's reaching the right users with the appropriate message. Since deploying AgilOne three years ago, Moosejaw's costs to acquire customers have come down by 10 to 15 percent as a percentage of sales. Revenue is approaching $100 million a year and increasing 25 to 27 percent annually.
"One challenge a lot of e-tailers have is recognizing an individual customer," said Eoin Comerford, Moosejaw's chief executive officer. "Scrubbing and making sure you have a holistic view of the customer is important."
AgilOne is playing in a rapidly expanding business that's broadly called predictive marketing. Companies such as TellApart and Custora are building competitive technology, while Adobe has been investing in its marketing cloud through acquisitions and internal development.
In April, AgilOne raised $25 million from investors including Sequoia Capital, Tenaya Capital and Next World Capital. Aside from Moosejaw, customers include The Body Shop, HarperCollins Publishers and a host of small to mid-size retailers that are all going up against larger competitors.
AgilOne CEO Omer Artun is a data scientist and former marketing director at Best Buy. He equates the experience of AgilOne to a butcher shop in the 1960s, when the owners knew their customers by name, their birthdays and their kids' names and birthdays.
It's a one-to-one experience, but in a world where knowing by memory that Harry Smith always buys 12 ounces of top sirloin on Tuesdays and picks it up around 2 p.m. isn't going to be enough. Today's online merchants have tens of thousands of customers and millions of data points to sift through to determine what's relevant and important.
Read MoreAmazon's next purchase: Best Buy or RadioShack?
"That's the only way you're going to win against somebody who has the best supply chain in the world," said Artun, who founded the Mountain View, California-based company in 2006. "If you're selling a product that's available in other places, the only thing that will differentiate you is service or the human touch element. The only way to do that for millions of customers is using technology like us."
For Moosejaw, the industry giant is REI. But like all e-retailers, Amazon.com is always lurking. Outdoor specialty retailers hold their own against Amazon because consumers tend to prefer the expertise and options that those stores provide. Still, Amazon changes the way that all retailers sell, said Dan Pingree, Moosejaw's vice president of marketing. Free shipping with Amazon Prime has conditioned consumers to expect more from everyone, he said.
"They keep upping their game when it comes to how you support and service customers in the Amazon marketplace," Pingree said. "Amazon is always pushing the envelope to get better and to offer new functionality and features. That keeps us on our game."
Lest you think that predictive marketing is the only area where big data is being used by consumer brands over the holidays, here's another example.
At Quri in San Francisco, founder and CEO Justin Behar has dreamed up a totally different way to provide retail brands with lots of data, but he's selling to the product creators rather than the stores. Employing an Uber-like workforce, Quri pays people across the country to take pictures from their smartphones in retail outlets.
They're hired to photograph brand displays (those giant cardboard creations you see in every large retailer) so that the companies paying for the promotions can determine if they're being set up properly, or at all.
According to Quri's reseach, brands have promotion compliance issues in half of stores. By showing companies like Tyson, Nestle and Dannon exactly what their in-store promotions look like, they're giving them the ability to do something about it.
"Understanding that data and addressing potential issues before the holiday sales periods can mean the difference between missing the mark or making the year for a brand," said Behar. "You can't improve on what you don't measure, so having an in-depth insight into the data around your product is a must."