Tech

Enjoy brick-and-mortar shopping? Get ready to be watched very closely

Facial recognition, computer vision and artificial intelligence may sound like creepy technology buzzwords, but they are quickly becoming a part of everyday brick-and-mortar shopping.

Marketers hope these technologies can reveal what you are thinking in the crucial period between entering the store and making a purchase — that means recording everything you do and analyzing that data.



Zhong Zhenbin | Anadolu Agency | Getty Images

Companies like Amazon already have granular data on what consumers look at, click on, and add to their carts before making a final purchase online — but when it comes to store purchases, like food, companies know less about us. That's why companies from Apple to Google have been helping retailers fill in the gaps, using shoppers' smartphone data.

"One of the things that marketers have been trying to do is get beyond the data of our keystrokes," said Daniel Newman, principal analyst at Futurum Research. "Whether its cookies tracking you through a retail website or what we're talking to a friend about on Facebook."

Check out Amazon's latest concept: A store without lines or registers
VIDEO1:1301:13
Check out Amazon's latest concept: A store without lines or registers

Concerns about tracking shoppers in-store intensified this month when Amazon debuted a new automated concept store, Amazon Go. A patent filed in 2014 by the company showed how an automated checkout store could work: Cameras and other technologies would follow individual shoppers — and could identify them by their skin tone — and analyze their behavior throughout the store. It's unclear if this is the system behind Amazon Go. The company declined to comment for this story.

It comes after retailers' apps have spent years trying to track users with a different technology: geolocation "beacons" that ping your smartphone as you visit stores. Beacons are tiny gadgets, positioned throughout the store, that can communicate with your smartphone to match your identity to your location.

"One of the side benefits of Amazon doing this in their announcement is it's going inspire retailers and consumers of what can happen when you merge technology and brick and mortars," said Rob Murphy, vice president of marketing at Swirl, a marketing technology company. "This is pretty interesting and compelling, it doesn't matter if it's Amazon. I think just the very fact that they've been able to do that is going to inspire retailers to use beacons to do better experiences."

In Europe, even a tenth of what we do in America you couldn't do there.
Marc Janssens
Fujitsu Americas

Data on your location in stores has been "the shiny object of marketers' dreams," said David Bairstow, a vice president of product management at Skyhook, which helped Apple build its location services in the early iPhone days.

If you've ever gotten a notification from Google to review a restaurant near where you are, or you have the app of a retailer on your phone, chances are high you've interacted with a beacon, Bairstow said. The "dream" is having the level of detail and knowledge to text a user a coupon while they're in the store for something they've been eyeing either on the shelf or online.

Most marketers interviewed by CNBC don't see either beacons or Amazon Go as "creepy." But in a survey of more than 2,000 adults last year, nearly 40 percent of smartphone owners said they weren't interested in participating in beacon technology. Eight percent said they participate, 9 percent said it "sounds great" and they would "love" to participate, and the remainder was divided, according to Epsilon research cited by eMarketer.

Jay Stanley of the American Civil Liberties Union's ‎speech, privacy and technology project said these technologies may test the line between "big data" and the personal data of individuals.

"I think the bigger issue is how people will feel knowing that every movement is being monitored and more importantly, what else is being done with the technology," Stanley said. "It could be pretty powerful micro-monitoring of people's behavior on video."

While most retailers have cameras in their stores now, few have Amazon's data-crunching infrastructure to match footage to individual users, said Marc Janssens, head of retail for Fujitsu Americas, which makes many self-checkout machines.

"All of them have access to that technology. The biggest problem is legacy. The camera right now is just used for surveillance, one step away to use for analytics," said Janssens. "Not everyone is Wi-Fi enabling their store, especially grocers, due to the cost of it and legacy of current system."

Stores like Macy's, American Eagle Outfitters and Sephora have adopted beacon technology, but it hasn't caught on that widely. Nearly a quarter of retailers had implemented beacons as of February, according to Retail Systems Research cited by eMarketer. Another quarter of retailers had no plans to implement them.

To get really in-depth data, retailers may need 50 or even 100 beacons in a store, and sophisticated software, which is simply too much overhead for many stores, Bairstow said. Plus, without cameras, it's hard to know what is drawing people to a location.

"It's still really hard to convert 'dwell time,' since I may be standing somewhere because I got a phone call, not because I'm more interested in that sweater," Bairstow said.

New technology may make it easier to spread adoption of customer-tracking technology. Janssens said communicating with users' smartphones no longer requires Wi-Fi or Bluetooth, and instead of downloading an app, some beacon features are now embedded in smartphone operating systems.

The hardware for beacons is also becoming more subtle, integrated into light fixtures and Wi-Fi routers, said Murphy. Amazon's ideas may push that further.

"Some people talk about 'the last mile'... in retail, Bluetooth signals are really about the last aisle," Murphy said. "They are typically the level of what department, what section, that's the 'aisle' level. The Amazon technology is at the item level. Every individual item in the store."

Janssens said that laws in the U.S. around privacy are much looser than in Europe, where technology like beacons or computer vision would be nearly impossible to implement. He said that laws around ID in Europe go back to World War II, but that younger generations are more comfortable with sharing identifying information with the world, especially through social media.

"I think it's more cultural. I'm not from the U.S. originally but I've lived here for 35 years. In Europe, even a tenth of what we do in America you couldn't do there," Janssens said.