The promise of Amazon's new futuristic convenience store is no line, no wait and, frankly, little to no human interaction. But for now, the machines still need some help from our species.
Recode has learned that Amazon has staff on call behind the scenes to assist the computer vision system that is supposed to detect which items a shopper pulls off a shelf and carries out of the store.
More from Recode:
GoPro is relaunching its Karma drone after an embarrassing recall
The TV industry is facing a crisis and needs to immediately embrace new business models
Autonomous tech supplier Mobileye wants automakers to crowdsource maps for self-driving cars
An Amazon spokesperson confirmed the setup and said that Amazon staff is asked to help out when the system used in the new Amazon Go store can't make a determination.
"[W]hen the machine vision system isn't sure, it asks a human for confirmation," she said. "It happens a small fraction of the time."
The spokesperson said that the review work is being done "in house" and not through Mechanical Turk, the Amazon-owned website that pays out pennies per item for tasks similar to this one. She did not immediately provide answers to several follow-up questions, including queries about the frequency of the human intervention; who exactly is doing the work; and whether they are reviewing still images, video footage or something else.
Amazon announced its first "Amazon Go" store last month in Seattle, though it is currently only open to company employees. The store's technology allows customers to take items like sandwiches and drinks from shelves and leave without checking out. Amazon has dubbed its system "Just Walk Out Technology."
While Amazon wouldn't describe it this way, customers are essentially being tracked by cameras inside, which help identify which products are removed and by whom. An Amazon patent application unearthed by Recode in 2015 suggested that such a store may also include microphones to locate someone and track their movements. The company has said that sensors also play a role. (Update: A spokeswoman said the store does not contain microphones.) The company has said that sensors also play a role.
It is not unusual for humans to help train computer vision algorithms, and the Amazon spokeswoman said fewer manual reviews will be necessary over time as the technology gets smarter.
But knowing there are people working behind the scenes does makes Amazon's announcement video seem a little less picture-perfect.
—By Jason del Rey, Recode.net.
CNBC's parent NBCUniversal is an investor in Recode's parent Vox, and the companies have a content-sharing arrangement.
