Wearable makers, like Apple Watch and Fitbit, are getting better at tracking heart rate, according to a new study from researchers at Stanford University School of Medicine.
But those that measure calories are far from accurate.
The researchers conducted a study to assess the quality of wearable trackers after finding a lack of data in peer-reviewed journals. "Anytime we get data from a patient via a device, we have questions about the accuracy," said Euan Ashley, an associate professor at Stanford, who focuses on cardiovascular medicine.
These devices aren't regulated, as they aim to optimize health rather than to detect disease, so they're not held to the same standards as their medical device counterparts. But many patients will still use them on a regular basis.
So Ashley's team evaluated 7 devices -- the Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn and Samsung Gear S2 -- with a group of 60 volunteers using rather than using treadmills and stationary cycles in the lab. The researchers compared the devices to FDA-approved gold standards, rather than to each other.
Here's what they learned:
- Not all wearables are created equal. The Apple Watch was a clear winner in both heart rate and energy expenditure, while Samsung's device reported the highest error rates.
- The devices were consistently terrible at tracking energy expenditure, with the most accurate device off by an average of 27 percent. Ashley said the error rate should be less than 10 percent when these devices are used in non-medical settings.
- Heart rate measurements have improved over the years. He described some of the early wearables as "random number generators." Ashley said that users can rely on this data-point.
- The devices were better at measuring data collected during cycling than walking.
- Errors also tended to be more common in men versus women, those with a greater body mass index, and with a darker skin tone.
Why are these devices so off when it comes to energy expenditure? "People are so variable," said Ashley. "Some people walk smoothly and others waddle along, and that has an impact."
Ashley now advises people to avoid trusting data like calories burned and using that to make decisions about what to eat. He warns against eating ice cream, for instance, simply because a tracker suggests that a user has burned sufficient calories.
Ashley hopes that wearable makers will respond to studies like these by releasing more data, whether it's positive or negative. The next iteration of his study will involve the volunteers wearing the devices in their daily lives rather than the lab.