A new study of seven major fitness trackers finds that calorie counts are often wildly inaccurate.
Researchers at Stanford University launched a study of 60 volunteers sporting Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn and the Samsung Gear S2. Six of the devices measured heart rate accurately — with an error rate of less than 5%. Apple Watch had the lowest median heart rate error at 2%, while Samsung Gear had the highest error rate at 6.8%.
All of the devices, however, measured energy expenditure, or calories, very inaccurately. The median error rate across all devices and tasks ranged from 27.4% for Fitbit Surge to 92.6% for the PulseOn. Apple Watch, which released a special edition Nike-branded running watch last September, received the highest overall score when taking into account every category across heart rate and calories.
Fitness trackers accurately measure heart rate but not calories burned, Stanford study finds: https://t.co/DDuzZUJH33 @euanashley pic.twitter.com/CVrefN2y3R
— Stanford Medicine (@StanfordMed) May 24, 2017
The researchers said factors such as skin color and body mass index affected measurements, with darker skins and higher BMIs leading to greater errors. Each device also uses its own proprietary algorithm for calculating energy expenditure, which can impact results.
Much of the reason for the discrepancy between the brands is that consumer devices aren’t held to the same standards as medical-grade devices, said Euan Ashley, a professor of cardiovascular medicine, genetics and biomedical data science at Stanford University.
Since over-the-counter fitness trackers are not as closely monitored by U.S. regulatory authorities, such as the Food And Drug Administration, which evaluates medical devices, doctors often don’t know what to make of the heart-rate data and other data produced by a patient’s wearable device, he said.
Get The Latest Sports Tech News In Your Inbox!
“People are basing life decisions on the data provided by these devices,” Ashley said. “The take-home message is that a user can pretty much rely on a fitness tracker’s heart rate measurements. But basing the number of doughnuts you eat on how many calories your device says you burned is a really bad idea.”
Of the study participants, 31 were women and 29 were men. Each volunteer’s heart was simultaneously measured against a “gold standard” device, such as a medical-grade electrocardiograph and an instrument for measuring oxygen and carbon dioxide in their breath, which researchers said was a good proxy for metabolism and energy expenditure.
In the study, cycling was the activity of the three — stationary cycling, treadmill running and walking — that produced the most accurate results, while walking was the most inaccurate.
The researchers have started work on a second iteration of the study that would evaluate the same devices while volunteers wear them and go about their normal day, outside a controlled laboratory setting.