Advertising companies are increasingly talking about discrepancies between Facebook's measurements and outside information, highlighting a digital industrywide problem over advertising standards.
"We get concerned with how much money we placed there and how effective our marketing dollars are," said CEO Bill Abbott of Crown Media Family Networks, which runs the Hallmark Channel.
"It's a big dilemma for the advertising business — not only for television, not only for video, not only for us at Hallmark."
The Video Advertising Bureau, where Abbott sits on the board of directors, released a report earlier this month bringing up multiple issues with Facebook's ad measurement techniques. Other board members include executives from NBCUniversal, A+E Networks, Scripps Networks, ESPN, Discovery and Turner, among others.
The report's findings include:
- Facebook's platform "reached" more people in every state than the 2016 population of those states according to the U.S. census. In the case of New York, Facebook reported the potential reach was 42 percent higher.
- The differences were more pronounced in the top 10 major cities. For example, Facebook's potential reach for people between 18 to 34 in Chicago was 1.9 million people, while the census reported only 808,785 in the category, a difference of 135 percent. Dallas' potential Facebook reach was 1.2 million, but the census only recorded 379,567 people in the age group (216 percent more).
- Facebook's reported estimated daily reach posted on its advertiser dashboard was up to 12 times higher than if customers manually calculated the reach using its own data.
VAB's study follows other instances of questionable metrics.
In September, Pivotal Research Group senior analyst found Facebook's advertising management system said it could reach 41 million 18- to 24-year-olds in the U.S. while census data showed there were only 31 million people in the group living in the U.S. — a discrepancy that Facebook explained by saying that the reach numbers included visitors.
But that explanation doesn't make sense for the more localized examples in the VAB study, because the numbers Facebook reports for its potential reach for "everyone in this location" and "people who live in this location" are identical.
A Facebook spokesperson explained: "Reach estimations are based on a number of factors, including Facebook user behaviors, user demographics, location data from devices, and other factors. They are designed to estimate how many people in a given area are eligible to see an ad a business might run. They are not designed to match population or census estimates. We are always working to improve our estimates."
Advertisers have also grown more skeptical after Facebook itself reported several measurement issues this year, including overestimating the average viewing time on video ads and differences on ad performance metrics on its mobile app and internal dashboard.
To be sure, there could be multiple motives behind the report. The companies on the VAB board are mostly TV networks, and are competing against Facebook for advertising money.
At the same time, these companies are heavily involved in advertising on Facebook — both because they put their own video content on Facebook and sell ads against it, and because they buy ads on Facebook to promote their offerings. So their criticism comes at least partly from experience.
Meanwhile, nonmedia brands have also criticized Facebook's targeted advertising metrics, with Procter & Gamble notably telling the Wall Street Journal last year that it was scaling back on these specific ads because they were not that effective.