- A new report by advertising group Video Advertising Bureau found major discrepancies in Facebook's metrics compared to census data and even its own reported information.
- Companies tell CNBC that the issue is widespread across the industry, and there needs to be universal digital ad measurements.
- Facebook and Google have been reluctant to share data or methodology, but as they become bigger advertisers, they're finding they have to be more open.
Advertising companies are increasingly talking about discrepancies between Facebook's measurements and outside information, highlighting a digital industrywide problem over advertising standards.
"We get concerned with how much money we placed there and how effective our marketing dollars are," said CEO Bill Abbott of Crown Media Family Networks, which runs the Hallmark Channel.
"It's a big dilemma for the advertising business — not only for television, not only for video, not only for us at Hallmark."
The Video Advertising Bureau, where Abbott sits on the board of directors, released a report earlier this month bringing up multiple issues with Facebook's ad measurement techniques. Other board members include executives from NBCUniversal, A+E Networks, Scripps Networks, ESPN, Discovery and Turner, among others.
The report's findings include:
- Facebook's platform "reached" more people in every state than the 2016 population of those states according to the U.S. census. In the case of New York, Facebook reported the potential reach was 42 percent higher.
- The differences were more pronounced in the top 10 major cities. For example, Facebook's potential reach for people between 18 to 34 in Chicago was 1.9 million people, while the census reported only 808,785 in the category, a difference of 135 percent. Dallas' potential Facebook reach was 1.2 million, but the census only recorded 379,567 people in the age group (216 percent more).
- Facebook's reported estimated daily reach posted on its advertiser dashboard was up to 12 times higher than if customers manually calculated the reach using its own data.
VAB's study follows other instances of questionable metrics.
In September, Pivotal Research Group senior analyst found Facebook's advertising management system said it could reach 41 million 18- to 24-year-olds in the U.S. while census data showed there were only 31 million people in the group living in the U.S. — a discrepancy that Facebook explained by saying that the reach numbers included visitors.
But that explanation doesn't make sense for the more localized examples in the VAB study, because the numbers Facebook reports for its potential reach for "everyone in this location" and "people who live in this location" are identical.
A Facebook spokesperson explained: "Reach estimations are based on a number of factors, including Facebook user behaviors, user demographics, location data from devices, and other factors. They are designed to estimate how many people in a given area are eligible to see an ad a business might run. They are not designed to match population or census estimates. We are always working to improve our estimates."
Advertisers have also grown more skeptical after Facebook itself reported several measurement issues this year, including overestimating the average viewing time on video ads and differences on ad performance metrics on its mobile app and internal dashboard.
To be sure, there could be multiple motives behind the report. The companies on the VAB board are mostly TV networks, and are competing against Facebook for advertising money.
At the same time, these companies are heavily involved in advertising on Facebook — both because they put their own video content on Facebook and sell ads against it, and because they buy ads on Facebook to promote their offerings. So their criticism comes at least partly from experience.
Meanwhile, nonmedia brands have also criticized Facebook's targeted advertising metrics, with Procter & Gamble notably telling the Wall Street Journal last year that it was scaling back on these specific ads because they were not that effective.
Abbot is quick to point out that while the report picks apart Facebook's flaws, there's an issue with how every digital company measures advertising success on its platform, including online advertising market leader Google.
The root stems from the fact that digital companies self-report their data for the most part, and there are no universal digital ad measurements. And by showing audience numbers to be wildly different than the census, it throws into question what other metrics may be off.
"They're probably the easy target because they are so big, and everybody utilizes them," said Abbott.
While companies have bought ads on Facebook and Google for years, the two giants have been taking more of the pie, with eMarketer estimating they'll take in more than 60 percent of U.S. digital ad revenue this year.
Last year was also the first time digital ad spending surpassed TV. According to an Interactive Advertising Bureau report by Pricewaterhouse Coopers, U.S. digital advertising revenue reached $72.5 billion, while media agency Magna estimated TV ad spending hit $67 billion.
"The concentration of (ad buying) activity wasn't quite as obvious years ago," said David Cohen, president of Magna in North America which manages $37 billion worth of global marketing budgets on behalf of brands. "Years ago, there was much more of a healthy ecosystem outside of Google and Facebook in regards to concentration of dollars."
Matt Pharr, chief analytics officer at digital advertising agency MXM which works with companies including Benjamin Moore, Allergan and Sun-Maid, says that as advertisers spend more money on Facebook, they're under pressure to explain exactly how effective those ads are.
"People start asking these kinds of questions," Pharr said. "They want to be highly confident in the decision they are making on the dollars they are spending on these platforms."
One reason why advertisers are suspicious of digital measurement is because it's hard to reliably track individuals online. For example, people may have multiple accounts or log in from different devices, or multiple people may share the same account.
"They have a hard time understanding how many times they are hitting a unique individual," explained Pharr. "It's not unique to Facebook. It's across all of digital media."
Many advertisers and media companies believe the solutions are to have a third-party company create universal standards for measuring digital advertising reach, and for Facebook and other digital companies to release their data to advertisers so they can analyze it themselves.
In TV, most people hold third-party Nielsen ratings to be paramount despite its faults and flaws. Newspapers and magazines had circulation numbers and sales. But no such universal measurement standards exist in digital. Each company uses its own metrics.
"The linear TV business has been built on standards and norms to ensure numbers are represented to clients accurately," said Ben Price, Discovery Communications president of U.S. ad sales and a VAB board member. "As consumers are viewing content on new and different platforms at any given time, it's important for all stakeholders involved that those standards are extended and all numbers are accurately represented."
Magna's Cohen said, "Google and Facebook have lots of different products and great measurement solutions, but it's all in their environments."
Brands do rely on third-party companies to audit and measure ad campaigns against the standards advertising organisation Media Rating Council (MRC) sets, Cohen explained. Companies also use measurement firms like comScore, Datalogix and Moat. But while Facebook is willing to give third parties some of the information, it isn't willing to give advertisers the data directly. A Facebook spokesperson cited privacy-related concerns as the reason.
"Slowly, but surely we're chipping away at that as an industry, but historically Google and Facebook have been hesitant to do it," Cohen said.
The MRC — which is in the process of auditing Facebook and Google — is working on creating digital media audience measurement standards, with the first phase focused on video specifically. Ideally the standards will align measurement across television, over-the-top content and digital video. But measuring across platforms is difficult and will take time, and in the meantime advertisers are frustrated.
"Everyone has to be on a level playing field because Facebook and others in the digital space self-report, and they don't have that oversight," said Crown's Abbott. "I think that they're able to make some claims, and quite frankly put their advertisers in a position where (Facebook) might not be delivering what their objectives are."
Disclosure: The head of ad sales for CNBC's parent company NBCUniversal is on the VAB's board of directors.
Correction: An earlier version of this article misattributed to David Gunzerath of the MRC a quote that was from Magna's David Cohen.