Fighting Trolls And Fake Responses Through Validation Insights

Federal Communication Commission Commissioner Jessica Rosenworcel addresses protesters 

In the age of trolls and bots hijacking the internet, data scientists, market researchers, and insights managers are more important than ever. It's our job to report unbiased feedback and observations. It's our superpower and personal joy to identify fake or biased responses and offer true insights for decision makers and the everyday person. One of our proudest moments of 2017 is being a part of a major net neutrality story. 

On December 14th, 2017, Forbes and other news outlets broke a net neutrality story that "Millions Of Pro-Repeal Comments Likely Used Stolen Identities."  Survey Analytics and QuestionPro, alongside  Startup Policy Lab, helped identify stolen identities - 88% of the Pro Repeal comments on the FCC page were stolen identities.

The study found that many of the 23 million comments posted to the FCC site were likely submitted by bots or other non-human sources. According to Jeff Kao, Data Scientist at Metis & co-author of the study, 16 percent of the responses his team received from the commenter email addresses suggested that the Net neutrality comments were submitted without permission. 

The study highlights these key findings:

1. One pro-repeal spam campaign used mail-merge to disguise 1.3 million comments as unique grassroots submissions.

2. There were likely multiple other campaigns aimed at injecting what may total several million pro-repeal comments into the system.

3. It’s highly likely that more than 99% of the truly unique comments³ were in favor of keeping net neutrality.

A bar graph shows the number of survey respondents who say they didn't submit a comment filed under their name, with faked pro-repeal comments in red and pro-net neutrality ones in green. (Credit: Jeff Kao/Startup Policy Lab)

Although the FCC stated they did not have the resources to validate comments, Kao and his team, using the Freedom to Information Act, were able to reach out directly with an emailed QuestionPro survey to 450,000 individuals who had supposedly submitted those comments. The validation survey findings were quite shocking.

“It seemed obvious that someone had cheated,” says Vivek Bhaskaran, founder of survey company QuestionPro, which helped design and conduct the study. “We were expecting the people who said, ‘No, that’s not my comment’ would be evenly distributed between pro-Net neutrality commenters and anti-Net neutrality commenters. When we looked at the data, it seemed obvious that one camp had clearly orchestrated a campaign” opposing Net neutrality.

Both Kao and Bhaskaran declined to speculate which entity or entities might be behind the fake comments.

In 2018, we expect the fight against bots and trolls using validation insights to rise. Companies looking to capture true insights will want to implement an iron-clad validation process to ensure responses and feedback are not manipulated by unknown entities.