How it works
Each participant starts with a bad actor score of 0. When they submit a response that is flagged as sub-par or not true, a point is added to their bad actor score. Participants with bad actor scores above 3 get excluded from being able to do surveys again.
Bad Actor Score Events
Quality Check Surveys
Quality check survey that assess whether someone is answering the survey properly and not responding with random numbers. These surveys have 3 types of questions:
An MCQ question that requires the participant to figure out the answer in a comprehension like manner
An MCQ question that is asked to participants who completed a particular survey that has an associated answer based on the response they gave in a survey
An open text question that requires them to give their opinion
Their answer to the MCQ question is incorrect
Their answer to the open text question is 1 word
Their answer to the open text question is a keyboard scramble (”hdhddjd”)
These surveys occur after signing up and before they complete a survey on an ad hoc basis.
Scramble checking survey responses
An automated assessment of all open text answers is done to ensure that answers are not non-sensical
Manual check of answers
Look through answers provided by participants for a survey once they complete
Sample Checks for data accuracy and identity verification
We randomly sample our respondents to prove the answers they gave are accurate and credible (e.g. prove they work for x as y).
We will question a random sample of participants to prove any of the info they gave about who they are
Random sample of people is selected and marked in the database as needing to be checked.
A template message is created that asks the participants to verify one of the responses they gave (e.g. prove you are a policy holder)
The marked sample of people is added to a new campaign and sent the template message to prove they were telling the truth
If they do not respond they will be paid half the airtime for that survey
If they respond with non-sensical then a point added to bad actor score