Bad Actor Score

How it works

Each participant starts with a bad actor score of 0. When they submit a response that is flagged as sub-par or not true, a point is added to their bad actor score. Participants with bad actor scores above 3 get excluded from being able to do surveys again.

Bad Actor Score Events

Quality Check Surveys

Quality check survey that assess whether someone is answering the survey properly and not responding with random numbers. These surveys have 3 types of questions:

An MCQ question that requires the participant to figure out the answer in a comprehension like manner

An MCQ question that is asked to participants who completed a particular survey that has an associated answer based on the response they gave in a survey [Red Herring]

An open text question that requires them to give their opinion

Points are added if:

Their answer to the MCQ question is incorrect

Their answer to the open text question is 1 word

Their answer to the open text question is a keyboard scramble (”hdhddjd”)

These surveys occur after signing up and before they complete a survey on an ad hoc basis.

Scramble checking survey responses

An automated assessment of all open text answers is done to ensure that answers are not non-sensical

Manual check of answers

Look through answers provided by participants for a survey once they complete

Sample Checks for data accuracy and identity verification

We randomly sample our respondents to prove the answers they gave are accurate and credible (e.g. prove they work for x as y).

We will question a random sample of participants to prove any of the info they gave about who they are

Process

Random sample of people is selected and marked in the database as needing to be checked.

A template message is created that asks the participants to verify one of the responses they gave (e.g. prove you are a policy holder)

The marked sample of people is added to a new campaign and sent the template message to prove they were telling the truth

If they do not respond they will be paid half the airtime for that survey

If they respond with non-sensical then a point added to bad actor score

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.