icon picker
Case Study: Specific User Spamming Hunch Primary Feed

Problem Statement

We are facing an issue where a particular user’s polls (i.e. BihariBabu) are dominating the Hunch Primary Feed of many users. Click the below link to see an illustration: ​

Problem Root Cause Analysis

Following is the statistics of the Creators of current Active Polls:
amanalok88@gmail.com 40 0.090090
syedumar21142019@gmail.com 28 0.063063
priya99.d53@gmail.com 21 0.047297
vipin.bitspilani2020@gmail.com 18 0.040541
wajiha1627@gmail.com 17 0.038288
dsr19112001@gmail.com 13 0.029279
hearttalks25@gmail.com 13 0.029279
dipankarjoshiparul@gmail.com 12 0.027027
meetahaldar1001@gmail.com 11 0.024775
yekkaliprem@gmail.com 11 0.024775
ananyasinghtomar52@gmail.com 11 0.024775
tanishqbatra2016@gmail.com 10 0.022523

There are 444 active polls from 128 unique Users (Creators). The topmost creator is (BihariBabu) who has created 40 of the active polls. Thus,
Likelihood of a poll being part of top-K recommendation feed is 40 / 444 = 0.090090
Further, below is the the statistics of active polls based on hunches > 200 and their creators:
amanalok88@gmail.com 12 0.117647
vipin.bitspilani2020@gmail.com 9 0.088235
drshreya1609@gmail.com 7 0.068627
syedumar21142019@gmail.com 7 0.068627
ananyasinghtomar52@gmail.com 6 0.058824
priya99.d53@gmail.com 6 0.058824
As we can observe, the likelihood for polls having hunches greater than 200 increases for the user BihariBabu.
Our AWS Personalize Feed gets recommendations from 3 different models:
User-Personalization Model This model recipe has been tuned in a manner to recommend more hunched polls from the active polls at the top.
Trending-Now Model The Trending-Now model recipe generates recommendations for polls that are rapidly becoming more popular with our users.
Popularity-Count Model Popularity-Count recommends the most popular polls (more hunched polls) based on our user interactions data.

Given that of the 3 model recipes, 2 of them are biased to show more hunched polls at the top of the Personalised User Feed, “BihariBabu” dominates the feed as he has the most number of polls amongst active polls and they also come within the most hunched polls.
Since more than 95% of this user’s polls are of 30 days duration, these polls have accrued enough hunches to come in the most hunched polls while also being active.
The only model recipe which is not affected in bias towards the user “BihariBabu” is the Trending Now Model Recipe.

Prospective Solution (To be discussed)

We would need to do the following:
Re-train the User-Personalization Model Recipe to have less bias towards “most hunched polls”.
Re-think the strategy of how to mix recommended polls from the 3 recipes as one of them (Popularity-Count) will always bias towards most hunched polls amongst active polls.
Right now following is the strategy to mix the recommended polls to create the User Feed:

Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
) instead.