Hunch RecSys

OP - Amazon Personalize Model Optimization

"Not Interested" Button & "Category" Widget: Data Science Perspective

Case Study: Specific User Spamming Hunch Primary Feed

Weekly Digest: Weekly Polls Recommendations to Cohorts

Explore

Case Study: Specific User Spamming Hunch Primary Feed

Problem Statement

We are facing an issue where a particular user’s polls (i.e. BihariBabu) are dominating the Hunch Primary Feed of many users. Click the below link to see an illustration:

https://hunchkins.slack.com/files/U5W4XHZ46/F056E196TAB/rpreplay_final1683614409.mp4⁠

⁠

Problem Root Cause Analysis

Following is the statistics of the Creators of current Active Polls:

CREATED_BY NUM_POLLS LIKELIHOOD

amanalok88@gmail.com 40 0.090090

syedumar21142019@gmail.com 28 0.063063

priya99.d53@gmail.com 21 0.047297

vipin.bitspilani2020@gmail.com 18 0.040541

wajiha1627@gmail.com 17 0.038288

dsr19112001@gmail.com 13 0.029279

hearttalks25@gmail.com 13 0.029279

dipankarjoshiparul@gmail.com 12 0.027027

meetahaldar1001@gmail.com 11 0.024775

yekkaliprem@gmail.com 11 0.024775

ananyasinghtomar52@gmail.com 11 0.024775

tanishqbatra2016@gmail.com 10 0.022523

There are 444 active polls from 128 unique Users (Creators). The topmost creator is

amanalok88@gmail.com⁠

(BihariBabu) who has created 40 of the active polls. Thus,

Likelihood of a poll being part of top-K recommendation feed is 40 / 444 = 0.090090

Further, below is the the statistics of active polls based on hunches > 200 and their creators:

CREATED_BY NUM_POLLS LIKELIHOOD

amanalok88@gmail.com 12 0.117647

vipin.bitspilani2020@gmail.com 9 0.088235

drshreya1609@gmail.com 7 0.068627

syedumar21142019@gmail.com 7 0.068627

ananyasinghtomar52@gmail.com 6 0.058824

priya99.d53@gmail.com 6 0.058824

As we can observe, the likelihood for polls having hunches greater than 200 increases for the user BihariBabu.

Our AWS Personalize Feed gets recommendations from 3 different models:

User-Personalization Model This model recipe has been tuned in a manner to recommend more hunched polls from the active polls at the top.

Trending-Now Model The Trending-Now model recipe generates recommendations for polls that are rapidly becoming more popular with our users.

Popularity-Count Model Popularity-Count recommends the most popular polls (more hunched polls) based on our user interactions data.

Given that of the 3 model recipes, 2 of them are biased to show more hunched polls at the top of the Personalised User Feed, “BihariBabu” dominates the feed as he has the most number of polls amongst active polls and they also come within the most hunched polls.

Since more than 95% of this user’s polls are of 30 days duration, these polls have accrued enough hunches to come in the most hunched polls while also being active.

The only model recipe which is not affected in bias towards the user “BihariBabu” is the Trending Now Model Recipe.

Prospective Solution (To be discussed)

We would need to do the following:

Re-train the User-Personalization Model Recipe to have less bias towards “most hunched polls”.

Re-think the strategy of how to mix recommended polls from the 3 recipes as one of them (Popularity-Count) will always bias towards most hunched polls amongst active polls.

Right now following is the strategy to mix the recommended polls to create the User Feed:

⁠

Phase 2: Hunch RecSys Engine⁠

⁠

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.