Explore

PRD - Contextual PDP widgets

Problem statement

Current VC of Mall through PDP Reco is very low as compared to the category benchmarks because of historical conversion being put as one of the parameters. And we don’t have any avenues of intent channelisation on PDP with different recommendations like most of the competitors have.

Objective

Hence, utilising PDP to build awareness of Mall(and other programs) and channelise intent towards program specific similar products within a horizontal scroller.

Designs

Proto link:

⁠

https://www.figma.com/proto/1Ga9hGmsN6qEnTl8mx7hOH/Gold-PDP-Widget?page-id=97%3A26483&type=design&node-id=534-30515&viewport=-415%2C-3177%2C0.38&t=1KveKAlM4yxPDIKB-1&scaling=scale-down&starting-point-node-id=534%3A27366&show-proto-sidebar=1⁠

⁠

Lottie:

⁠

https://drive.google.com/file/d/1Tf8wtnhU1ZyVawZ4uzjGPSckUGWwSpKH/view?usp=sharing⁠

⁠

json and lottie link:

https://www.figma.com/file/1Ga9hGmsN6qEnTl8mx7hOH/Gold-PDP-Widget?type=design&node-id=995-23398&mode=design&t=8SCAMtUTmJSJkE9y-4⁠

⁠

Tech requirements

Taking help of an example, will try to expand on these requirements:

Ex: We have recently onboarded Bata as a brand on our platform. Due to very few interactions, and most of them from CLPs(Mall widgets/bottom nav), the conversion is poor as compared to other products in mens sandals. Hence, we want to create a PDP widget to help users get differentiated upfront discovery of Bata sandals when a user is looking at any Mens sandals’.

Capabilities for PDP widget

Capabilities for PDP widget

User story from example

Requirement

Admin panel configurability

Priority

POCs

Comments

Define to show widget on Mens sandals’ only above the price range of 400rs as starting range for Bata sandals is 500rs

Defining which parents’ products to show the widget for and which program widget to show

Must have

Indexation of global filter applied parent products being explored right now, which needs a DS POC→effort of 0.5 sprint

Alternate: Calling other services like taxonomy to add global filter of supplier_id→additional effort of 5-7 days

Getting Bata sandals similar to the parent Mens sandals

Reading from DS source to get child products for the parent product (content)

Should have

intent vs content

Defining widget to be visible only to users who have purchased at least one product of more than 500rs

Defining widget visibility across user cohorts(AB capability)

Must have

Showing widget only if we more than 3 similar to parent product sandals in Bata

Defining widget visibility only if more than X products pass the threshold from DS source

Not required

logic to be built at backend, if no of products returned<X, then send null response

Communicating introduction of Bata’s sandals , reinforcing them to be original and of high quality

Adding animation to the widget to scroll products horizontally

Must have

Three sample lottie animations will be configured for each program: Gold, Mall and loyalty which will then be used as reference for the app.

App release of 7th March being targeted for the same. Once released, same animation can be added to any other SDUI widget without app release.

Showing only those Bata sandals which are rated>3.5

Defining guardrails basis filters to further select the child products

Should have

no further filtering on child products possible in MVP

Showing sandals where users can earn loyalty coins in a different widget called Earn coins on PDP

Ability to define different sources to get child products from and sharing the program flag further or consuming widget-id for frontend to configure UI accordingly

Details mentioned below :

Requirement to utilise this for conducting other experiments for Gold, loyalty, trends etc?

⁠

Must have

backend will read this source from admin panel, and then pass same info to DS to consume child products of belonging config

Lottie animation would run for 2 sec as soon as widget comes on user's view port

Not required

Animation should not play if:

A user scrolled past the widget

A user saw the animation and went past and then scrolled back up again

A user interacted with the widget, scrolled or clicked and then came back

The transition from lottie to product cards would happen under 800 ms

Not required

Showing only Bata sandals and not the name

Removing product name from product card on horizontal scroller

Not required

Showing widget animation for 3 times if the user does not interact then not showing anymore

The lottie animation and transition would only be visible 3 times per widget per user, post which cards would be upfront visible

If user scrolls or clicks on widget before 3 times, don’t show animation post that.

Must have

Sunset to be handled via dynamic audiences

Not showing the Bata sandals present in PDP Reco

Deduping with PDP Reco

Should have

Showing communication for Bata introduction only for first time user

Ability to define different state of widgets for first time experience and other stages

Should have

Adding image as background

Adding pattern as background on widget group

Must have

Giving option of visiting Bata’s CLP to see all the products

Ability to define redirection to different CLP on sunset

Must have

Understanding how many users interacted with this widget, scrolled and clicked

Instrumentation of product views ( We’ll utilise the method RV does by using a query to calculate views at a widget level)

Not required

Defining budget for newly added Bata products and Ads on Bata products to maintain density

Ability to define budget views within the child products feed: explore or any other defined bucketing

Should have

There are no rows in this table

⁠

Requirement to utilise this for conducting other experiments for Gold, loyalty, trends etc?

Expanding on the above

example⁠

Mall stands for original branded products from select categories sourced from trusted sellers, whereas Gold is a premium proposition for unbranded occasional festive wear clothing which makes it high priced as compared to the rest of the platform. Both of these operate in limited and mutually exclusive categories as of now( in most scenarios) As both solves have different purposes to solve for, we would want to experiment on PDP widget for both separately. What is required here is the ability to experiment on showing Gold widget on PDP in the similar way as being solutioned for Mall. ex: Suggesting Gold anarkali kurtis when the user is looking at an anarkali kurti in the price range of 50% delta of average selling price of kurtis in Gold. Another extension to same requirement: For a supercoins/loyalty high affinity user, showing similar kurtis onboarded on loyalty where they can earn additional coins as incentive.

Detailed solve

Source: Mall catalogs from same sscat

Similarity score>’X’

PDP reco existing model or contextual DS model logic for similarity score

New model for similarity score

Priced within range of Y from product price(X)

does the Y change as X increases

Ratings and review filter: not less than Xpp from product ratings

Ranking:

based on historic conversion ?

Widget visibility criteria: At least Z products to qualify

CTA: should it be there or not what should it lead to

Deduping on or off( P0 or P1)

Particular user cohorts

Next steps:

Comparing intent vs content vs FLAVA: manually compare the output/relevance score being generated by all three models: intent is limited to 12M

Data help to get Mall/Gold products

Pareto products to check response on (DS+BI+product)

Identification of Mall products(tech solutioned)

Creating index for products: to check if indexing exist on Marvel

⁠

Similarity score generation

apply filters for parent products

for different programs

Rankers :

either being taken by the model we choose: combination of CTR and CVR models

or we define a new ranker

Creation of that feed source:

backend integration

building condition for widget visibility(X products to be there to show widget)

Sunset logic

Solution approaches discussed and MoM(15/02/24)

Parameter

Approach 1

Approach 2

Solve

Using PDP reco current model(combination of intent and content) to generate child products

Using fallback PDP reco model(OFS) to generate child products basis sscat

How is relevance ensured?

Intent: nearest neighbours to interaction on parent product

Content: basis name and description of products

belonging to same sscat

How are products ranked

ML ranker

ODNR

Pros

Variety and more relevant products will be generated using the current PDP Reco Model

Impact should be able to compensate for platform downside due to PDP Reco being pushed down

Can be built faster and used for experimentation

Cons

As a pre-requisite, team has suggested to build simulation to understand and confirm on the coverage of this widget

Filtering based approach is not suggested and indexation approach is suggested due to the output of filter not being in majority

Same set of products will be shown for all products belonging to that sscat(As a fix or hack, we can randomise ranking within these)

Confidence on impact being able to compensate for platform downside is lower

There are no rows in this table

⁠

Other points of discussion:

Parent product filtering is not required as we looked at the below data:

2. Showing only to cohort with high intent towards high ASP as a variant 3. Child products can have horizontal filters like ratings( if required) 4. We are checking with monetisation team parallely to understand state and coverage of Brands widget. If coverage is considerable, an experiment variant with keeping it along with our widget will be experimented, else only the variant with replacing ads widget will be experimented. Our hunch is that coverage should be limited as it is dependant on supply.

5. Discussed with Gagan and Nawazish, If the coverage of Mall products is not enough using current PDP Reco model within the feed of 200, then we can also understand ways or different models to generate more child products for such products.

Open points:

How will this variant scale with monetisation widget being present on PDP? (Dependency: Monetisation team)

How granular configuration will be done for CTA redirection as it needs to be done from admin panel itself? Hence, this will be operationally handled by creating different widget groups for different categories to define CTA redirection. (Confirmation to be taken form Mall biz)

Can this be used by other teams like loyalty where a filter can be applied for child products and all other capabilities be reused?

Do we need to build tenants for this real estate? (Dependency: Monetisation team)

Metrics

North star: Mall NMV/Vi

Secondary metrics:

NMV/Vi

Mall O/Visitors

Superport Mall OC

Mall V/Vi

Mall OC

V/Vi

VRS

Mall VC

Check metrics:

O/Vi

PDP Reco VC

PDP Reco OC

Variants planned

Variants planned

Variant

Users

No of widgets exposed on PDP

Sunset

Confidence

Reach

Impact on platform

Impact on VC

Priority score

Priority

All

Low

100

All

2(Along with Ads widget)

sunset if user doesn’t interact after seeing X times

Medium

100

All

sunset if user doesn’t interact after seeing X times

Medium

100

All

2(Along with Ads widget)

Low

100

Mall affinity users

2(Along with Ads widget)

Medium

Mall affinity users

High

Mall affinity users

2(Along with Ads widget)

sunset if user doesn’t interact after seeing X times

Medium

Mall affinity users

sunset if user doesn’t interact after seeing X times

High

There are no rows in this table

⁠

Parameters for experiment variants: parent product, child products, position, users, sunset logic, ads widget

As parent product will be a global filter, can’t be experimented at present as different variants.

Position will be decided basis the PDP framework. But to begin with, we want to start experimentation with below ratings and reviews.

Cohort based experiments are not possible with the abacus experimentation, hence post facto analysis on cohorts will be done.

Thus, six variants planned (prioritised marked in green):

Showing only national brand and D2C brand products’ PDP widget to all users

Showing all Mall products’ PDP widget to all users

Showing only national brand and D2C brand products’ PDP widget to all users along with scaled Ads widget to all users

Showing all Mall products’ PDP widget to all users along with Ads widgets to all users

Instrumentation

Interactions possible for widget:

Viewing widgets

Scrolling on widgets

Clicking on products within widget

Views need to be tracked for products. At present, all the widget views is essentially the same as product views for first two products but these don’t get tracked.

Product views need to be tracked in silver.mixpanel_android__widget_views_report table.

⁠

The following fields will have to be filled in this table :

widget_group_parent_catalog_id (parent catalog of the PDP page)

origin_metadata_widget_session_id

product_index

source_screen

User_id

widget_group_id

widget_group_parent_catalog_id(parent catalog of the PDP page)

product_id (of the products shown in the carousel)

origins (”pdp_widget”)

primary_real_estates (”pdp_widget”)

Will be adding this under :”themes”

(These fields are being reused from the Brand ads widget being experimented on PDP)

Product clicks will be tracked in silver.mixpanel_android__widget_clicked table

The following fields will have to be filled in this table:

Widget group ID

product_id

widget_group_type

reco_widget_product_id (Catalog ID is required)

origin_metadata_widget_session_id

product_index

User_id

origin (”pdp_widget”)

primary_real_estate (”pdp_widget”)

Will be adding this under :”themes”

reco_widget_product_id (Catalog ID is required)

catalog_opened: similar items(RV V1 code): widget_group_id, parent_catalog_id

order_attribution:

check for widget_group_id

Experiment design plan:

Variant 1: All Mall

Variant 2 : Only national and D2C brand products

Control

Experiment reads:

Early reads:

⁠

https://docs.google.com/spreadsheets/d/1Ey8sTRCVtUyhivKeaTH247E98ko3J4qdKEna3HFqLCM/edit#gid=0⁠

⁠

Coverage: ( Amount of PDP sessions where a user sees widget when compared with all PDP open sessions)

Reads:

Where is the widget visible? Coverage of widget : Mall vs Non mall in absolute product percentange %, VC covered

How were the north star metrics(L0) impacted due to this? Plat and Mall metrics: t1 vs t2

Why did t1 perform better than t2? AOV of orders on seller vs AOV of orders from national and D2C

What attraction has this widget been able to draw? What was the order contribution within different brand archetypes?

Were the check metrics impacted due to the experiment?

Appendix

For feed source creation[Deprioritised]

MVP(Evaluated and deprioritised due to high effort)

Ideal requirement: A SQL model table will be generated which maps parent product of Malls to child products on Mall belonging to same sscat. Hence, the ability to read from a table on the mappings and show result widget group. Details of schema mentioned below:

Parent product_id

Child product_id

Widget_id

Product index(ranking)

Position

Priority

Workaround: A SQL model table will be generated which maps parent product of Malls to CLPs. Hence, the ability to read from a table on the mappings and show result widget group. Using the same working of feed-in-feed widget creation but automating its creation via backend, as creation of widgets on individual products will be operationally impossible. Details of schema mentioned below:

Parent product_id

CLP

Widget_id

Position

Priority

3. PDP widget[P0]

Problem statement

Current VC of Mall through PDP Reco is very low as compared to the category benchmarks because of historical conversion being put as one of the parameters.

Objective

Utilising PDP to build awareness of Mall and upsell/ show similar program specific products within a horizontal scroller.

Solution

Proto link:

⁠

Lottie:

⁠

https://drive.google.com/file/d/1Tf8wtnhU1ZyVawZ4uzjGPSckUGWwSpKH/view?usp=sharing⁠

⁠

MVP capabilities for PDP widget

MVP capabilities for PDP widget

Requirement

Admin panel configurability

Priority

Defining which parents’ products to show the widget for and which program widget to show

Must have

Reading from DS source to get child products for the parent product (content)

Should have

Defining widget visibility across user cohorts(AB capability)

Must have

Defining widget visibility only if more than X products pass the threshold from DS source

Not required

Adding animation to the widget to scroll products horizontally

Must have

Defining guardrails basis filters to further select the child products

Should have

Ability to define different sources to get child products from and sharing the program flag further or consuming widget-id for frontend to configure UI accordingly

Details mentioned below :

Requirement to utilise this for conducting other experiments for Gold, loyalty, trends etc?

⁠

Must have

Lottie animation would run for 2 sec as soon as widget comes on user's view port

Not required

The transition from lottie to product cards would happen under 800 ms

Not required

Removing product name from product card on horizontal scroller

Not required

The lottie animation and transition would only be visible 3 times per widget per user, post which cards would be upfront visible

If user scrolls or clicks on widget before 3 times, don’t show animation post that.

Must have

Deduping with PDP Reco

Should have

Ability to define different state of widgets for first time experience and other stages

Should have

Adding pattern as background on widget group

Must have

Ability to define redirection to different CLP on sunset

Must have

Instrumentation of product views ( We’ll utilise the method RV does by using a query to calculate views at a widget level)

Not required

Ability to define budget views within the child products feed: explore or any other defined bucketing

Should have

There are no rows in this table

⁠

For feed source creation: (SQL model table)

MVP:

Parent product_id

Child product_id

Widget_id

Product index(ranking)

Position

Priority

Parent product_id

CLP

Widget_id

Position

Priority

Detailed solve

Creating a new DS model source:

Approach

Understanding of experiments done by PDP Reco and Ads team: contextual DS model

Categorising and understanding important parameters for similar products in these categories

Defining the product selection attributes and weightages for our model

Dedupe on or off?

Tech/DS solutioning for the model

Tech execution

Considerations:

Source: Mall catalogs from same sscat

Similarity score>’X’

PDP reco existing model or contextual DS model logic for similarity score

New model for similarity score

Priced within range of Y from product price(X)

does the Y change as X increases

Ratings and review filter: not less than Xpp from product ratings

Ranking:

based on historic conversion ?

Widget visibility criteria: Atleast Z products to qualify

Deduping on or off( P0 or P1)

Want to print your doc?
This is not the way.

Try clicking the ··· in the right corner or using a keyboard shortcut (

CtrlP

) instead.