Skip to content

09/08/24

Metrics

Platform filter usage has reached prior baseline at ~5% with restoration of legacy HVF models on 02/08

Key discussion points -

We observe a net ~2.74% jump in O/V offset by a ~2.83% drop in V/Vi leading to -0.17% negative impact on platform orders / visitor. Net T/Vi negative at -0.14% (Pre period was homogenous)
Gold - ~25% jump in V/Vi offset by ~15% drop in conversion leading to ~6.13% improvement in platform order / visitor. Platform T/Vi better for test by ~5.43%
PDP reco - While O/V has improved for test by ~9%, It’s been offset by a sharp V/Vi drop of ~42% leading to a net negative impact on O/Vi for the real estate
Usage / CTR - Click through rates for non FTUX variant stands at ~5%, while FTUX variant stands at ~30%
image.png
Interaction profile - ~55% users are swiping to change the tabs while ~45% users are tapping on tabs
Reason for T/Vi drop -
At a platform level, we see V/Vi dropping massively leading to O/Vi and T/Vi drop
V/Vi at RE level signifies the major drop is stemming from engagement going down for PDP reco
Screenshot 2024-08-09 at 10.56.15 AM.png
Pending next steps -
Map control’s V/Vi to ascertain drop is coming from the parallel feeds tab or native reco
Build the view for affected super portfolios at a SSCAT level to understand the exact cases which are worst affected
To check -
Drop is coming from people who switched the tab
Check V/Vi for normal reco between test and control
DoD trend - For plat metrics as well as PF level V/Vi
Bounce rates - PLP → PDP click through rate between test and control
P1 - Price bucket wise trends
Details at a SSCAT level for affected super portfolios
Gender level cut (User level trends per gender)
Instrumentation alignment - We would be recording parallel_feed as part of primary real estate in base views and clicks events. Decision to be aligned cross org to align stakeholders on funnel attribution changes. TAT - 09/08
Resolution build for FTUX variant pushed in Aug 01st app release. Re launch planned for 13/08
IG on Wishlist
Design-tech handover for this piece is done and we have carefully de-scoped certain pieces to try and target mid Aug app release for sufficient experimentation and scale up duration pre code freeze
In tech solutioning, delivery timelines to be shared by early next week
MVP scope -
v1 would only solve for integration of content recos to be showcased in bottom sheet
Bottom sheet to be triggered via the following variants -
Screenshot 2024-08-09 at 10.51.58 AM.png
Screenshot 2024-08-09 at 10.52.06 AM.png
Screenshot 2024-08-09 at 10.56.14 AM.png
Build would go out across the following REs -
Wishlist / CLP / Collections / FY and Search
FTUX version will be built in for one of the above 2 variants owing to low timelines for MVP app release (Targeted for 15th Aug)
A/B configuration would allow choosing between variants and enabling FTUX. Real estate level configuration is not supported in MVP due to high timelines
Fallback feed for cases where PDP recos don’t exist to be finalised between ranking and BEx devs by today
HVF - HVF v2 was launched on Aug 7th across REs, We are testing the following variants -
Variant
Dynamic hero enabled
Model type
Control_1
No
Old HVF
Control_2
No
No HVF
Test_1
Yes, Across REs
Meritocratic build live for all REs
Test_2
No
Meritocratic build live for all REs
Test_3
Yes, Across REs
Label wise slotting for all REs
Test_4
Yes, on CLP
Meritocratic build live for CLPs, Other REs on old HVFs
Test_5
Yes, on Search
Meritocratic build live for search, Other REs on old HVFs
Test_5
Yes, on Collections
Meritocratic build live for collections, Other REs on old HVFs
There are no rows in this table
IF - Model build for IF was picked on 08/08.
Approach
Work in slow burner due to model prioritisation for hero product restoration
Model setup TAT - 16th Aug, Experiment start TAT - 21st Aug
Hero product model restoration -
Current hero product logic works -
At catalog level, we give priority to product with most number of transactors basis lifetime
If there's a tie between products, then we take last uploaded pid in the catalog
This logic changes for sscats from number of transactors to first first uploaded pid in Education+Healthcare part of Digital SuperPorts which has negligible orders.
We deboost PIDs which have been invalid in the past two hours, while ranking.
Next steps alignment - Given A/B would need significant dev effort across taxonomy and feed aggregator, we can approach this in the following way -
SSCAT / Portfolio level A/B - Update for certain categories and observe data against other similar categories
Straight rollout across SSCATs
Logic updates -
Factoring in order recency and moving away from unique transactors to net orders - Building the model based on last 1/3 months orders(for recency) + adding 1 year orders fallback for catalog with no recent orders
Analytics check - Team will close checking on the OC which will get impacted at a super portfolio level with the update, To be closed 09/08
Success metric (TBC) -
From a pre-post lens, team’s proposal is to compare conversion metric of the new model ( Pre vs Post) on RE X super portfolio level [ ObyV, CbyV , ObyC]
Updates - Do a product level A/B (Check with Kunal / Rishita)
DS filter label rationalisation - Post deployment, devs found an issue with new DS label values not being properly tagged to catalogs due to increase in config size and heavy compute. Dev is on-going here and In parallel, we’ve initiated QC of the new labels from Biz.
Update - While dev would continue, deployment would be planned post MBS sale due to ramifications of the flow increasing latency of core services
Roadmap / CPD -
Given decent user comprehension of parallel feed MVP, team will start the conjoined product + design solutioning to -
Design - Extend the build across REs
Product - Identify high conversion intent levers to be surfaced per parallel feed (Category etc)
TBP - Last week of Aug
IC via search -
Given search’s high conversion, We will pickup solutioning for integration of relevant high converting search queries as part of a feed
Pre requisite query classification to enable this has already been developed as part of IFs
TBP - Mid Sep
UX for null filters showing on feed
Selection outline of filters can be better



Want to print your doc?
This is not the way.
Try clicking the ··· in the right corner or using a keyboard shortcut (
CtrlP
) instead.