Metrics
Platform filter usage has reached prior baseline at ~5% with restoration of legacy HVF models on 02/08
Key discussion points -
We observe a net ~2.74% jump in O/V offset by a ~2.83% drop in V/Vi leading to -0.17% negative impact on platform orders / visitor. Net T/Vi negative at -0.14% (Pre period was homogenous) Gold - ~25% jump in V/Vi offset by ~15% drop in conversion leading to ~6.13% improvement in platform order / visitor. Platform T/Vi better for test by ~5.43% PDP reco - While O/V has improved for test by ~9%, It’s been offset by a sharp V/Vi drop of ~42% leading to a net negative impact on O/Vi for the real estate Usage / CTR - Click through rates for non FTUX variant stands at ~5%, while FTUX variant stands at ~30% Interaction profile - ~55% users are swiping to change the tabs while ~45% users are tapping on tabs At a platform level, we see V/Vi dropping massively leading to O/Vi and T/Vi drop V/Vi at RE level signifies the major drop is stemming from engagement going down for PDP reco Super port level trends - Map control’s V/Vi to ascertain drop is coming from the parallel feeds tab or native reco Build the view for affected super portfolios at a SSCAT level to understand the exact cases which are worst affected Drop is coming from people who switched the tab Check V/Vi for normal reco between test and control DoD trend - For plat metrics as well as PF level V/Vi Bounce rates - PLP → PDP click through rate between test and control P1 - Price bucket wise trends Details at a SSCAT level for affected super portfolios Gender level cut (User level trends per gender) Instrumentation alignment - We would be recording parallel_feed as part of primary real estate in base views and clicks events. Decision to be aligned cross org to align stakeholders on funnel attribution changes. TAT - 09/08
Resolution build for FTUX variant pushed in Aug 01st app release. Re launch planned for 13/08
Design-tech handover for this piece is done and we have carefully de-scoped certain pieces to try and target mid Aug app release for sufficient experimentation and scale up duration pre code freeze In tech solutioning, delivery timelines to be shared by early next week v1 would only solve for integration of content recos to be showcased in bottom sheet Bottom sheet to be triggered via the following variants - Build would go out across the following REs - Wishlist / CLP / Collections / FY and Search FTUX version will be built in for one of the above 2 variants owing to low timelines for MVP app release (Targeted for 15th Aug) A/B configuration would allow choosing between variants and enabling FTUX. Real estate level configuration is not supported in MVP due to high timelines Fallback feed for cases where PDP recos don’t exist to be finalised between ranking and BEx devs by today HVF - HVF v2 was launched on Aug 7th across REs, We are testing the following variants - IF - Model build for IF was picked on 08/08. Work in slow burner due to model prioritisation for hero product restoration Model setup TAT - 16th Aug, Experiment start TAT - 21st Aug Hero product model restoration - Current hero product logic works - At catalog level, we give priority to product with most number of transactors basis lifetime If there's a tie between products, then we take last uploaded pid in the catalog This logic changes for sscats from number of transactors to first first uploaded pid in Education+Healthcare part of Digital SuperPorts which has negligible orders. We deboost PIDs which have been invalid in the past two hours, while ranking. Next steps alignment - Given A/B would need significant dev effort across taxonomy and feed aggregator, we can approach this in the following way - SSCAT / Portfolio level A/B - Update for certain categories and observe data against other similar categories Straight rollout across SSCATs Factoring in order recency and moving away from unique transactors to net orders - Building the model based on last 1/3 months orders(for recency) + adding 1 year orders fallback for catalog with no recent orders Analytics check - Team will close checking on the OC which will get impacted at a super portfolio level with the update, To be closed 09/08 From a pre-post lens, team’s proposal is to compare conversion metric of the new model ( Pre vs Post) on RE X super portfolio level [ ObyV, CbyV , ObyC] Updates - Do a product level A/B (Check with Kunal / Rishita) DS filter label rationalisation - Post deployment, devs found an issue with new DS label values not being properly tagged to catalogs due to increase in config size and heavy compute. Dev is on-going here and In parallel, we’ve initiated QC of the new labels from Biz. Update - While dev would continue, deployment would be planned post MBS sale due to ramifications of the flow increasing latency of core services Given decent user comprehension of parallel feed MVP, team will start the conjoined product + design solutioning to - Design - Extend the build across REs Product - Identify high conversion intent levers to be surfaced per parallel feed (Category etc) Given search’s high conversion, We will pickup solutioning for integration of relevant high converting search queries as part of a feed Pre requisite query classification to enable this has already been developed as part of IFs UX for null filters showing on feed Selection outline of filters can be better