Skip to content
Slack alerts for net sort usage dropL2 cuts on type of sort drop w.r.t. REDashboard setup to be expeditedDocument client side sort and filter failures
Support ops to quickly check filters visible for any given feed - Navigating to a given CLP on app is cumbersome - P0Mark filters as invalid for a given feed - P0Manual tagging of missing filter labels or filter label values as HVFs / IFs - P1Help establish an ops filter relevance metric - To be aligned, since this would need recurring ops bandwidth
Static slotting of HVFs basis historical usageAbsence of post application engagement features like O/V / C/V / Scroll depthStatic slotting - Only Gold will have static slotting across REsRanking of filter label values - Updated features for ranking - Usage (30% weight) : Denotes filter usage contribution for a given filter value out of all filter application for the feedEngagement (70% weight) : Denotes C/V, O/V post filter applicationNormalised C/V post filter application (30% weight) - Clicks happening on filtered feed catalogs within the same sessionNormalised O/V post filter application (40% weight) - Orders happening on catalogs viewed post filter application within the same dayFallback - Use real estate level O/V or C/V of labels surfacing in a filtered feed for cases with sparse filter interactioneg - C/V and O/V for Black coloured catalogs within a given CLP ID / collection ID / PSTInteractions to be considered per filter label value if data is coming in from >=100 users (Includes usage and engagement levers)Normalisation approach for O/V and C/V - Min max normalisation, i.e.Normalised metric = (x-xmin)/(xmax - xmin)FLV Rank = (0.3* usage contribution)+(0.3* Normalised C/V)+(0.4* Normalised conversion)Excluding redundant filter label values - Exclude FLVs where average scroll depth post filter application is lower by 60% or more than average scroll depth when any filter is applied on the feedExclude the FLVs per feed which were marked redundant by category / business inputsExclude size as a HVFCategory and gender to be skipped for homogenous real estates / feedsHomogeneity definitionStep 1 - Define a feed as homogenous basis CLP / collection name or search query via LLMThe prompt should be contextual to define a feed as SSCAT specific if both Gender and category information is baked onto the name for gender specific categories (Eg - Watch vs Men’s watch)For heterogenous classified feeds - Check SSCAT level OC in that feed, classify feeds as homogenous if 50% or more OC is from a single SSCAT, orCheck catalogs part of the feed, If 80% or more catalogs in the feed map to 1 SSCAT, classify homogenousFallback HVFs - In the case, final HVF counts getting handed over to tech <15 then - Prioritise fallback HVFs from a mix of - Mall / Low price / New arrival / Smartcoin / Top rated / Price filtersPositioning of filter label values per feed - Variant 1 - Meritocratic mix of FLVs as per final rank Variant 2 - Roll up FLV rank to their mapping labels and sort by label rank desc
Delays - Design updates - Parallel feedExperiment was launched on 23rd Jul but had to be stopped due to tech services going latent. Internal experiment to be relaunched today to check for service latency.
If things look fine post testing, we’d make the build live externally starting 25th Jul midnight (Midnight to support cache warmup with minimal RPS on the new service)Dynamic hero product Configs merged for the build within HVF - We have started an external experiment to test the tech stability and performance over this week. Experiment will be killed once we start HVF relaunchDS filter label rationalisation Post deployment, devs found an issue with new DS label values not being properly tagged to catalogs due to increase in config size and heavy compute. Tech solutioning is on-going here and we have parallely initiated QC of the new labels from Biz. ETA awaited from techPlanned experiments / tasks / CPD - IF modelling - To be kickstarted post HVF A/B start, 1st week of Aug onwards. Launch ETA - 2nd week of AugImage curation / QC for all filter label values (IF and HVF v2 UI) - To be kickstarted next weekIG on wishlist - Design review with Kirti to be scheduledParallel feeds designs for FY and other feeds - Design solutioning to begin in last week of JulPrice sort gaps - Price sort is still broken even after fixes for comp swaps and discounts, Root cause indicates wrong pricing data getting stored on our end due to API call failures from pricing. , In parallel pod’s prioritising run time sort on client end for top 100 catalogs to bypass price sort leakages due to user / zonal discounting / other undetected issues
Hero product model re-instatement and A/B - Deprioritised since analytics bandwidth within team primary allocated to HVF and IF revamp. To be tentatively started in sec week of Aug, Would need additional analytics bandwidth to expedite thisWidget framework for PDP and search - Discussed in Kaizen with Megha. We’d need to build out the prioritisation framework for widgets part of PDP, widgets and IFs part of feeds. Needs work since a lot of cross teams are leveraging FIFs for their specific use case i.e. Gold / Mall / Non GST / Pricing - Parked / De-prioritised, To be picked post IF / parallel feed experiment startIF LLM experiment - Recommendation is to de-prioritise IF LLM build due to - Competing ops bandwidth requirements to - Parallel feed image QCDS label QCImage QC for filter label value for broader IF model startAnalytics and product bandwidth primarily focusing on HVF re-modelling / parallel feed experimentation and IF baseline model setupLimited audience needed for HVF re experiment / DS label experiment and parallel feedsTeam would pick up the LLM variant testing with broader IF model experiment
Issues in price sort - Incorrect catalogs getting returnedIssues in all label and label values not showing up in static filter screens - Eg Size etcFound this driven by label configs marked invalid on sort and filter infra - Bug resolution ETA - 27th JuneIntegration of more homogenisation levers in ABACUS - RE specific metrics like PDP reco V/Vi / O/Vi / NMV/Vi etc needs to be integrated onto ABACUS for future experiments - To be picked
Dedicated QA support to - Test HVF build / dynamic hero product testing etcEnabling recurring CPD by continually testing sort and filter performance across REs
Agenda
Metrics -
Key discussion points -
Sort drop deep dive - We have observed sort usage dropping for search starting Jul. Team to get back on the RCA. Alongside, team would be setting up
Other sort issue - High to low and discount failures due to pricing API fetch failures. Erroneous price values leading to incorrect sort operation, Team adding API reattempts to curb such events organically. Proper fix requires a cache refresh / bootstrap run which needs to be aligned with engineering.
Ops filter QC console
We are planning to picks an admin console to review,QC and edit final filters visible for a given feed to -
HVF RELAUNCH DESIGN -
The v1 experiment for HVF concluded with a +0.15% improvement in O/V offset but a -0.31% downside in V/Vi. The model build failed to achieve the expected O/Vi jump due to -
Team is reworking the model to include clicks, views, and orders post-filter application. Additionally, an experiment with a real estate variant will slot Gold as static to analyse the program's view upside versus platform conversion downside.
Experiment relaunch is slated for 31st July.
Updated approach / New features for HVF ranking -
Experiment design -
HVF experiment would be co launched with dynamic hero product capability for CLP and collections, This however cannot be extended to search due to a different planes needed for search and likely overlap with 2 similar experiments running on sibling planes.
Proposed variants -
Variant
HVF model
Dynamic hero support
Static slotting for Gold
Experiment plane
TEST_CLP
CLP - New
Collections - Old
Search - Old
CLP - Yes
Collections - Yes
Search - No
No
Future use 8
TEST_COLLECTIONS
CLP - Old
Collections - New
Search - Old
CLP - Yes
Collections - Yes
Search - No
No
Future use 8
TEST_SEARCH
CLP - Old
Collections - Old
Search - New
CLP - No
Collections - No
Search - No
No
Future use 8
TEST_ALL_RANDOMISED
CLP - New
Collections - New
Search - New
CLP - Yes
Collections - Yes
Search - No
No
Future use 8
TEST_ALL_LABEL_SORTING
CLP - New
Collections - New
Search - New
CLP - Yes
Collections - Yes
Search - No
No
Future use 8
Control_Old_Model
CLP - Old
Collections - Old
Search - Old
CLP - No
Collections - No
Search - No
No
Future use 8
There are no rows in this table
PARALLEL FEED EXPERIMENT DESIGN -
PARALLEL FEED EXPERIMENT DESIGN
Experiment tracker / Planned experiments / Delays -
Experiment design
De-prioritisation alignment -
CPD -
Support required -
KR updates -
Want to print your doc?
This is not the way.
This is not the way.
Try clicking the ··· in the right corner or using a keyboard shortcut (
CtrlP
) instead.