Metrics
Key discussion points -
We observe a net ~3.05% jump in O/V offset by a ~3.14% drop in V/Vi leading to -0.18% negative impact on platform orders / visitor Gold - ~28% jump in V/Vi offset by ~18% drop in conversion leading to ~5% improvement in platform order / visitor. Platform T/Vi better for test by ~5% PDP reco - While O/V has improved for test by ~9%, It’s been offset by a sharp V/Vi drop of ~42% leading to a net negative impact on O/Vi for the real estate Usage / CTR - Click through rates for non FTUX variant stands at ~3%, while FTUX variant stands at ~20% Interaction profile - ~55% users are swiping to change the tabs while ~45% users are tapping on tabs Instrumentation update - Parallel feed was added as an origin and screen for base views and clicks tables. This has led to downstream issues in analytics processes where VC from PDP reco was found dropping due to attribution to the new real estate.
We have a discussion with ranking to define the SOP for new RE instrumentation since this might also affect ranker features. Suggestion from ranking team is to restore origin and screen mapping in views and clicks events back to ‘Recommendation’ while persisting additional fields for parallel feed tabs and position. This would need changes on our client end as well as modifications in order attribution models with a TAT of next week’s app release as well as some downtime for adoption. Ranking team to get back on whether this requires the experiment to be paused.
*Resolution build for FTUX variant to be pushed in Aug 01st app release.
Design-tech handover for this piece is done and we have carefully de-scoped certain pieces to try and target mid Aug app release for sufficient experimentation and scale up duration pre code freeze In tech solutioning, delivery timelines to be shared by early next week v1 would only solve for integration of icon / CTA per product card to showcase relevant Ingestion of a fallback parallel feed at the end of wishlist has been dropped owing to higher dev effort Other elements like removing shared / viewed sub tabs will be reviewed basis dev effort and might be descoped from the v1 build Ranking for comp swap products - The cheaper duplicates would have rating and price profile present to be able to decide the sort order of FIF. Planning to sort by price descending w/o baking in rating profile - To be checked and aligned. The new model for HVF has been setup for CLP and collections, Experiment for which will begin 1st Aug. Search to begin 5th Aug. Hero product experimentation - Model revamp picked Aug 01st, completion TAT - 6th Aug A/B support would need considerable effort from backend devs across taxonomy, discovery and ranking - Next steps be aligned with mentors Model creation to begin 06th Aug and timelines for release ~2nd week of Aug Feed classification for category / gender homogeneity - To help classify category / gender as valid labels for mid feed filters Mix of LLM based name classification and REs click profile Filter labels would be prioritised basis existing demand signal for the Real estate and mapping SSCAT RE demand signal (50% weight) - Signified by historical filter usage for the respective real estate - Reduce it further SSCAT demand signal (40% weight) - Search NER based which defines most searched attribute for a given category, Sample output - motorcycle sticker blue→ {'category': 'sticker', 'color': 'blue', 'type': 'motorcycle'} plain shirt blue color →{'category': 'shirt', 'color': 'blue', 'pattern': 'plain'} polyester backpack for women → {'category': 'backpack', 'material': 'polyester', 'gender': 'women'} 1 meter velvet blouse piece → {'category': 'blouse piece', 'material': 'velvet', 'size': '1 meter'} kid boy sandal → {'category': 'sandal', 'gender': 'boy', 'age group': 'kid'} Post filter label engagement score - TBU Value priority - Value ranking would remain similar as HVF - FLV Rank = (0.3* usage contribution)+(0.3* Normalised C/V)+(0.4* Normalised conversion) Skip label values where IF images don’t exist Values for price / size / combo / rating would follow a static approach Fallback - For feeds with sparse label data, fallback IFs would be selected from a mix of category invariant labels like price / size / colour / category / gender / combo and rating Fallback to be prioritised basis SSCAT level usage for feed’s mapping category Labels and values would be skipped per RE basis - Manual Biz / Ops QA input LLM / Name based exclusion - Signifies removing filter labels which are present in CLP name / search query, Ex - Limit labels where we are getting <4 unique values as part of model output Ensure the finalised values per labels have IF images on the BE We have initiated the image QA for interstitial filters with ops resourcing in parallel to the model setup
Other Planned experiments / delays -
DS filter label rationalisation Post deployment, devs found an issue with new DS label values not being properly tagged to catalogs due to increase in config size and heavy compute. Dev is on-going here and In parallel, we’ve initiated QC of the new labels from Biz. Resolution ETA - Mid of Aug Filter bar and basic filter screen UX refresh - Product / design solutioning to be kickstarted from next week Team to build a PoV on spending design bandwidth here vs parallel feed for other REs post experiment maturity Price sort gaps - Price sort is still broken even after fixes for comp swaps and discounts, Root cause indicates wrong pricing data getting stored on our end due to API call failures from pricing. Away resourcing to be planned to get this picked Run time sort from client - On hold due to dev allocation to other features