Skip to content

01/08/24


Metrics

Summary TBU -

Key discussion points -

We observe a net ~3.05% jump in O/V offset by a ~3.14% drop in V/Vi leading to -0.18% negative impact on platform orders / visitor
Gold - ~28% jump in V/Vi offset by ~18% drop in conversion leading to ~5% improvement in platform order / visitor. Platform T/Vi better for test by ~5%
PDP reco - While O/V has improved for test by ~9%, It’s been offset by a sharp V/Vi drop of ~42% leading to a net negative impact on O/Vi for the real estate
Usage / CTR - Click through rates for non FTUX variant stands at ~3%, while FTUX variant stands at ~20%
Interaction profile - ~55% users are swiping to change the tabs while ~45% users are tapping on tabs
Instrumentation update - Parallel feed was added as an origin and screen for base views and clicks tables. This has led to downstream issues in analytics processes where VC from PDP reco was found dropping due to attribution to the new real estate. We have a discussion with ranking to define the SOP for new RE instrumentation since this might also affect ranker features. Suggestion from ranking team is to restore origin and screen mapping in views and clicks events back to ‘Recommendation’ while persisting additional fields for parallel feed tabs and position. This would need changes on our client end as well as modifications in order attribution models with a TAT of next week’s app release as well as some downtime for adoption. Ranking team to get back on whether this requires the experiment to be paused.
*Resolution build for FTUX variant to be pushed in Aug 01st app release.
IG on Wishlist
Design-tech handover for this piece is done and we have carefully de-scoped certain pieces to try and target mid Aug app release for sufficient experimentation and scale up duration pre code freeze
In tech solutioning, delivery timelines to be shared by early next week
v1 scope -
v1 would only solve for integration of icon / CTA per product card to showcase relevant
Cheaper duplicates
Similar content options
Ingestion of a fallback parallel feed at the end of wishlist has been dropped owing to higher dev effort
Other elements like removing shared / viewed sub tabs will be reviewed basis dev effort and might be descoped from the v1 build
Ranking for comp swap products - The cheaper duplicates would have rating and price profile present to be able to decide the sort order of FIF. Planning to sort by price descending w/o baking in rating profile - To be checked and aligned.
HVF
The new model for HVF has been setup for CLP and collections, Experiment for which will begin 1st Aug. Search to begin 5th Aug.
Output comparison -
Screenshot 2024-08-01 at 10.37.50 AM.png
Hero product experimentation -
Model revamp picked Aug 01st, completion TAT - 6th Aug
A/B support would need considerable effort from backend devs across taxonomy, discovery and ranking - Next steps be aligned with mentors
Interstitial filters -
Model creation to begin 06th Aug and timelines for release ~2nd week of Aug
IF modelling approach -
Feed classification for category / gender homogeneity - To help classify category / gender as valid labels for mid feed filters
Mix of LLM based name classification and REs click profile
Label priority -
Filter labels would be prioritised basis existing demand signal for the Real estate and mapping SSCAT
RE demand signal (50% weight) - Signified by historical filter usage for the respective real estate - Reduce it further
SSCAT demand signal (40% weight) - Search NER based which defines most searched attribute for a given category, Sample output -
motorcycle sticker blue→ {'category': 'sticker', 'color': 'blue', 'type': 'motorcycle'}
plain shirt blue color →{'category': 'shirt', 'color': 'blue', 'pattern': 'plain'}
polyester backpack for women → {'category': 'backpack', 'material': 'polyester', 'gender': 'women'}
1 meter velvet blouse piece → {'category': 'blouse piece', 'material': 'velvet', 'size': '1 meter'}
kid boy sandal → {'category': 'sandal', 'gender': 'boy', 'age group': 'kid'}
Post filter label engagement score - TBU
Value priority - Value ranking would remain similar as HVF -
FLV Rank = (0.3* usage contribution)+(0.3* Normalised C/V)+(0.4* Normalised conversion)
Skip label values where IF images don’t exist
Values for price / size / combo / rating would follow a static approach
Fallback - For feeds with sparse label data, fallback IFs would be selected from a mix of category invariant labels like price / size / colour / category / gender / combo and rating
Fallback to be prioritised basis SSCAT level usage for feed’s mapping category
Label exclusion -
Labels and values would be skipped per RE basis -
Manual Biz / Ops QA input
LLM / Name based exclusion - Signifies removing filter labels which are present in CLP name / search query, Ex -
image (23).png
Limit labels where we are getting <4 unique values as part of model output
Ensure the finalised values per labels have IF images on the BE
We have initiated the image QA for interstitial filters with ops resourcing in parallel to the model setup

Other Planned experiments / delays -

DS filter label rationalisation Post deployment, devs found an issue with new DS label values not being properly tagged to catalogs due to increase in config size and heavy compute. Dev is on-going here and In parallel, we’ve initiated QC of the new labels from Biz. Resolution ETA - Mid of Aug
Filter bar and basic filter screen UX refresh - Product / design solutioning to be kickstarted from next week
Team to build a PoV on spending design bandwidth here vs parallel feed for other REs post experiment maturity
Sort issues -
Price sort gaps - Price sort is still broken even after fixes for comp swaps and discounts, Root cause indicates wrong pricing data getting stored on our end due to API call failures from pricing. Away resourcing to be planned to get this picked
Run time sort from client - On hold due to dev allocation to other features
Want to print your doc?
This is not the way.
Try clicking the ··· in the right corner or using a keyboard shortcut (
CtrlP
) instead.