Skip to content
VariantsControl_1: No HVFs shown for all real estatesControl_2: Old model HVFs shown for all real estatesTest_CLP: New model HVFs shown for CLPTest_Search: New model HVFs shown for SearchTest_Collections: New model HVFs shown for CollectionsTest_all: New model HVFs shown for all three real estates UpdatesO/Vi and GMV/Vi are not yet significant.NMV/Vi and Net_orders/Vi are yet to matureObservationsWith old model as the control variantOrders per views have improved by 0.2% for the Test_all variant, whereas Views per visitor have dropped by 0.33% leading to a drop of 0.13% in Orders per visitor. Users ordering more but viewing less catalogs could be a possible reason for this trend.The Test_CLP variant showed a drop in all the funnel metrics. Orders per visitor and GMV per visitor for gold products have shown improvement against both control variants Views per visitor for mall products has increased against both control variantsNext stepsLow V/Vi - Understanding the real estates and the label values driving this - Could highlight the filter values where results are very offLow numbers for CLP - Isolating most affected CLPs and understanding if conversion is lagging for certain filter label values, like high usage of Mall / Gold / smartcoins
Platform funnel metrics -
Programs funnel metricsGoldMall
RE specific funnel metrics
RE wise V/Vi comparison
Total filter usage on PlatformHVF usage on Platform
HVF views - Position wise
HVF CTR - Position Wise
HVF CTR - Label Wise
Relaunch the experiment with updated models to limit filters with poor engagement/scroll depth affecting V/Vi, Current model only ranks basis historical filter usage. Estimated launch : 3rd week of JulOther launches on improved HVFs accessibility and usability to improve net CTR. ETA 2nd week of Aug
Relevance experiment
Detailed experiment reads across Programs and real estates combined
HVF experiment reads -
Team to debug reason for -
Metrics -
Old model as control
Old model as control
test_variants
O/V
V/Vi
O/Vi
GMV/Vi
Net_orders_Vi
NMV/Vi
Net AOV
Control_2
0.02%
0.02%
0.04%
0.05%
0.01%
-0.08%
-0.10%
Test_CLP
-0.04%
-0.10%
-0.14%
-0.02%
-0.15%
-0.14%
0.01%
Test_Collections
-0.08%
-0.07%
-0.15%
-0.10%
-0.21%
-0.29%
-0.08%
Test_Search
0.00%
-0.11%
-0.11%
0.04%
-0.10%
-0.08%
0.02%
Test_all
0.15%
-0.32%
-0.17%
-0.10%
-0.18%
-0.13%
0.04%
There are no rows in this table
No HVF as control
No HVF as control
test_variants
O/V
V/Vi
O/Vi
GMV/Vi
Net_orders/Vi
NMV/Vi
Net AOV
Control_1
-0.02%
-0.02%
-0.04%
-0.05%
-0.01%
0.08%
0.10%
Test_CLP
-0.05%
-0.12%
-0.18%
-0.07%
-0.16%
-0.06%
0.10%
Test_Collections
-0.10%
-0.09%
-0.19%
-0.14%
-0.23%
-0.21%
0.02%
Test_Search
-0.02%
-0.13%
-0.14%
-0.01%
-0.11%
0.00%
0.11%
Test_all
0.13%
-0.34%
-0.21%
-0.15%
-0.19%
-0.05%
0.14%
There are no rows in this table
1. CLP
2. Search
3. Collections
HVF v1 was launched across REs on Jun 26th.
The experiment led to Platform O/V improvement of 0.18%, but this was offset by a V/Vi drop of 0.31%, leading to a negative impact on orders per visitor.
For Gold, views have considerably improved whereas orders / view have dropped leading to a net positive impact on orders / visitors. NMV jump was limited due to low CTRs for the construct.
Metric cut
O/V
V/Vi
O/Vi
GMV/Vi
NMV/Vi
Platform
+0.15%
-0.32%
-0.17%
-0.1%
-0.13%
Gold
-1.25%
1.72%
0.45%
+0.8%
+2.35%
There are no rows in this table
Next Steps:
Want to print your doc?
This is not the way.
This is not the way.
Try clicking the ··· in the right corner or using a keyboard shortcut (
CtrlP
) instead.