Explore

v1 launch RCA

Parallel feed launch deep dive -

The launch of the parallel feed feature resulted in a ~3% increase in Orders per View (O/V) but a ~3% decrease in Views per Visitor (V/Vi), leading to negligible changes in Orders per Visitor (O/Vi) on the platform. The PDP recommendations (PDP reco) were the most affected, with a notable drop in V/Vi.

The click-through rate (CTR) for the feature was 3.5% to 5% of all PDP opens, which contributed to ~13% drop in reco’s V/Vi.

Analysis showed that users who visited only the first recommendation tab had significantly lower V/Vi compared to those navigating parallel feeds, which had ~41% higher V/Vi than the control group.

About 85% of users primarily consumed recommendations from the first tab, contributing to the V/Vi drop. Issues identified included -

Scrolling problems on the parallel feed, causing users who previously contributed >40 catalog views to shift to fewer views, impacting V/Vi by ~40%.

⁠

https://meesho.slack.com/files/U0403H452F5/F07GP4MD4HY/vid_20240813140339.mp4⁠

⁠

Fast upward scrolling, which skipped the recommendation section, also reduced duplicate view counts, contributing ~15% to the V/Vi drop

⁠

https://meesho.slack.com/files/U0403H452F5/F07GP4N3MQS/vid_20240813140549.mp4⁠

⁠

The analysis suggested that V/Vi drops were consistent across categories, indicating relevance or selection was not the main driver. A list of SSCATs to potentially skip in future relaunches is being considered, based on further analysis of user behaviour - TBU 14th Aug

Next steps -

Navigational issues leading to V/Vi drop for first / similar tab is being investigated by Android, could need an app update for fix

Team to finalise the SSCATs to be skipped where parallel feed V/Vi is significantly poorer than control’s V/Vi - Analysis taking time due to instrumentation gaps

Instrumentation fix to be picked by Android as part of app release for (1)

Details -

On an overall platform front, O/V increased by ~3% for test w.r.t control. However platform V/Vi went down by ~3% leading to a negligible change of O/Vi on platform

Metric

Test

Control

Delta

platform

TbyVi

0.13

-0.15%

platform

ObyV

3.07%

platform

VbyVi

1,773.89

1,828.48

-2.99%

platform

ObyVi

0.22

-0.03%

pdp reco

VbyVi

779.05

941.34

-17.24%

There are no rows in this table

⁠

Diving deep into the dropping V/Vi, highlights PDP reco as the worst affected RE -

⁠

CTR trend for the feature stood at ~~3.5%-5% of all PDP opens and didn’t effectively add up to ~13% drop in REs V/Vi

⁠

To analyse this further, team looked at PDP reco visitor metrics for users using parallel feeds vs those not (Analysis done at user cohort’s funnel metrics per day, session level wasn’t possible due to instrumentation gaps) -

Users who just visited the recommendation tab had a significantly lower Views per Visitor (V/Vi) compared to those navigating through parallel feeds. Specifically, users visiting parallel feeds had ~41% higher V/Vi than control

~85% users primarily consume reco from the first tab which is indicative of feature’s CTR leading the primary V/Vi drop

Cohort

VbyVi

ObyVi

TbyVi

User Distribution

scroll_depth

PDP view share

Test - Users only browsing Similar

265.18

0.69

0.08

85.10%

32.82

72.3%

Test - Users browsing parallel Feed

580.41

1.23

0.12

14.90%

39.87

27.7%

Control - PDP recos

411.26

2.38

0.09

100%

31.17

There are no rows in this table

⁠

Upon stress testing the build and hypothesising issues that could lead to drop in similar / first tab’s V/Vi, highlight the following issues -

Scroll down gets stuck when a user lands on parallel feed

⁠

https://meesho.slack.com/files/U0403H452F5/F07GP4MD4HY/vid_20240813140339.mp4⁠

⁠

Users who contributed >40+ catalog views on an average on PDP recos in test cohort shifted to buckets contributing <20 views with a significant switch to <=5 views after the launch of experiment (26th July)

⁠

Sizing and impact - 54% of control users give 91% views from 40+ bucket. A 5% shift of this user base to cohorts a and b, leads to a drop of at least 6.75% in total views contributing to ~40% of net V/Vi drop

Up scrolls on a parallel feed directly takes the user to product description

Usually duplicate views are computed per catalog even if the user is scrolling back towards the parent listing / PDP start. This trend has been falling as well post the launch of experiment which could likely be due to the fast scroll up which skips reco section.

To analyse the impact, we calculated V/Vi using two approaches -

V/Vi (Distinct catalogs) - Distinct catalog views / visitors

V/Vi (Total) - Total catalog views / visitors

The delta between both types of V/Vi between test and control was consistent pre period, but took a sharp dip post launch indicating V/Vi (Total) dropped more significantly than V/Vi (Distinct).

⁠

When analysing VbyVi for users who were only in similar feed and saw more than 40 catalogs, VbyVi distinct dropped by 5.6% but VbyVi total dropped by an additional 2.3%. Platform impact would be at least 2.07% contributing to ~15% of net V/Vi drop.

To assess V/Vi

In depth analysis of the drop in V/Vi at a super portfolio and SSCAT level revealed that V/Vi was impacted irrespective of the number of catalogs of that SSCAT in Gold and Mall categories. Thus indicating that relevance/selection is not the major driver of the V/Vi drop

V/Vi delta between test with respect to control variant at Super portfolio level -

⁠

Tentative list of SSCATs that could be skipped from the next relaunch - To be finalised by assessing super portfolio and SSCAT level funnel behaviour between users who navigate to parallel feeds vs control

⁠

Appendix : Input for GPT :

Parallel feed launch deep dive -

The click-through rate (CTR) for the feature was 3.5% to 5% of all PDP opens, which contributed to ~13% drop in reco’s V/Vi.

Analysis showed that users who visited only the first recommendation tab had significantly lower V/Vi compared to those navigating parallel feeds, which had ~41% higher V/Vi than the control group.

About 85% of users primarily consumed recommendations from the first tab, contributing to the V/Vi drop. Issues identified included -

Scrolling problems on the parallel feed, causing users who previously contributed >40 catalog views to shift to fewer views, impacting V/Vi by ~40%.

⁠

https://meesho.slack.com/files/U0403H452F5/F07GP4MD4HY/vid_20240813140339.mp4⁠

⁠

Fast upward scrolling, which skipped the recommendation section, also reduced duplicate view counts, contributing ~15% to the V/Vi drop

⁠

https://meesho.slack.com/files/U0403H452F5/F07GP4N3MQS/vid_20240813140549.mp4⁠

⁠

Next steps -

Navigational issues leading to V/Vi drop for first / similar tab is being investigated by Android, could need an app update for fix

Team to finalise the SSCATs to be skipped where parallel feed V/Vi is significantly poorer than control’s V/Vi - Analysis taking time due to instrumentation gaps

Instrumentation fix to be picked by Android as part of app release for (1)

Details -

On an overall platform front, O/V increased by ~3% for test w.r.t control. However platform V/Vi went down by ~3% leading to a negligible change of O/Vi on platform

Metric

Test

Control

Delta

platform

TbyVi

0.13

-0.15%

platform

ObyV

3.07%

platform

VbyVi

1,773.89

1,828.48

-2.99%

platform

ObyVi

0.22

-0.03%

pdp reco

VbyVi

779.05

941.34

-17.24%

There are no rows in this table

⁠

Diving deep into the dropping V/Vi, highlights PDP reco as the worst affected RE -

The data table shows metrics comparing Control and Test groups across different real estate placements within an application. Key metrics like views, viewers, and delta are listed for different areas. Here's an interpretation:

Columns in the Table:

Real_estate: Different app sections or placements being analyzed.

Control_viewers: Number of unique viewers who interacted with the real estate in the control version.

Control_views: Total number of views the real estate received in the control version.

Test_viewers: Number of unique viewers who interacted with the real estate in the test version.

Test_views: Total number of views the real estate received in the test version.

Delta: The percentage change or relative difference between the control and test versions, indicating the impact of changes made.

Summary of Real Estate Placements:

PDP_Reco:

Test version experienced a notable decrease in views with a delta of -13.24%.

catalog_listing_page:

Test version saw a -1.51% decrease in views, suggesting a slight drop in engagement.

single_collection:

Test views increased by 1.23%, indicating positive engagement.

catalog_search_results:

Test version had a positive 3.67% delta, indicating improved engagement with search results.

FY_feed:

Significant increase in engagement with a 4.31% delta in the test version.

Summary of CTR Trend Graph for Premium and Branded Tabs

The line graph shows the Click-Through Rate (CTR) trends over time for different types of tabs: Combined, Premium, and Branded. Here are the key insights:

Graph Elements:

X-axis: Represents the date, ranging from 2024-07-27 to 2024-08-08.

Y-axis: Represents the CTR values.

Three Trend Lines:

CTR_combined (Blue): Represents the CTR for both Premium and Branded tabs combined.

CTR_premium (Red): Represents the CTR for the Premium tab.

CTR_branded (Yellow): Represents the CTR for the Branded tab.

CTR Trends:

CTR_combined:

Started around 6 and gradually declined over time, reaching 4 by 2024-08-08.

CTR_premium:

Started at around 4.5 and showed a decline, reaching around 3 by the end of the observation period.

CTR_branded:

Started around 2 and continued to decrease slightly, ending at just above 1.5.

General Insight:

All three CTR trends show a decline over the given period, with CTR_combined experiencing the steepest decrease.

The Premium tab consistently outperformed the Branded tab in terms of CTR, while the Branded tab had the lowest CTR throughout the time range.

Combined Summary:

The data table suggests mixed performance, with some sections showing positive improvement in views, while others, like PDP_Reco, experienced a significant decline.

The CTR trend graph indicates a general decrease in user engagement for Premium and Branded tabs over time. The Premium tab had a better CTR compared to the Branded tab, but both saw a downward trend.

This information can be used to assess the effectiveness of the recent changes made in the application and highlight areas that need improvement, particularly in the PDP Recommendations and maintaining CTR for Premium and Branded tabs.

4o with canvas

window.__oai_logHTML?window.__oai_logHTML():window.__oai_SSR_HTML=window.__oai_SSR_HTML||Date.now();requestAnimationFrame((function(){window.__oai_logTTI?window.__oai_logTTI():window.__oai_SSR_TTI=window.__oai_SSR_TTI||Date.now()}))

CTR trend for the feature stood at ~~3.5%-5% of all PDP opens and didn’t effectively add up to ~13% drop in REs V/Vi

~85% users primarily consume reco from the first tab which is indicative of feature’s CTR leading the primary V/Vi drop

Cohort

VbyVi

ObyVi

TbyVi

User Distribution

scroll_depth

PDP view share

Test - Users only browsing Similar

265.18

0.69

0.08

85.10%

32.82

72.3%

Test - Users browsing parallel Feed

580.41

1.23

0.12

14.90%

39.87

27.7%

Control - PDP recos

411.26

2.38

0.09

100%

31.17

There are no rows in this table

⁠

Upon stress testing the build and hypothesising issues that could lead to drop in similar / first tab’s V/Vi, highlight the following issues -

Scroll down gets stuck when a user lands on parallel feed

⁠

https://meesho.slack.com/files/U0403H452F5/F07GP4MD4HY/vid_20240813140339.mp4⁠

⁠

The graph represents Day-over-Day (DOD) viewer delta for different view buckets compared to a control group. The lines illustrate the changes in viewer behavior across different segments from 2024-07-21 to 2024-08-11.

Graph Details:

X-axis (Date):

The timeline ranges from 2024-07-21 to 2024-08-11, depicting how viewer deltas change over time.

Y-axis (Delta Percentage):

The Y-axis represents the percentage change in viewers compared to the control group.

Values range from -10.00% to 10.00%.

Line Representation of View Buckets:

a. < 5 (Blue Line):

Represents the delta for viewers with fewer than 5 views.

Initially stays close to 0%, but increases around 2024-07-28 and maintains a positive delta above 5% from early August onwards.

b. 5-20 (Red Line):

Represents viewers with between 5 and 20 views.

Slightly increases around 2024-07-28 and hovers around 4-5% after that.

c. 20-40 (Yellow Line):

Represents viewers with 20 to 40 views.

Remains steady at around 0% throughout the entire period with minimal fluctuation.

b. 40+ (Green Line):

Represents viewers with more than 40 views.

Displays the most significant negative drop around 2024-07-28, reaching close to -10%, but gradually recovers and approaches 0% towards the end of the observed period.

Insights:

Positive Delta Segments:

Viewers in the < 5 and 5-20 view buckets show an increase in delta over time, especially from the end of July onward. The < 5 bucket shows a consistent positive delta above 5% in early August, suggesting higher engagement among new or infrequent viewers.

Stable Segment:

The 20-40 view bucket remains steady, with minimal fluctuation and no significant deviation from the control, indicating consistent engagement.

Negative Delta Segment:

The 40+ view bucket experienced a notable decline in delta around late July, dropping to nearly -10%. This indicates a significant drop in engagement among frequent viewers during that period, although it starts to recover towards mid-August.

General Conclusion:

The < 5 and 5-20 view segments show a positive increase, which suggests that the changes may be more appealing to newer or less frequent users.

The 40+ view segment experienced a significant decrease but showed signs of recovery towards the end of the observation period.

The 20-40 segment remained unaffected, with stable engagement throughout the timeline.

This graph provides insights into how different segments of viewers are reacting to changes in the product or content, with a focus on whether these changes are affecting frequent viewers differently from occasional viewers.

Up scrolls on a parallel feed directly takes the user to product description

To analyse the impact, we calculated V/Vi using two approaches -

V/Vi (Distinct catalogs) - Distinct catalog views / visitors

V/Vi (Total) - Total catalog views / visitors

The delta between both types of V/Vi between test and control was consistent pre period, but took a sharp dip post launch indicating V/Vi (Total) dropped more significantly than V/Vi (Distinct).

⁠

To assess V/Vi

V/Vi delta between test with respect to control variant at Super portfolio level -

Summary of Data Table for Super Portfolio

The table shows various super_portfolio categories along with their delta, gold_catalogs, mall_catalogs, and VC (Visibility Coverage). Each category represents a different product type in the portfolio, with metrics highlighting performance, representation in catalogs, and overall visibility.

Column Descriptions:

super_portfolio:

Represents the different product categories or types of items within the portfolio.

delta:

Indicates the percentage change (presumably negative, as they are all in red) in performance, perhaps reflecting a drop in metrics like sales, visibility, or engagement compared to previous periods.

gold_catalogs:

Number of products or catalogs designated as "Gold" (often implying high quality or special curated items) for each category.

mall_catalogs:

Number of products or catalogs available in the Mall section for each category.

VC (Visibility Coverage):

Indicates the percentage of visibility or coverage each product category gets. Higher percentages mean greater visibility in the marketplace or application.

Key Highlights by Category:

High Delta Declines:

WWW (Women, Wear, and Wellness):

Delta of -16.41%. Significant decline.

Gold Catalogs: 6231.

Mall Catalogs: 57,604.

VC: 11.31%.

Men Fashion:

Delta of -15.77%.

Gold Catalogs: 1,877.

Mall Catalogs: 31,141.

VC: 2.84%.

Women Sarees:

Delta of -15.22%.

Gold Catalogs: 5,946.

Mall Catalogs: 2,173.

VC: 12.25%.

Women Kurtis, Kurta Sets & Suits:

Delta of -14.92%.

Gold Catalogs: 5,780.

Mall Catalogs: 7,659.

VC: 19.02%.

Moderate Delta Declines:

Bags, Luggage & Travel Accessories:

Delta of -14.16%.

Gold Catalogs: 4,318.

Mall Catalogs: 5,131.

VC: 2.01%.

Books:

Delta of -12.98%.

Gold Catalogs: 0.

Mall Catalogs: 25,000.

VC: 0.11%.

Kids:

Delta of -11.76%.

Gold Catalogs: 10,079.

Mall Catalogs: 11,772.

VC: 8.98%.

Mobile Cases, Covers and Screen Guards:

Delta of -10.91%.

Gold Catalogs: 0.

Mall Catalogs: 6.

VC: 0.58%.

Low Delta Declines:

Watches:

Delta of -10.37%.

Gold Catalogs: 1,218.

Mall Catalogs: 1,544.

VC: 0.80%.

Women and Men Accessories:

Delta of -9.49%.

Gold Catalogs: 8,136.

Mall Catalogs: 7,319.

VC: 1.68%.

Health & Wellness:

Delta of -6.25%.

Gold Catalogs: 0.

Mall Catalogs: 7,858.

VC: 0.51%.

Eye Utility:

Delta of -2.35%.

Gold Catalogs: 5.

Mall Catalogs: 0.

VC: 0.01%.

Mobiles, Electronics Accessories & Small Appliances:

Delta of -1.69%.

Gold Catalogs: 1.

Mall Catalogs: 1,037.

VC: 0.47%.

General Insights:

Declines in the delta metric are present across all categories, with varying levels of severity.

WWW, Men Fashion, and Women Sarees exhibit some of the highest delta declines, indicating a major drop in metrics such as visibility, engagement, or sales.

Gold Catalog Presence is quite high for certain categories like Women Jewellery (34,452) and Home Decor & Furnishings (35,863).

Mall Catalogs have considerable representation for WWW, Women Jewellery, and Beauty & Personal Care.

Visibility Coverage (VC) is highest for Women Kurtis, Kurta Sets & Suits (19.02%) and WWW (11.31%), indicating these categories have the most exposure.

This table provides a detailed look at the portfolio's performance across product categories, highlighting areas where visibility and catalog representation need attention to address significant declines.

4o with canvas

Summary of Super Portfolio Table

The table provides details about different product categories (super_portfolio) and their respective subcategories (ss_cat). Metrics presented include platform visibility, delta in performance, total catalogs, gold catalogs, and mall catalogs. Each row details the current state of a particular subcategory.

Column Descriptions:

super_portfolio:

Broad categories of products, like WWW (likely Women, Wear, and Wellness), Women Kurtis, Sarees, Jewellery, Kids, etc.

ss_cat:

Subcategories within each super portfolio, e.g., Tops & Tunics, Dresses, Kurtis & Kurtas, Suits & Dress Materials, etc.

plat vc (Platform Visibility Coverage):

Percentage of how visible these categories are across the platform. This gives an idea of how prominent each subcategory is for users.

VbyVi delta:

Indicates a change in performance compared to a reference period (likely negative as all values are below zero). This could represent changes in metrics such as views or conversions for each subcategory.

Total catalogs:

The total number of catalogs available for each subcategory.

Gold catalogs:

Catalogs designated as "Gold," implying higher quality or top picks.

Mall catalogs:

Number of catalogs available in the Mall section for each subcategory.

Key Highlights by Category:

High Delta Declines:

Trousers & Pants (WWW):

VbyVi delta: -16.8% (one of the highest declines).

Total Catalogs: 44,935.

Gold Catalogs: 443.

Mall Catalogs: 2,270.

Rakhi (Women Jewellery):

VbyVi delta: -16.2%.

Total Catalogs: 96,559.

Gold Catalogs: 4,454.

Mall Catalogs: 527.

Moderate Delta Declines:

Tops & Tunics (WWW):

VbyVi delta: -16%.

Total Catalogs: 216,000.

Gold Catalogs: 2,323.

Mall Catalogs: 9,846.

Jeans (WWW):

VbyVi delta: -15.6%.

Total Catalogs: 77,334.

Gold Catalogs: 182.

Mall Catalogs: 2,400.

Kurtis & Kurtas (Women Kurtis, Kurta Sets & Suits):

VbyVi delta: -15.2%.

Total Catalogs: 672,261.

Gold Catalogs: 2,473.

Mall Catalogs: 5,484.

Low Delta Declines:

Jewellery Set (Women Jewellery):

VbyVi delta: -13.0%.

Total Catalogs: 449,921.

Gold Catalogs: 8,697.

Mall Catalogs: 3,612.

Frocks & Dresses (Kids):

VbyVi delta: -13.2%.

Total Catalogs: 97,733.

Gold Catalogs: 2,951.

Mall Catalogs: 1,198.

General Insights:

High Visibility Categories:

Sarees under Women Sarees have the highest platform visibility coverage (9.5%), with a significant number of total catalogs (692,305).

Kurtis & Kurtas under Women Kurtis have a platform visibility coverage of 6.5%.

Substantial Negative Delta:

Categories such as Trousers & Pants and Rakhi are experiencing substantial declines (over -16%), suggesting areas where improvements or more visibility may be needed.

Catalog Representation:

Many subcategories have a considerable number of Gold Catalogs, like Jewellery Set (8,697) and Kurtis & Kurtas (2,473), showing that there are still high-quality items represented even with the performance declines.

Mall Catalogs are also well-represented for many categories, such as Tops & Tunics and Tshirts under

WWW⁠

Conclusion:

The table highlights performance changes across several product categories and their subcategories. Most categories show negative delta, indicating a decline in performance. Categories such as Trousers & Pants and Rakhi have particularly high declines. Visibility and catalog representation across the platform also vary significantly, with some categories having higher platform visibility and a greater number of curated ("Gold") catalogs.

This analysis can be used to prioritize corrective actions, boost visibility, or refine the offering in certain underperforming subcategories.

Want to print your doc?
This is not the way.

Try clicking the ··· in the right corner or using a keyboard shortcut (

CtrlP

) instead.