Skip to content
Product interviews (shared)
Product sense
MECE
Scoring product sense
Product Design
Gilads Product Sense framework
Product Execution
Share
Explore

Product Execution

Collected by
Ishay Rosenthal
⁠

Frameworks

Metrics questions

Ben Erez
Product rational (<5 min)
Measuring Impact (15 min)
5 metrics in a dashboard - a basic pulse check of the
Specific + time frame e.g. orders per day
Counter metrics / guardrail
Setting goals (10 min)
Evaluating tradeoffs (10 min)
James
Set Goal (company / user goal)
User actions (user journey)
Acquisition & Activation:
Retention & Engagement
Monetisation & Revenue:
prioritise actions based on goals

Metrics (pick 3)
turn each action to quant metric
Individual or Aggregate - Can you group many actions for an overview?
Magnitude vs. Ratio
Ratio (percentages, rates, or per-unit measures)
Quantity/Absolute Numbers (raw counts or amounts)
Does it matter more for you to measure the overall magnitudes?
Intrinsic vs. Heuristic (direct vs. proxy)
Can you derive more knowledge from the measure? Or do you have to rely on heuristics?
Leading vs Lagging
Leading indicators help predict future performance
Session duration, Sessions/user, Activation
Facebook - seven friends in the first ten days
Lagging indicators give insight into past outcomes
ARR, ARPU
Trade-offs and limitations of these metrics
Diego
Clarify and reduce scope
In this case of measuring success of {company_X}, I believe there are many parts of the product we can focus on, for example we can talk about measure the success of {product_Y}. Are you ok with that assumption?
2 out of 3: mission, product summary, user journey
Set a goal
Explain why
⁠
image.png
⁠
⁠
Metrics
North star (tracking the goal)
only 1
tied to the goal
simple
Support metrics
evidence that we are moving the north star
Guardrail metrics
Make sure we are not hurting the biz
churn, uninstall
Risks & Trade-offs
Look back and check if we’re tracking the right things
Dianna Yau
Product, users, value
Describe the product (e.g. marketplace)
Mission / differentiation
Users, when are they ‘successful’?
North star metric
The intersection of value between users and company
Breakdown the north star
Levers to impact the metric
e.g. views, orders, cancellations,
Tradeoffs
Counter metrics
Quality
Eden Rozin (Meta)
Mission
Goals
Business (how they make money)
Marketplace
global/local
Audience (light segmentation)
Goals
Metrics
User flows

Metrics
Adoption, Activation, Engagement, Monetisation
Thinking through extremes
Healthy ecosystem
KYC
North star
Closest proxy to the goal
Try not to use ratios or avg.
Hard to move
Numerator / Denominator - can be gamed
Disadvantages
What is the north star missing?
Will you change the north star?
What levers effect the north star?
Trade-offs

RCA

Dianna Yau 1 (tradeoff + debug)
Product, Users, Value
Describe the product (e.g. marketplace)
Mission / differentiation
Users, when are they ‘successful’?
Hypotheses
Metrics
Ship, no ship, retest
Dianna Yau 2 (debug)
Understand the product
Breakdown the metric
Ask for data
Hypotheses
validate Hypotheses
Diego
MECE
System
instrumentation
metric formula
External
Competitors
Channels (e.g. facebook is down)
Trends
Demographics
Seasonality
Out of control
Regulation and politics
Natural disasters
Internal
product change
ux
new product, cannibalisation
bugs
device/platform

James
RCA
individual or team
org
environmental
3x Solutions, prioritise impact/effort/confidence
Action plan
process
timeline
teams to involve
experimentation
Risks and trade offs
revenue, cost
acquisition, activation, engagement, retention
Regulatory, brand
Technical
suggest mitigations
Evaluate success (of the solution)
primary
support
guardrails
Eden Rozin
Mission
Goals
Business (how they make money)
Marketplace
global/local
Audience (light segmentation)
Goals
Is it a critical issue? (unlikely that it isn’t)
debug the metric - ask questions
external
seasonality
providers (AWS)
competitors
Internal (self inflicted)
platforms
geo
releases
experiments
What do you do?
live with it
unroll

Metrics definitions
⁠

Messy bit that needs cleanup

Context

Understand the product, business model, use cases - ask or state your assumptions.

Understand the issue - clarify the metric.

Timeline - when, slope, trend
Extent of the problem - geo, platform, demographic, are other metrics dropping?
Where in the user journey (e.g. users starting session from notification, new, power, resurrected)
- Technical
- Value
- Experience
- Awareness
- Attention
- acquisition
- onboarding
- active use
- resurrection

RCA Factors

Internal

Technical - Engineering - deployment, infra, bug reports, crashlytics
Technical - Instrumentation
Value & Experience - Product - release, UX change, test (start or end), other teams or products in ecosystem
Value - Pricing and incentives (start or end)
Awareness - Marketing, announcements
Ops
Value - Supply side

External

Provider (e.g. AWS, library, platform)
Attention - Seasonality (e.g. Christmas)
Event and news (e.g. power outage, weather, big game)
Bad actors
Competition
Macro
Regulatory & politics
Trends
Solutions


Want to print your doc?
This is not the way.
Try clicking the ··· in the right corner or using a keyboard shortcut (
CtrlP
) instead.