PM Handbook @jabondano

Frameworks

Automations

Digital Marketing

PM Lectures & Videos

APIs

Explore

PM role & responsibilities

Making Good Decisions as a Product Manager

⁠

Brandon Chu⁠

⁠

Jul 11, 2017⁠

· 11 min read

While product managers may not build the actual product, they do produce something very tangible for a team: decisions.

July 23 update: in retrospect and from feedback, this framework applies to any role, not just product management.

These decisions can be about anything: small ones like a line of copy in the docs, to big ones like what the MVP of a new feature should be.

The decisions PMs make are the ones that unblock their team so they can continue to build. They don’t need to make every decision, but they are responsible for ensuring a decision gets made — whether by them, their team, or their stakeholders.

Product managers are the hedge against indecision, and it’s uniquely our job because we tend to have

the most context⁠

in a company. Typically, the most important decisions that a PM makes are, in order:

Why a team exists (vision and the impact they aspire to create)

What the general approach is to accomplish the vision (strategy)

What to build now, and what not to build (project prioritization)

What the smallest thing that can be built is that achieves the impact (MVP)

Since we’ve

already⁠

discussed⁠

these decisions before, and observed a

really bad one⁠

, in this post we’re going to focus on something more general: What does it mean for a PM to be a good decision maker?

Being a Good Decision Maker

Intuitively, being right a lot matters. At the same time, and in contrast, there are often factors outside of our control that determine outcomes.

Maybe you’re building an iOS app and then Apple changes their terms of service so your app is in breach. Maybe a law passes that makes your on-demand services illegal in your biggest markets.

These situations all happen in a flash, are hard to predict, and can negate any good decision you otherwise made. In such an environment, how can one objectively analyze their decisions?

I propose that being a good decision maker means doing two things:

Making decisions using the right amount of information

Making decisions as quickly as possible

Making good decisions using the right amount of information

Deciding how important a decision is, is the most important decision you can make. For people that make decisions for a living, understanding when one is really important vs. not-that-important is the most critical skill.

Amazon’s 2016 shareholder letter⁠

, Jeff Bezos touches on this concept with what he calls type 1 and type 2 decisions.

Type 1 decisions are not reversible, and you have to be very careful making them.

Type 2 decisions are like walking through a door — if you don’t like the decision, you can always go back.

What Bezos drives at in the letter is that the impact of a decision guides how much effort you need to put into making it. You can be right 99% of the time, but if you’re wrong the 1% of times when it really matters, you’re not an effective decision maker. The takeaway is that when the stakes are high, you should work a lot harder at making the right decision.

Although I don’t see Type I and II decisions as binary, I do see them as representing the spectrum for a framework for determining “decision importance”:

⁠

Plot your decision against each spectrum to get a gist of the importance of the decision

This framework breaks down decision importance into three dimensions:

Resource investment from a decision

The overall impact of a positive outcome

The impacts of a negative outcome (shown more granularly to help you think it through, but the sub-dimensions could be different for you)

The idea here is to plot a decision along these spectrums in order to determine an overall importance. Let’s take a look at three examples.

First, changing a “Sign Up” text-link on the homepage to a button.

⁠

This is clearly a low importance decision. If you’re wrong, it’s very easy to undo. The impact of being wrong is limited only to the number of users affected during the period of change. The impact of being right is only incremental.

Note, this doesn’t mean it’s not worth doing, it means the decision to do it or not isn’t that important, and as we’ll see later that should affect the speed at which we make the decision.

Another example: changing your company’s logo

⁠

This is a medium to high importance decision. It might not take a lot of resources to execute, but the impact (both positive and negative) to the company’s brand and customer base is huge. Remember when Uber changed their logo and no one could find the app? Or when AirBnB did and

no one could stop seeing genitals⁠

One more: a “moonshot” project to open up a new market for the company

⁠

This particular decision is medium to low importance. It’s important in that it will consume material resources, but also not important in that failure has no lasting downside.

This is a common decision type for tech companies that have reached scale and are looking for new ways to grow. The high investment + low impact of failure + high impact of success creates a risk/reward ratio that is very attractive to companies that can afford to spend the resources, so much so that they often create dedicated teams to repeatably experiment with these types of projects. Think Facebook’s

Building 8⁠

, or Google’s

X⁠

This framework, or any variation of it, will create common ground for you to compare decisions and articulate the importance of them to everyone you work with. Tweak it based on your company’s context, and recognize that defining decision importance informs everything else about how you should approach making the decision. So do it first.

Once you decide how important a decision is, you should adjust how long you‘re willing to spend on it.

I have a proposition for you, which is that good decision makers are also the ones who make most of their decisions quickly. To get to that conclusion, we’re going to unbundle the following statements:

The less important a decision, the less information you should try to seek to make it.

Gathering information follows a Pareto principle, meaning you can get 80% of the information quite easily, but getting the final 20% requires a lot of effort.

Most decisions are not important

Let’s go through each of them.

1. The less important a decision, the less information you should try to seek to make it.

Products, especially those used by many people, are complex systems where predicting all the ramifications of a decision are practically impossible, even when you have 100% of the available information. Since we cannot perfectly predict outcomes, this effectively means that all decisions are bets we are making.

We choose to make these bets based our confidence in a positive outcome, and that confidence is based on the amount of information we have. The more information we have as a percentage of all available information about a decision, the higher our confidence level should be.

“Information” in this context can be anything relevant to the decision. Using our previous sign-up button example, we could research button patterns from other sites as information to support a positive outcome.

Here’s the key question, though: How much information should one be gathering before they make a decision?

Whether consciously or not, few people gather anywhere near 100% of available information before making a decision. What ends up happening is that at some point, every decision maker subconsciously thinks to themselves “I’m ~70% sure adding this green button will increase conversion.”, and then they simply make it.

It’s that line of thought that reveals two things about how decisions actually get made by an individual: first, they assess their confidence level (70%), and secondly, they define a confidence threshold required to make that decision.

That confidence threshold is based on the importance of the decision. You may be willing to add the sign-up button with 60% confidence that it will create a positive outcome, but you wouldn’t change the company’s logo with only 60% confidence. The latter is a more important decision, and therefore the bar for confidence should be higher.

To summarize:

your confidence level on predicting a decision’s outcome is a function of how much information you have about the decision

the more important a decision, the higher confidence you require and thus the more information you need

We can visualize these points in the model below:

⁠

This reveals something counterintuitive about decision making: your goal shouldn’t be to always make the right decision, it should be to invest the right amount of time in a making a decision relative to its importance.

The ramifications of subscribing to this are two-fold. The downside is that you will end up making more wrong decisions, but when you do, they’re likely not important. The upside is that you will end up making more decisions, which as a PM means you’re creating more output.

How much more? That’s what the next section is about.

2. Information gathering effort follows the Pareto principle

What’s the difference in effort required for gathering 35% of information versus 70%? It’s not 2x — most of the time you will already have a significant amount of the relevant information close by: data about past user behaviour, previous research studies, previous attempts by the company, etc.

The Pareto principle applies here in that you can usually get ~80% of the information you need with little investment, whereas getting 100% will take a lot of effort.

Say you‘re a PM at Facebook and you’re deciding whether or not to get into the peer to peer buying/selling space. As evidenced by Craiglist, Letgo, and others, you know the opportunity is big so you start exploring the need for such a feature on Facebook.

You assess the importance, and come up with “medium important” because of the resource investment and the potential legal risks of facilitating transactions and commerce on Facebook.

⁠

What information would you gather to make this decision?

Well, probably the first place you’d look is the existing user behaviour within Facebook Groups. Are communities of people already doing this without having specific features for it? You can probably analyze the names of groups and the content being posted to them in order to identify how many users are already buying/selling.

By looking at the posts made in the groups, you can also probably figure out what is being sold, the composition of pictures being used, and how a transaction is happening even though there’s no payment features. To address the legal risk, you can also work with your legal team to research the implications to the company. All this information is readily available.

What you probably won’t be able to figure out so easily is: why are people choosing to use Facebook for this instead of the established marketplaces? To find that out you’ll likely need to do more qualitative research directly with users, which is going to take a lot of time.

Do you need to? Is it worth the extra time?

Probably not. The readily available information provides ~80% of the context for figuring out if people want the feature and if the company can stomach the legal impact. Although knowing why they’re using Facebook for buying/selling will be important knowledge for designing the feature, it’s not really needed to decide whether or not to do it in the first place.

Generalizing this into a model, we can say that the time (effort) to gather information for decisions follows the Pareto principle:

⁠

Notice this curve is exactly the same as we showed in the previous section when we showed that the amount of information you need is based on the importance of the decision.

Merging these two curves together, we can actually model the appropriate speed of making a decision as a function of the importance.

⁠

The combination of these models reveal something important. The speed of your decision making not linearly correlated to how important you think a decision is, it’s exponentially correlated. Incorrectly assessing a low importance decision as high means you waste a significant amount of time gathering excess information.

3. Most decisions are not important

The model below shows that decision importance follows an inverse power law, where the vast majority of decisions are not important.

⁠

To save your time, I’m just going to assert that this model is directionally, intuitively, true.

We make decisions all day working at product companies — they are not all be of equal importance, and frankly most are trivial. Should we use the oxford comma, or not? Should we roll out to 5% of users, or 10%? Lunch meeting, or eat fast and meet up in 15?

Bringing it all home

Let’s review our three statements and the conclusion they form:

The less important a decision, the less information you should try to seek to make it.

Effort in gathering information follows a Pareto principle. Getting 100% of information is exponentially harder than getting 80%.

Most decisions are not important

Therefore, the vast majority of decisions should be made quickly. Overlaying the last two models we covered illustrates this clearly.

⁠

Good decision makers, are quick decision makers.

Be a great decision maker

Applying these principles and frameworks are incredibly tough in practice. No one likes to be wrong, and as a PM being wrong is very public and can feel career damaging.

Understanding that decisions are your output as a PM can help you suppress your fears and focus on impact. If you packed boxes for a living, you wouldn’t think packing 3/hr flawlessly is better than 100/hour with a 2% failure rate. So why think that way about your output?

If you manage PMs, recognize that PMs that are uber careful and always right usually aren’t the ones who are driving the most impact. Impact is driven by those who are driving lots of decisions quickly, and are right when it’s really important to be.

The sign up button decision should be made fast, almost real time.

The Facebook buy/sell marketplace decision needs some time to think through, but it’s not the end of the world if you’re wrong.

You should take your time with the logo decision.

Before you go, some things to consider:

Recommend or share this if you found it useful. It gives me 🔋 to write knowing people find value in it.

Subscribe to

The Black Box of Product Management⁠

if you want more reads like this, or if you consider submitting to it so we can share your story.

⁠

The Black Box of Product Management⁠

⁠

Shining light on the PM discipline

7K

Thanks to NM Coutinho.

⁠

Product Management⁠

⁠

Startup⁠

⁠

Decision Making⁠

⁠

Product Design⁠

⁠

Leadership⁠

⁠

7K claps

34 responses

⁠

WRITTEN BY

⁠
`Brandon Chu`⁠
⁠

GM, Platform @shopify. Still a PM at ❤️.

⁠

⁠
`The Black Box of Product Management`⁠
⁠

Shining light on the PM discipline

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.

Making Good Decisions as a Product Manager

Making Good Decisions as a Product Manager

Being a Good Decision Maker

Making good decisions using the right amount of information

Once you decide how important a decision is, you should adjust how long you‘re willing to spend on it.

Bringing it all home

Be a great decision maker

Before you go, some things to consider:

Shining light on the PM discipline

7K

Thanks to NM Coutinho.

7K claps

⁠Brandon Chu⁠⁠

GM, Platform @shopify. Still a PM at ❤️.

⁠The Black Box of Product Management⁠⁠

Shining light on the PM discipline

⁠
`Brandon Chu`⁠
⁠

⁠
`The Black Box of Product Management`⁠
⁠