Internal tool to analyze churn reasons

Untitled page embed

Explore

Churn analysis & personalized outreach

Internal tool to analyze churn reasons & automatically generate personalized outreach

Samyak Rai

Objective & Outcomes

Customer churn and alternatively retention, are important metrics for SaaS teams to work on and optimize. Generally product, success, support and sales teams are collectively responsible. This is a critical issue and needs proper frameworks around it to be managed efficiently.

Our objective is to build a tool that automates this framework to the larger extent, provides deeper insights into possible reasons and re-engages churned customers with personalized outreach.

The directly expected outcomes from such a tool would be:

Clear framework within teams for managing churning customers

Alternatively, if a system was already in place, then - Reduced manual effort and efficiency improvements

Informed communication and accurate overall visibility of churn reasons across the org

Better tracking & measurement of customer churn and retention

Churn & possible reasons for it

For a product like Fireflies.ai, customer churn would generally be defined by workspaces which choose to cancel an ongoing subscription. Since, Fireflies.ai also provides a free forever subscription plan - we can extend our definition to also include users who signed up without credit card info and then stopped using the platform.

So, how can we define churn at Fireflies.ai?

Whenever a workspace cancels their subscription

Whenever a free workspace softly stops using the app:

If the notetaker is not added to a call on the user’s calendar for more than 2 weeks

Now that we know what actually is churn, let’s look into what could be the broader categories of churn reasons:

Product related reasons

Feature gaps

Usability or UI/UX issues

Pricing / competitor related reasons

Lack of realized value for the price

Cheaper alternative

Customer support / success related reasons

Poor support experience

Lack of proactive support & success plan

External reasons

Budget cuts

Organizational / structural / workflow changes

Based on my current experience these are the high level churn reasons I have noticed.

Tool overview & workflow diagram

This proposed tool will be able to collect all the incoming feedback, process it & extract data points with the help of LLMs, prepare a dashboard to in-depth analyze feedback and data points, generate personalized outreach emails and deliver a daily churn report via Slack.

For better analysis and action, the tool should have connections via webhooks to the CRM managed by Fireflies.ai team - to bring in critical customer info like revenue, current subscription plan, etc.

The LLMs will also need context of the help desk of Fireflies.ai and comparison pages against competitors.

Workflow diagram

⁠

Detailed spec & requirements

In this section, let’s talk about the above mentioned workflow in more detail and in-depth specifications of the flow.

Feedback collection from churned customers

Type of customer

How to identify churn?

Action to take

Paid plan

Deletes workspace

Revokes credit card access

Prompt user to give feedback on why they are cancelling their subscription

For now, let’s consider that we only collect feedback in a single text based field

Free plan

No dashboard activity

Notetaker has not been added to any calendar meetings for atleast 2 weeks

Trigger a personalized email to the user, asking about their inactivity and suggesting actions to re-activate them as well.

In the same email, we could ask for them to reply back to the email or via a form link, to provide a reason for them discontinuing.

There are no rows in this table

⁠

This will ensure that we are able to reach out to all churning teams / users and get feedback. Once we have collected their feedback, we will move on to the second part of the process.

Centralizing feedback & CRM data

Once the feedback is collected from the user, our aim is to centralize the raw data along with some customer data enrichment in our churn analysis tool. For this:

The feedback along with the workspace_id will be sent to the churn analysis tool.

On feedback submission, through the workspace_id, we will use a custom built webhook for internal purposes to fetch data from Fireflies.ai‘s CRM (HubSpot / Salesforce generally provides relevant documentation for this). Data to fetch:

Organization name

ARR (Annual recurring revenue)

Number of seats purchased / members in the workspace

Current subscription plan (Free / Pro / Business / Enterprise)

Customer segment (Enterprise / SME / others)

This data set will ensure that the tool will be able to process the feedback and customer data to provide valuable insights.

Processing data via LLMs

Our next step will be to process these feedback via LLMs and take out valuable data points. These data points will be the building blocks for:

Personalized outreach emails

Daily churn report

Churn analysis dashboard

Data points to be extracted

What data points are needed?

Personalized outreach email

To create an outreach mail, we need to extract:

Category of churn reason

Sub-category of churn reason (this will help formulate response template)

Reason (this will be the actual sub-part talking about why they churned, can be a sentence)

Daily churn report

To generate a daily churn report, we need to extract:

Category of churn reason (helpful for high level classification)

Sub-category of churn reason (helpful for high level classification)

Reason (individual reasons will be clustered to identify higher level insights ion why users churn)

Competitor names mentioned (we can provide a fixed list of competitors to identify if any have been mentioned)

Mentioned features (this will help surface features where users have significant pain-points)

Churn analysis dashboard

To generate an analysis dashboard, we have enough data already to start analyzing. Going forward, we can extract more contextual data points based on requirements.

There are no rows in this table

⁠

We have figured out a list of data points that are supposed to be extracted from the user feedback. Now, we can work on a prompt that will help extract these and output the result in a JSON format.

Additional: Cluster ‘reasons’ to generate insights

For the data point ‘reasons’ we can ask LLMs to cluster all generated reasons till now, and output them as clusters which are similar based on categories, sub-categories as well as text based similarity. Every cluster can be given a name via the LLM as well to generate the insight’s name.

These insights will be able to provide high level reasons for why users are churning. They will be especially helpful to look back on and understand behaviour as to why a certain customer segment, customers of a subscription plan or any type of user chooses to churn from the platform.

Note: LLMs are not great at clustering. If this is something we want to be accurate at, we might need to train a model and specialize it to generate insights.

Churn analysis dashboard

The dashboard will include 3 modules mainly:

Feedback analysis

This module’s primary objective is to give a snapshot of the customer feedback. The module will provide insights into why customers churn, related feedback backing up every insight, top categories of churn reasons and sub-categories to zone in on the issues along with other useful details.

This is where internal team will come for in-depth analysis, maybe after viewing a Slack report or even without it.

Trends & focus areas

This section will contain emerging trends to focus on. For example:

“25% churning customers mentioned about Avoma”

“40% churning customers mentioned issues with feature Smart search”

“Pricing has been the primary concern of churn leading to a net loss in revenue of $100,000”

“Budget cuts has been the specific reason of churn leading to a net loss in revenue of $70,000”

“In the past 6 months, 20 Enterprise customers have churned”

“In the past 3 months, 50 customers have churned from the Pro plan”

This section essentially removes the need for manual discovery of problem areas on the dashboard.

Category wise distribution

Every user feedback will be mapped to a unique category of churn reason

A pie-chart distribution can be shown to user to give an understanding of the category of concern

Clicking on a category should list out all the related user feedback to it

Clicking on a category should also display a distribution of user feedback in each sub-category

Sub-category wise distribution

Every user feedback will be mapped to a unique sub-category of churn reason

A pie-chart distribution can be shown to user to give an understanding of the sub-category of concern

Clicking on a sub-category should list out all the related user feedback to it

Competitors mentioned

Count of times a competitor name was identified in user feedback (unique counts only)

The list of competitors can be fixed unless manually created on dashboard

Features to improve

Count of times a feature name was mentioned in user feedback (unique counts only)

The list of features can be fixed unless manually created on dashboard

Subscription plan comparison for churn

Here, we can show a comparative bar graph for each subscription plan.

Number of customers in that subscription plan vs. how many churned

Allow users to change which month or week they want to view this for

Customer segment comparison for churn

Here, we can show a comparative bar graph for each customer segment.

Number of customers in that customer segment vs. how many churned

Allow users to change which month or week they want to view this for

Email outreach

This module will allow the team to control the configurations of sending automated email, viewing sent emails and responses and other email analytics. The module will include:

Communications list

A list of all the ongoing or previously sent out communications by organization name & email ID

Clicking on each communication can open its email thread

In communication details, option to also reply back with an email.

Analytics

Total outreach communications sent

Open rate of emails

Click through rate (CTR) on the CTA

Line chart to view these analytics WoW and MoM

SMTP configuration

Domain name (example - @fireflies.ai)

Email (should be as per the verified domain)

Sender name

Note: To some extent, the SMTP configurations are not very useful, as a lot of this can be hardcoded and managed directly from backend. Having fronted configurations for this may be an overkill.

Note on how automated outreach emails will be generated

First, we will need to identify the category and sub-category of feedback submitted. Based on these, the automated response structures will differ.

Below, all the templates are listed. To generate each email we will leverage LLMs.

Template of outreach emails:

Subject: We Miss You at Fireflies.ai - Let’s Reconnect!

Introduction & Acknowledgement of reason: Mention & acknowledge the reason provided in user feedback.

Suggested action / Query resolution: This will be based on the identified sub-category of churn reason:

if, Feature gap: check if mentioned feature or its workaround exists by referencing help articles

if it does not exist: Let the user know that we will consider adding to our roadmap. Share link of roadmap.

if it exists: if the mentioned feature actually exists as per help articles, then the user may not have access to it

refer user’s existing subscription plan and cross check if the mentioned feature is actually on another subscription plan

offer to connect with sales team for a feature demo

if, Usability or UI/UX issues: reference help articles by mentioned feature and provide links to relevant articles. Also, provide Calendly links to support team, to seamlessly help clarify doubts.

if, Lack of realized value for the price: here, we will primarily connect with sales teams to convince them churned customers about value through premium features.

provide a brief of premium features in other plans and provide links to help articles

provide a Calendly link to connect with sales team

if, Cheaper alternative: here, we will primarily re-activate churned customers by offering discounts and highlighting where Fireflies is better.

links to comparison pages - these are blog posts written by Fireflies.ai team

offer 10% discount

provide a Calendly link to connect with sales team

if, Poor support experience: apologize and connect to a support agent, offer discounts

if, Lack of proactive support & success plan: apologize and connect to a success agent, offer discounts

if, Budget cuts: suggest free forever plan to retain customer

if, Organizational / structural / workflow changes: suggest free forever plan to retain customer

Note: To reference help articles in the best way, we will have to tag them to be referenced easily by LLMs. They need to be tagged by subscription plans and features.

Note: We will also be referencing comparison pages to show value of Fireflies over alternatives

Slack report

This module will include the configuration settings for the Daily churn report to be sent on Slack. These settings will include:

Where to send? - This will be a drop-down option containing a list of public & private channels in the Fireflies.ai workspace.

When to send? - Here, the internal team can decide on a time in the work day to receive the Slack report.

What to send? - Here, we can allow the internal team to customize the contents of the Slack report. The list of options will include the aggregated (at the day level) versions of the data points to choose from:

Total churned customers (default)

Revenue impacted (default)

Category wise distribution (default)

Sub-category wise distribution (default)

Customer segments impacted

Number of users churned

Subscription plan wise distribution

List of churned customers

Note: To some extent this module is not as useful, as a lot of this can be hardcoded and managed directly from backend. Having a frontend for this may be an overkill, unless it needs to be customized for every individual receiver.

Question: Daily churn reports might be too much, wouldn’t it be better to evaluate it weekly?

Sample outreach email

(cheaper alternative)

Subject: We Miss You at Fireflies.ai - Let’s Reconnect!

Hi [Customer Name],

I hope this message finds you well. We noticed that you decided to leave Fireflies.ai, and we understand that you found a cheaper alternative in Avoma. While we’re sorry to see you go, I wanted to take a moment to acknowledge your feedback and offer some additional information that might help you reconsider.

At Fireflies.ai, we continuously strive to deliver unparalleled value to our users, and we believe that the features and support we offer are designed to enhance your productivity in ways that other tools may not. To help you better evaluate your options, we’ve created a comparison guide that highlights the key advantages of using Fireflies.ai over other alternatives. You can check it out

here⁠

Additionally, as a token of appreciation for your previous partnership with us, I’d like to offer you an exclusive 10% discount should you decide to return. We truly value your experience and would love to help you get the most out of our platform.

If you’d like to discuss further or have any questions, I’m here to help. Feel free to

schedule a call with our sales team⁠

at your convenience. We’re more than happy to walk you through how Fireflies.ai can better meet your needs and provide you with the best value.

Thank you for your time, and we hope to welcome you back to Fireflies.ai soon!

Best regards, [Your Name] Sales team Fireflies.ai

Sample churn report

Daily churn report

Date: August 13th, 2024

Total customers churned: 10

Revenue impacted: $25,000

Category wise distribution:

Product related reasons: 30%

Pricing related reasons: 40%

Customer support / success related reasons: 5%

External reasons: 25%

Sub-category wise distribution:

Feature gaps: 15%

Usability or UI/UX issues: 15%

Cheaper alternative: 40%

Lack of proactive support & success plan: 5%

Budget cuts: 20%

Organizational / structural / workflow changes: 5%

⁠

View in dashboard

⁠

Objective & Outcomes

Churn & possible reasons for it

Tool overview & workflow diagram

Detailed spec & requirements

Feedback collection from churned customers

Centralizing feedback & CRM data

Processing data via LLMs

Additional: Cluster ‘reasons’ to generate insights

Churn analysis dashboard

Sample outreach email

Sample churn report

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.