Gallery
Share
Explore

In-Product refunds experience to retaining our worker partners.

Note: I can’t show graphs of data due to confidentiality reason but happy to describe at high level.

Agenda
Context on Refunds: What, Why, Who
Refunds MVP Spec
Challenges
Outcomes
⁠

Refunds Context

⁠
image.png
⁠
⁠
⁠
invoice K.png
⁠
⁠

Not all projects are created equal...

⁠
chat K.png
⁠
⁠
This is the clearest example (i.e. customer cancel the project) but there are other reasons (i.e. getting the same lead twice, project violates ToS, project details change significantly). Some of the pain worker were experiencing was due to the refunds policy rather than the mechanism.
This resulted in very low pro NPS/LTC
Problems:
Really frustrating user experience
Inefficient from an operations perspective and application of refund policy.

Pain Points

Worker
As a worker, there isn’t an easy way to request a refund when I receive a bad project.
As a worker, I don’t get refunded for all kinds of bad projects (i.e. customer doesn’t even view my response).
As a worker, I don’t feel that Kanggo has my back because it’s overly stingy with how it gives out refunds.
Customer Support agent
As a CS agent, if a worker is on the phone, I feel that I have to give them a refund even if it’s not in the policy because it’s directly tied with how I get evaluated.
Kanggo
Worker Likelihood to Continue (LTC) score is low and chargeback rate is high, signaling that there are product-market fit issues with our worker product.
There is no central team evaluating refunds, resulting in inefficient application of the refund policy.
⁠

Refunds MVP Scoping

Goals of Refund MVP
Metric
Outcome
1
Feature adoption & usage
Dominant refunds channel within 2 weeks of full rollout, processing 5+ refunds requests per week. 80% funnel completion rate.
2
Increase in Worker LTC and Retention
LTC increase when compared to comparable channels with similar grant rates (53% vs 47% for email).
Retention: didn’t examine until we changed refund policy significantly w/ price assurance, which resulted in ~5% increase.
3
Improved operational efficiency (support cost per refund case)
Never fully measured. but estimated savings ~IDR 50 millions per year (it’s small because the company stage is also still very early, and the market still premature).
4
Guardrail on revenue
Didn’t surpass revenue guardrail.
There are no rows in this table
⁠
What’s the best refund strategy to optimize for the above goals?
Option
Description
Improve worker retention, LTC, or chargeback?
Improve operational efficiency?
Revenue impact
1
A: Be stingy (status quo).
Give fewer refunds and only when there is a clear breach in our promise to the worker (e.g. ToS).
No - Worker currently perceive policy as unfair and inconsistent.
No - Increased operational cost given time/$$ spent scrutinizing.
Minimal given that this is the current world.
2
B: Make workers work for it
Given refund for places even where there isn’t a breach in our promise to workers, but require workers to submit extensive proof and/or work.
Maybe, although this sends a message to workers that we don’t trust them.
Maybe - we’ll have clearer guidelines but it will still require agent time to go through them.
Mixed - Depending on implementation, we could be giving out fewer refunds, but there is still operation cost.
3
C: Use refunds as “Product experience crutch”
Use refunds as a mechanism to close product gaps. This means we would lean towards granting refunds rather than litigating refund requests and will monitor for feature abuse.
Yes - We’ll give refunds for worker frustration moments, and use as a carrot to educate workers
Yes - We’ll default to giving out refunds rather than scrutinizing, which will reduce ops time.
Some impact as we’ll be giving out more refunds, but we’ll be saving $$ on operational cost.
There are no rows in this table
⁠

Refunds MVP

Overall theme: Build the infrastructure for future changes to refunds.
Workstream 1: Build in-product refund flow in-product across mobile app.
Workstream 2: Semi-automate refunds review process on the operations side.
Post-MVP:
Iterations to the refunds flow
V2 features to refunds flow (i.e. refunds view status page)
Make changes to the refunds policy (i.e. unresponsive customer).
Refunds MVP Teams + Tasks
Workstream
High-Level Tasks
Teams
1
Build in-product refund flow in-product across mobile app.
Designs completed + research
Build out new backend to track refunds + status, connect with Qontak for CS tracking
Front-end implementation of refunds flow
GTM rollout
Engineer
UI/UX
Legal
(5 Peoples)
2
Semi-automate refunds review process on the operations side.
Create new team solely focused on refunds processing, align with Trust & Safety.
Get alignment on V0 refund policy with leadership
Train agents
GTM rollout
Customer Supportt
Legal
Field Officer
(5 Peoples)
There are no rows in this table
⁠
⁠

Challenges

Challenge 1: MVP designs

Challenge: to launch this project by end of Q4, we needed designs ASAP so engineer can start scoping.
However, the designer to this project was new to this area, and wanted to conduct research to make sure we were on the right track.
Solution:
Designer taken off other projects to solely focus on ramping up on in-product refunds.
Did user research of MVP designs with field officer instead of workers to get feedback faster.

Challenge 2: Trust & Safety w/ Operations Rollout

Challenge:
Trust & Safety (T&S) needs to review all free text responses coming into TT for any T&S concerns, but we didn’t consider this requirement in MVP scoping.
Solution:
Setup meeting with T&S and refunds operations folks to align on how we’d align the process and SLA for T&S reviewal of refund requests. Didn’t require any significant headcount adjustments.
Aligned on scope for how to deal with refund processing if there is a T&S concern (came up with new email copy).

Challenge 3: How to define launch readiness + rollout

Challenge
There wasn’t a clear idea of what is/isn’t a launchblocking bug because no frontend TL.
Customer Support (CS) had concerns about an increase in refund volume after launch and wanted a way to understand how to allocate appropriate staffing to maintain refunds SLA.
Solution
I put together a process on defining launchblocking bug w/ design + engineer (P0 = blocking critical user journey and/or easy polish, P1 = nice-to-haves medium-sized bugs, P2 = complicated non-launch blocking bugs).
We put together a staged rollout plan (i.e. 5%, 25%, 50%, and 100% rollout) so CS had enough time to get data on refund volume and adjust staffing accordingly.
⁠

Outcome

Note: I left the team in end of Q1.
View of Goals of Refund MVP
Metric
Outcome
1
Feature adoption & usage
Dominant refunds channel within 2 weeks of full rollout, processing 5+ refunds requests per week. 80% funnel completion rate.
2
Increase in Worker LTC and Retention
LTC increase when compared to comparable channels with similar grant rates (53% vs 47% for email).
Retention: didn’t examine until we changed refund policy significantly w/ price assurance, which resulted in ~5% increase.
3
Improved operational efficiency (support cost per refund case)
Never fully measured. but estimated savings ~IDR 50 millions per year (it’s small because the company stage is also still very early, and the market still premature).
4
Guardrail on revenue
Didn’t surpass revenue guardrail.
There are no rows in this table
⁠

Personal Learnings

Grew by:
First large cross-functional project across two different engineering teams.
First time working intensively with our operations team.
First time at Kanggo building relationships between different types of teams.
Could be betters for next time:
Spend more time with engineering scoping MVP.
Conduct product review earlier to capture feedback from broader product development team.
More analytics resources to fully measure outcomes of the project
⁠

Appendix

1st week of Nov:
Week 3 Nov:
Week 1 Dec:
Week 2 December:
Week 1 Jan:
Week 1 March:
1
kickoff in-product refunds project
Designs finalized
Backend spec finalized
new refunds team finalized, training scheduled.
5% rollout for refunds
100% rollout for refunds
There are no rows in this table
⁠

What are the entry points for refunds?

In-product entry points
On-track to be shipped
In-product scam reporting flow (IPSR)
Report button within conversation (Decline flow)
Payments history
Other ideas
Automatically grant refunds (with education on how to request refund in the future)
Within overflow menu as a separate CTA
Phone (i.e. agents on phone tell workers who have requested refunds to do it in product)
Help center
In-App mail campaign to educate workers

Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.