Share
Explore

Split Software - New User Experience for Feature Flags

Webflow Presentation - January 2024

Product Context

Split is a feature flag and experimentation platform.
Feature Flags: enable software developers to test features in their production environment without showing the feature to users and then slowly release features to users with the ability to shut them off instantly in emergencies.
Experimentation: enables product managers to run targeted experiments such as A/B tests.
Split’s feature flagging product can be used stand-alone but the experimentation product requires the implementation of feature flags before it can be used.

Business Strategy

This project stems from a joint effort I lead with the marketing team working towards two annual goals.
Change the perception that “Split is unfriendly for software developers”.
Improve the free trail to paid conversion rate for pioneering teams at enterprise organizations trialing Split.
We hypothesized that the root cause behind these issues was due to the organization’s focus on leading with our experimentation product. Split’s experimentation platform is a key competitive differentiator but we found that product managers needed software developers to implement feature flags to get it working and software developers preferred our competitor’s product for feature flagging.
The team decided to re-focus efforts on feature flagging for software developers to meet these goals.

Persona Focus

Primary: Software developers trialing Split
Software developers are the main user during a trial experience and are the primary source of input for a VP’s buying decision.
Main user during trial experience
Prefer familiar brands
Like to pick things up quickly and are critical of unintuitive workflows
Prioritize feature flag functionality over experimentation capabilities
Secondary: Enterprise users already using Split
Any change we made to the product to improve the trial experience would be part of the core workflows for current users sensitive to change.
Not involved in trial experience but affected by changes
Concerned about production environment
Prioritize improvements but resist change

Product Vision

Software developers will spend their social capital to recommend Split publicly.

Problem Scope

After talking to customers we lost in sales deals I found that the feature flagging product had the necessary functionality but software developers preferred our competitor’s familiar UI. Digging into our free trial analytics, I found that users were dropping off during the feature flag configuration process. Circling back with the lost customers, I found that configuring a feature flag for the first time was unintuitive and we would actually need to be far more usable than our competitor to convince them to change tooling. So, improving usability of our feature flagging product became the product priority.
Specifically:
New users want to try a simple configuration first and were getting bogged down by all of the complex configurations available in the UI.
Users were getting confused by terminology, causing them to be unsure if they set something up correctly or not. This was a problem in the UI, the API, and the documentation.
Users were not confident that they were making the right changes in the right environment. They needed better visual indicators in the UI.

Success Metrics

The following measures were combined to create a usability score that we baselined before changes were shipped and measured at each milestone.
Quantitative:
Success rate of feature flag configuration
Time to configure feature flag
Qualitative:
In-app usability surveys

Solution

Scope
0
Milestone
Description*
Problem Solved
Impact
Tradeoffs/Limitations
1
1
Suppress explanations and improve terminology
Removed distracting and confusing text. Fixed confusing terminology.
✅ Reduced time to completion on page
UI and API terminology changes done separately.
2
2
Update UI/UX for environment selection
Clarified which environment the user was making changes to.
✅ First positive, public feedback came in ✅ Positive direct customer feedback ⚠️No change in success metrics
Parallel UI rebuild with scalable components.
3
3
Separate treatment & targeting setup
Further reduced cognitive load on the page.
⛔️Negative change in success metrics during A/B test. This feature was removed from production.
4
4
Revamp UI/UX for feature flag configurations
Simplified feature flag configuration and built the user’s confidence that they did things correctly.
✅ Public, positive feedback flowing in ✅Positive change in success metrics
Coincided with rebrand efforts.
5
5
API terminology changes
Fixed confusing terminology.
✅ UI & API consistency
6
Bonus
Hackday project: Visualize setup to clarify flag configuration
Visually showed what would happen with the configuration set up. Promoted sharing.
Won Hackday, slated to ship in 2024
7
There are no rows in this table
*The colored text corresponds with the areas circled in the screenshots below.

UI - Before

image.png

UI - After

image.png

Customer and Business Outcomes

6 G2 reviews in a month.
17% increase in feature flag usability.
No lost deals due to usability.
New pricing muddied project’s impact on free to paid conversion.

Key Learnings

Software developers are visual learners. This became one of our design principles.
Enterprise customers are uncomfortable with UI changes but we over-communicated timelines and gave “preview environments” which alleviated concerns.
Proactive updates to Split’s exec team built conviction in plans and prevented strong reactions to big changes.
Fighting brand familiarity is HARD. Although usability became a competitive differentiator after this, it was critical for our team to focus efforts after on a net new product area.


Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.