Explore

Split Software - New User Experience for Feature Flags

Webflow Presentation - January 2024

Product Context

Split is a feature flag and experimentation platform.

Feature Flags: enable software developers to test features in their production environment without showing the feature to users and then slowly release features to users with the ability to shut them off instantly in emergencies.

Experimentation: enables product managers to run targeted experiments such as A/B tests.

Split’s feature flagging product can be used stand-alone but the experimentation product requires the implementation of feature flags before it can be used.

Business Strategy

This project stems from a joint effort I lead with the marketing team working towards two annual goals.

Change the perception that “Split is unfriendly for software developers”.

Improve the free trail to paid conversion rate for pioneering teams at enterprise organizations trialing Split.

We hypothesized that the root cause behind these issues was due to the organization’s focus on leading with our experimentation product. Split’s experimentation platform is a key competitive differentiator but we found that product managers needed software developers to implement feature flags to get it working and software developers preferred our competitor’s product for feature flagging.

The team decided to re-focus efforts on feature flagging for software developers to meet these goals.

Persona Focus

Primary: Software developers trialing Split

Software developers are the main user during a trial experience and are the primary source of input for a VP’s buying decision.

Main user during trial experience

Prefer familiar brands

Like to pick things up quickly and are critical of unintuitive workflows

Prioritize feature flag functionality over experimentation capabilities

Secondary: Enterprise users already using Split

Any change we made to the product to improve the trial experience would be part of the core workflows for current users sensitive to change.

Not involved in trial experience but affected by changes

Concerned about production environment

Prioritize improvements but resist change

Product Vision

Software developers will spend their social capital to recommend Split publicly.

Problem Scope

After talking to customers we lost in sales deals I found that the feature flagging product had the necessary functionality but software developers preferred our competitor’s familiar UI. Digging into our free trial analytics, I found that users were dropping off during the feature flag configuration process. Circling back with the lost customers, I found that configuring a feature flag for the first time was unintuitive and we would actually need to be far more usable than our competitor to convince them to change tooling. So, improving usability of our feature flagging product became the product priority.

Specifically:

New users want to try a simple configuration first and were getting bogged down by all of the complex configurations available in the UI.

Users were getting confused by terminology, causing them to be unsure if they set something up correctly or not. This was a problem in the UI, the API, and the documentation.

Users were not confident that they were making the right changes in the right environment. They needed better visual indicators in the UI.

Success Metrics

The following measures were combined to create a usability score that we baselined before changes were shipped and measured at each milestone.

Quantitative:

Success rate of feature flag configuration

Time to configure feature flag

Qualitative:

In-app usability surveys

Solution

Scope

Scope

Milestone

Description*

Problem Solved

Impact

Tradeoffs/Limitations

Suppress explanations and improve terminology

Removed distracting and confusing text. Fixed confusing terminology.

✅ Reduced time to completion on page

UI and API terminology changes done separately.

Update UI/UX for environment selection

Clarified which environment the user was making changes to.

✅ First positive, public feedback came in ✅ Positive direct customer feedback ⚠️No change in success metrics

Parallel UI rebuild with scalable components.

Separate treatment & targeting setup

Further reduced cognitive load on the page.

⛔️Negative change in success metrics during A/B test. This feature was removed from production.

Revamp UI/UX for feature flag configurations

Simplified feature flag configuration and built the user’s confidence that they did things correctly.

✅ Public, positive feedback flowing in ✅Positive change in success metrics

Coincided with rebrand efforts.

API terminology changes

Fixed confusing terminology.

✅ UI & API consistency

Bonus

Hackday project: Visualize setup to clarify flag configuration

Visually showed what would happen with the configuration set up. Promoted sharing.

Won Hackday, slated to ship in 2024

There are no rows in this table

⁠

*The colored text corresponds with the areas circled in the screenshots below.

UI - Before

⁠

UI - After

⁠

Customer and Business Outcomes

6 G2 reviews in a month.

17% increase in feature flag usability.

No lost deals due to usability.

New pricing muddied project’s impact on free to paid conversion.

Key Learnings

Software developers are visual learners. This became one of our design principles.

Enterprise customers are uncomfortable with UI changes but we over-communicated timelines and gave “preview environments” which alleviated concerns.

Proactive updates to Split’s exec team built conviction in plans and prevented strong reactions to big changes.

Fighting brand familiarity is HARD. Although usability became a competitive differentiator after this, it was critical for our team to focus efforts after on a net new product area.

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.