Product / Engineering Doc

Explore

Alex's Happy Doc

Tim Treagus

Product Recap (Shahil) - March 2022

Epics and issues

Elmar Notes

Product Pieces

Business Model & Finances

A market research platform that allows you to survey a targeted audience segment that is typically hard to reach digitally by utilising our research panel. Surveys run entirely through WhatsApp (app with highest penetration rate in SA), allowing access to a much larger segment whilst also being faster, more cost-effective and highly personalised.

Roles & responsibilities

Team member

Role

Responsibilities

Louis Buys

Founder/CEO

Fundraising & strategy

Tim Treagus

Founder/Product

Growth, product decisions, customer feedback, strategy

Bruce Elliot

Backend engineer

Database design, product architecture

Alex Mathews

Product Lead

Strategy, data architecture

Jono Hart

Part-time growth

Growth

Paul

Database designer

Database architecture and assessment

There are no rows in this table

⁠

🌅 Early Days

AI-BO is owned by Old Mutual and was founded as as a solution to future financial security (AKA a pension), aimed at the mass market. The core mandate of AI-BO from OM was to provide income earning opportunities to people who struggle to earn any income. An extension of this was to organically create a savings behaviour by incentivising people to save their income by giving them access to more work if they do so.

AI-BO started out as a data labelling and acquisition business for the purposes of building vernacular language models. Our mechanism for collecting/labelling data was WhatsApp as it allowed us wide and seamless access to people. We thus developed the capability internally to build WhatsApp bots for the purpose of handling data.

In April 2021, after months of trying to land customers from varying industries with no success. We decided to pivot to utilising AI-BO as a market research tool.

Conducting market research in low-mid income markets can be expensive and inaccessible, especially with resistance to data usage. With AI-BO, you can access faster, more cost-effective and scalable consumer insights.

Research & Competitors⁠

The online market research industry is a whopping $23 billion dollars whilst only 5% of that coming from Africa. Currently, only 13% of research is done online in Africa vs 40% globally.

As Africa catches up to the rest of the world there is a massive growth opportunity to capitalise on the shift to digital research methods for the informal market

Particularly because our product is borderless and allows us to work across Africa whilst also appealing to international investors looking to enter the African market

⁠

⁠

Using WhatsApp

WhatsApp is the most downloaded and used app in South Africa. It uses very little data to send and receive messages. To access insights from people, we were not forcing them to change their behavior in any way: Don’t download a new app, don’t learn a new system, don’t go anywhere online you’ve never been. Thus, we hypothesised that it would give us access to a segment of people that were previously inaccessible through digital channels.

We will be able to access people through WhatsApp that are otherwise digitally inaccessible because of phone storage and data usage constraints

We will have higher response rates, engagement and completion of surveys because of how comfortably people use WhatsApp (most popular, downloaded and use app in SA)

We will able to be cheaper than existing low LSM research methods

🕵️ Validating Assumptions: Market Research Businesses

We hypothesised that the best customer segment to go after for market validation and to get insights from was the market research industry. In South Africa alone there are over 200 market research businesses. Most of these businesses have been around for a long time and their employees are hard to reach digitally (LinkedIn or email)

How work typically comes in:

Corporate → Marketing/advertising → Market research

Normally have to do in-person field research to mass-market

Some see us as a competitor

Wouldn't use 3rd party like AI-BO to get research (segment 1)

Want to use their own research panel

How do we verify accuracy?

What we found out is that the decision to use a different market research tool for acquiring their insights is a big decision that has direct consequences to their reputation as they will be presenting the insights that we give back to their client.

Competitor pricing

Competitor pricing

Type

Field Agent

African Response

Qualquarter

Survey54

Survey Monkey

Users

153,000

8,000

500,000

Setup

R3,200.00

Reporting

R3,200.00

Per survey of 10-15 questions

R250.00

R175.00

R300.00

R105.00

R86.00

5 Questions

R125.00

R42.00

Subscription 1

R540.00

There are no rows in this table

⁠

🔨 MVP 1 (May → Aug)

WhatsApp Bot-builder platform

Cognigy

To build a WhatsApp bot for this MVP using code with developers was unnecessary from a budgetary point of view and would result in significant delays in building new survey flows. Thus the decision was made to utilise a low/no-code bot-building platform that could be fully managed by a product manager. Cognigy was selected as the platform of choice. The core reason for choosing them was the free 3 month trial that they offered us as well as their platform being really good.

Twilio

The WhatsApp API and cell number was issued by Twilio. Messages from Cognigy are routed through Twilio’s platform.

Pricing structure: $0.005 per message received or sent.

Landbot

AI-BO moved to Landbot in May for the following reasons

Our Twilio bill was $870 for the month of May. Once the Cognigy cost gets added to this and we start growing, it is likely that monthly subscription fees will be north of $4,000

Landbot’s pricing structure is per active user in a month period: if 1 person does 3 surveys, you pay the same. Twilio’s pricing structure is counterintuitive because it charges per message.

Compared to Landbot, the NLU flow builder design takes longer to setup new flows/surveys

No menu functionality in Cognigy (because of the NLU design) making it really difficult to direct users to new surveys

Cant send template messages through Cognigy (easily)

Still haven’t got the balance store proc function working (Alex has access so it must clearly be very hard)

How it works

User signs up on WhatsApp on a signup flow

User arrives at a menu with: Surveys, Income Balance, Refer a friend, Help as options

When the user does a survey they are directed to a new flow to complete it and then directed back the menu on the signup flow

Variables are setup for the survey responses that map to the database through a webhook

Database Design

MS SQL database setup. What to say?

@Alexandra Matthews⁠

Tables

Users

Database Webhooks from Landbot

Setup the following store procs to handle different scenarios

Create_user → Mobile, Landbot ID, Name, Referrer mobile

Survey_response → Save survey response data

Transaction → Add income to a participant’s income balance

Income → fetch a participant’s income balance for the period

Retool

Retool is used to streamline the process of creating new variables for response data from surveys to map into the DB.

Create an organisation

Create a new survey that sits inside the organisation

Create survey questions and add variable names that correspond with the survey response

As a result, we can see which survey and organisation every variable comes from. The difficulty with this is that if we do 2 surveys asking the same question our responses will not relate with one another in the database

Webflow Landing Page

AI-BO decided to completely rebuild the website by purchasing a template and adapting it to what we needed. This decision gave us flexibility and autonomy to make changes in a streamlined and agile way. The ultimate goal was to be able to direct customers to a place that encompasses what we do, how it works, how much it would cost etc. to remove friction of businesses not wanting to use us because of having a lack of clarity on what we do + lack of credibility.

Analytics Tools

Google Tag Manager → Manage all web integrations

Heap → Event-based analytics tracker to see user journeys and button clicks etc

Google Analytics → Free analytics tool to track web traffic and sources of traffic

Hotjar → Heatmaps and user journey videos of how users interacted with site

Customer Order Forms: Typeform & Google Forms

I created a price estimation and question submission typeform that effectively acts as an order form for customers. The form is dynamic in that it calculates the order price based on your different order requirements and quantities.

Google Sheets

The responses for a survey and demographic attributes of the respondents are sent to a Google Sheet (setup in Landbot). This allows the client to see the results live as they come in and has been used for internal pilots. It also allows a simple connection to Data Studio.

Results/Analytics

Tableau workbooks are generated once the survey has been completed. The downside with Tableau is that

It depends on a BI resource to build the dashboard

A new Tableau license needs to be issued each time a report is shared with an external business - roughly R3,000

Data Studio was trialed as an alternative to Tableau for the following reasons

It’s free to use and can be shared with anyone

It can be embedded into Webflow

A BI resource is not relied upon to build dashboards for clients

Recruiting Participants

Facebook

Our WhatsApp account is connected to our Facebook business account which enables us to do ads where the CTA is a WhatsApp message to our account

Referrals

Participants are incentivised to refer their friends to sign up

🔬 Experiments

⁠

Interviews⁠

⁠

LinkedIn Post⁠

⁠

Calendly Sales Meetings⁠

⁠

Internal Pilots⁠

🤯 Risks

🤯 Risks

⁠

💄 Product Overview

AI-BO’s product success is driven by our ability to deliver quality insights to customers. This is based on the following factors

Responsiveness of participants

Efficient access to our customer’s target audience - having the right people

Accuracy and reliability of our results

Reporting, visualisation and guidance to actionable next steps to our customers

Customer experience of using AI-BO

Backend Rationale

Responsiveness of Participants

Efficient access to our customer’s target audience (Size of user base panel)

Size of panel

Ability to recruit the right people

Rich data on our participants that goes beyond high-level demographics

Accuracy of results

Frontend Rationale

Customer experience of using AI-BO

Interface to make an order for new insights. How much can they self-service?

Turnaround time of AI-BO

The interface that clients will be dealing with contributes to the credibility of AI-BO.

As a concept that is unfamiliar to most people above 40, we need to ensure that we demonstrate how AI-BO works in a very clear and obvious way. The mechanism to achieve this is by taking the prospective customer through an example of past work to show them exactly how to setup a campaign and the results of the demo campaign.

Client ordering workflow

Audience identifier/database structure

Tagging system for a variable to add to off the shelf attributes (enabling us to target people based on that variable that was acquired in the survey)

Map same/related question IDs into a single tag or category

When setting up a new survey, the creator can view all the related questions

Question success rate system that enables clients and survey creators to view different clients

Profiling/segmenting people (maybe using tag system)

Participants

The research/survey participants are the most crucial resource to our product. We want our participants to be happy for the following reasons

They refer their friends and we grow exponentially grow our panel without much effort

They produce accurate answers and put effort into their responses so that our clients get better insights. The opposing outcome is that they feel mistreated and so try to game AI-BO by producing rubbish results.

They are highly responsive giving us the ability to offer fast turnaround times to our clients.

Currently, our participant panel is made up of low-income people. Everything below describes the product for this segment of people

😁 The factors that contribute to our participants upholding these things are as follows

[Push] The frequency and quantity of surveys available for them to complete

[Push] Being paid on time

[Push] Being paid the right amount

[Pull] Being incentivised to produce better responses, respond faster and refer more people by getting paid more through a rating system

Paid more per survey completed

Priority on completing surveys (early batch access)

Priority notice through a template message an hour before others

[Pull] Being told when new surveys are out to remove uncertainty of when surveys are and allow participants to have a consistent expectation for new surveys

🔍 Recruiting

Facebook

AI-BO recruits using FB

Standard Referrals

Participants are incentivise to refer their friends and network

Agents

AI-BO pays certain people to post in related Facebook groups

Moya

⚒ 🔩 Product Build

Different Users/Roles

⁠

Client Ordering System⁠

: Web Dashboard

Primary User

Can Register Company/Main account

Can view demo campaign of setting up and results

Can create Research Campaigns

Can setup research objectives and goals

Can choose demographic attributes of audience to send survey to

Can edit audience selection after choosing

Can create Survey within Campaigns (See below)

Can select survey from template and edit

Can select from audience that they previously surveyed

Can create screener question with reject/accept criteria for the difference MCQ answers

Can add users / roles and assign to campaigns

Can comment on questions and tag people in survey questions

Can get a cost estimate for the survey campaign in the dashboard

Can submit the campaign for approval

Can be sent proposed changes to survey or campaign

Can resubmit campaign with changes

Can get an invoice for the amount they owe according to their order

Results

Can view results in real time

Can mark people they want to interview by

Secondary Users (for MVPv.2 can leave out now?)

Invited by Primary User

Can create Campaigns?

Can create surveys?

Guest User? (for MVPv.2 can leave out now?)

Can request access to view results

⁠

Survey Participant on WhatsApp⁠

Sign-up

Check runs that participant is above 18

Participant is asked to confirm privacy policy

Participant is asked whether they agree to receiving notifications about payments and new surveys. This enables us to send outbound messages to participants who have not chatted to use within the last 24 hours.

Participant gives demographic information

Quality checks survey that adds a point to their

Bad Actor Score⁠

if they submit the incorrect answer

Can not see other surveys until Quality Checks and Signup Survey is finished

Can see available surveys (only available to their demographic)

Can complete surveys and earn airtime

Can see income balance for the current period (after last payout). Payout happens every Thursday

Can get the referral link and message to send to their friends

Can view FAQs about payment, survey frequency

⁠

Database/Participant Management⁠

: AI-BO employee

Can clean data to remove non-sensical data (after surveys and throughout)

Can group open text responses into categories

Can communicate with product team to advise on how to enrich data or better construct questions

Can add tags to variables to associate them with various categories. Variables can have many associated tags (e.g. Finance → lending)

Should we have tag hierarchy?

Variables can be mapped into one another so that the answers from the same question from different surveys can be matched.

Survey answer checks

Bad Actor Score⁠

system

Once a survey has been completed, the following checks occur which impact the participant’s

Bad Actor Score⁠

(see how it works in the page)

Scramble checking survey responses

Manual checks of answers

Database automatically soft deletes user if there is a a duplicate of the same profile

@Paul Brittan⁠

Paul Brittan

11:28 AM⁠

⁠

The best option is to take the data from your App DB (marketing) and run it through ELT packages to clean the data into a new DB. This new DB with clean data is what we will then use to report off of.

We then have control of what info we are displaying to the reporting layer as well as keep our source data intacted

Questions

If we use Data Studio as our analytics tool, which DB should we use?

Which ETL system

⁠

Retool Views⁠

: For AI-BO employee

Finance sheet

Can view how much revenue is earned per month in a bar chart

Can view how much was paid out to participants for surveys and referrals in a month in a bar chart

Participant overview

Can see an overview of the demographics of participants (how many of x income etc.)

Can see a breakdown of the highest earning participants and where they earned their income from (total referral vs survey income)

Breakdown of where people came from (acquisition) between referred and not referred

Can view a record of the weekly payout and source of income for payout

⁠

⁠

Quality/verifiability

Exclude or flag people who lie.

Manually look through people’s answers. If we find answers that are obviously not accurate send them a suspension message to say that they will be suspended from surveys for 3 weeks

Add a rating system to people based on responses. This will be visible to them and motivate them to give good responses. Rating based on:

Length of answers in open text questions

Speed to complete survey after receiving template message notifying them that they are eligible to do a survey

How many people they refer

Initial quality screener survey

Uses a red herring technique to assess whether participants are trying to game the system

Automatic truth checker

Social penalisation

Query that finds all people that user has referred and been referred

⁠

https://blog.feedbackloop.com/feedback-loop-comprehensive-data-quality⁠

⁠

Insights Dashboard/Reports⁠

Analytics/insights

Currently being manually created using Tableau and Data Studio

Builds automatic reports based on question types.

Responses can viewed cross-sectionally against the demographic attributes of the respondents OR the MCQ answers of other questions

⁠

Survey Builder⁠

For User (AI-BO customer)

User can add a new question

User can choose what type of question they’d like it to be

MCQ

Open text

Number

Picture

Voice note

Video

User can add media to their question so that the survey participant receives the media (voice note, picture, video)

MCQ

User can add a new MCQ answer for an MCQ question

User can add and edit text for MCQ answers

User can create skip/conditional logic by specifying which question the participant goes to based on a particular MCQ answer

Multiple users can edit questions

AI-BO Backend

Each question created by the AI-BO customer generates a variable name

Question type gets transformed and mapped into bot-builder

Product Role Out

Features roll-out

Features roll-out

⁠

Database

Phase 1 Features roll-out

Phase 1 Features roll-out

Database/Participant Management

Understanding/planning where all activities take

Planning

ELT layer

Phase 1

Client ordering system

Wireframe/plan order journey

Planning

Mapping out user journey

Phase 1

Website

Retool views

⁠

Phase 2 Features roll-out

Phase 2 Features roll-out

Database/Participant Management

Scramble checking survey responses

Building

ELT layerBad Actor

Phase 2

Manual check of answers

Building

ELT layerBad Actor

Phase 2

Quality Check Surveys

Building

ELT layerBad Actor

Phase 2

Client ordering system

Create campaign

Building

Phase 2

Setup campaign objectives

Building

Phase 2

Specify audience

Building

Phase 2

See estimate pricing

Building

Phase 2

Submit for approval

Building

Phase 2

Website

Pricing page

Building

Phase 2

Participants page to sign up

Building

Phase 2

Case study

Building

Phase 2

Retool views

Participants view (acquisition, demographics etc.)

Building

Phase 2

⁠

Phase 3 Features roll-out

Phase 3 Features roll-out

Database/Participant Management

Sample Checks for data accuracy and identity verification

Building

ELT layer & Bad Actor

Phase 3

group open text responses into categories

Building

ELT layer

Phase 3

Client ordering system

Edit/make changes

Building

Phase 3

Make comments/collaborate

Building

Phase 3

Website

Retool views

Finance view (exp, rev etc.)

Building

Phase 3

⁠

💸 Revenue Model

For more about the finances and business model, go to

Business Model & Finances⁠

Pricing

⁠

⁠

📈 Go-To-Market

Continue to be active on LinkedIn and Twitter

Sales Funnel

⁠

⁠

⁠

Inbound

SEO

Organic site traffic will lead to conversions.

We can dip back into referring domains for at least 3 backlinks.

AI-BO will benefit from blog content relating to search queries that AI-BO solves for

Research

Concept validation, testing ideas

Recruiting, accessing people

Getting early customers

SEO will bring 10x more organic traffic than organic social traffic.

There will be more desktop traffic than mobile