Skip to content
Product / Engineering Doc
Product Spec for HBS
Engineering wiki
Alex's Happy Doc
Files
Experiments
Tim Product Roadmap
More
Share
Explore

icon picker
Product Spec for HBS

Data accuracy

Yazi Product for HBS & Berkeley

Below is a summary of the core features and process of Yazi’s system. HBS / Berkeley team would be able to use in the following way:
Use Yazi webapp and Postman to create and make changes to structured questions and directive flows on the fly
Use API endpoints to access User & Survey data
Get GPT data from CSV or setup custom history in GPT codebase
Monitor chats in Rather Chat app
⁠
image.png
⁠
⁠

How the bot works

⁠
image.png
⁠
⁠
Signup and onboarding process which captures agree/disagree to data T&Cs and agree/disagree with opt-in’s to outbound communication (template messages). The following demographic questions are also asked
Country
Age / Date of Birth
Gender
Occupation
Education
Children
Race
Location / Postcode
Business owner
Languages spoken
Main Menu
Options in menu: Survey Menu, Opt-in / Opt-out, FAQs
Announcements bar containing any information Yazi wants to share
Survey Menu
Shows surveys that participants qualify for
Starter phrases
Starter phrases are specific text that, if sent to Yazi’s bot, will take participants to a specific flow. I.e. Main menu jumps people to main menu. Starter phrases are used in buttons so that if a button is clicked, the starter phrase is sent and jumps a participant to a point in the flow.
Survey access: Accessed in 2 ways
Starter phrase: The title of the survey in the JSON
Survey menu: Available to participants who qualify for the survey based on their demographic
⁠
image.png
⁠
⁠

Webapp

Yazi has a webapp that comprises of 4 key components:
Dashboard - see all survey campaigns created for a particular workspace
Specify audience - Choose demographics of audience you’d like to target
Number of participants/respondents
Country
Age
Gender
Occupation
Education
Children
Create survey - Similar interface to Google forms or SurveyMonkey.
Types of questions: Single-select and multiselect multiple choice, Open text, Image/Video/Voice note
‘Other’ question toggle which automatically asks ‘Which other/s?’ if other is selected
Logic: Can skip or jump participants to certain parts of the survey or flow using logic in single-select multiple choice questions
Results - Bar graph of each response with ability to filter and select using both other questions and demographic information
Once the survey campaign has been launched in the webapp, the JSON generated by the webapp needs to be POSTed to a Rather Chat endpoint to be live on the bot.
Product notes:
Progress is not saved automatically, remember to save with the button
The webapp does not currently support ‘multi-player’ teams. One set of login details can thus be shared and used.
To tryout webapp, use the following account details:
URL:
https://app.askyazi.com/
⁠
EMAIL:
rclarke@hbs.edu
⁠
PASS: uUBD49*q5&PM
⁠
image.png
⁠
⁠

Connecting surveys to bot

Retool

Yazi uses Retool to send a survey from the webapp to the bot. Sharing user access to Retool with HBS & Berkeley is not ideal and so instead, postman can be used.
Connect or create launched survey to organisation in database
Create survey name in database and connect to survey created by webapp.
This automatically creates a survey ID and imports the survey JSON which contains:
Number of participants
Audience specification:
Age
Gender
Income
Job level
Children
Education
Custom variable
Post survey JSON to Rather Chat endpoint which automatically generates the bot ‘flow’.
⁠
image.png
⁠
⁠

Postman

Postman can be used to retrieve (GET) the JSON generated by the webapp after it has been connected to a survey in Yazi’s database and it can be used to POST the JSON to Yazi’s bot.
⁠
image.png
⁠
⁠

Rather Chat app

Send template messages to participants by uploading a CSV of cell numbers to send to
See chat history from last 24 hours (see below screenshot). A Rather Chat user can send a message to a participant as the bot in this chat
See bots or flows running + Team members + Channels/Numbers (if different cell numbers connected to FB account)
⁠
image.png
⁠
⁠
image.png
⁠
⁠

Meta WhatsApp Manager

This is where template messages are submitted for approval. HBS & Berkeley would need to be invited as admins to Yazi’s Facebook business account to get access here.
Templates typically take 1-3 days to be approved.
⁠
image.png
⁠
⁠
It also shows usage, conversations and engagement with the bot in a dashboard

User Data

User data can be fetched using an API endpoint that has been setup to be accessed via a link in a web browser. The user data is in a JSON format like the below:
{
"id": 21937,
"phone_number": "2547118540***",
"created_at": "2023-05-12T12:45:29.117203+02:00",
"notify": true,
"ban": false,
"first_name": "Dominic",
"birthday": "1997-09-27",
"business_owner": false,
"income": 250,
"postal": "2201",
"region": "Suburb: None, City: Gatunga, Province: Tharaka-Nithi County",
"user": "whatsapp_aibochat:254711854***",
"gender": null,
"country": "kenya",
"preferred_language": "english",
"race": null,
"relationship": null,
"children": null,
"job": null,
"education": null,
"languages": []
},

URL:

⁠
https://bot-yazi.comparisure.co.za/api/profiles/
⁠
⁠
The layout:
The count is the total number of profiles there are, these include completed & uncompleted profiles.
The results is a paginated list of 100 profiles, once the count is more than 100 it will be paginated.
You can use the next and previous links to offset the results.
The content:
The fields are pretty self-explanatory, however, if a field is null it means the user has not yet provided that information
The id is the "id" of the profile, not the user. The user's ID is stored in user. This is the same user ID used in the survey results CSV you receive.
ban is the field for the "bad actor score", which is used to prevent the user from seeing surveys on the menu, this does not prevent them from doing the survey if you send them a template
notify is the field that indicates whether the user has "opted-in/-out" for updates on new surveys.
Additional uses:
You can view individual profiles by adding the profile "id" in the URL e.g.
https://bot-yazi.comparisure.co.za/api/profiles/8/
⁠
⁠
You can update a user's profile by calling a PATCH request to the above mentioned individual profile endpoint, this could be useful for banning / unbanning someone
You can also delete a user's profile by calling a DELETE request to the individual profile endpoint
⁠
Screenshot 2023-03-16 at 16.23.42.png
⁠
⁠
Limitations:
It does NOT allow you to search according to a specific parameter (its possible, but would require additional work)
⁠
Screenshot 2023-03-16 at 16.31.33.png
⁠
⁠
URL with sort
⁠
https://bot-yazi.comparisure.co.za/api/profiles/?limit=100&offset=20600&ordering=id
⁠

Survey Data access

Each survey campaign created by the webapp has its own surveyID. An API endpoint has been created for survey data, which can also be accessed via a link in the browser. This data is a CSV file type and displays in the following format:
1
userId
userName
responseTimestamp
responseQuestionId
responseAnswer
There are no rows in this table
⁠
Yazi runs a script that transforms this data to have a single UserID per row and each question variable in a column. The data is joined with the User data and sent to a Google Sheet. It also checks for speeding and negligent open text characters.
responseQuestionId
The variable created and associated with a survey question. The variable’s response can be used in logic or in chains in the GPT codebase

URL of raw CSV data:

⁠
https://bot-yazi.comparisure.co.za/api/survey/104/report/?from=2023-04-09T00:00&to=2023-04-30T00:00
⁠
How URL is set up:
⁠
https://bot-yazi.comparisure.co.za/api/survey/{survey_id}/report/?from={start_date}&to={end_date
⁠
}

GPT Pre-Pilot example with Python script

Python script formats the data and does the following:
Reformats so that UserIDs are in rows and questions in columns, sorted in ascending question order
Joins signup data on survey data
Adds columns for start / end time
Calculates average time per response
Data accuracy
Speeding checker: under 7 seconds average per question = fail
Negligent open text: single letter characters like ‘a’ or ‘c’ are flagged as negligent in open text questions
Send all data to a specific Google Sheet tab
Show proportions of responses and respondents in a different Google Sheets tab
⁠
https://docs.google.com/spreadsheets/d/1wWDOdC7sqcbFraoJOAlIwQqf7MHVq5sROX1joqbgYio/edit#gid=1588847040
⁠

⁠
image.png
⁠
⁠

GPT Engagement Data

GPT prompt and response data is sent in a CSV file to
accounts@askyazi.com
⁠
everyday at 00:00 and it contains the last 7 days of data. Yazi joins the CSVs together to the full time period of data.
1
userId
responseTimestamp
prompt
responseAnswer
origin
There are no rows in this table
⁠
⁠
image.png
⁠
⁠

GPT bot system

Each model (trained or vanilla GPT-3.5 or GPT-4) is connected to a Starter phrase. Once the starter phrase is sent to Yazi’s bot, the following is sent:
“Welcome to ChatGPT business advisor. You can ask me any question like, “What are your top 5 recommendations to boost your small business’s profits and sales in South Africa?”
To protect or restrict access to GPT, the following can be done:
Starter phrases can be updated on a regular basis
Access to GPT could be restricted to specific user IDs
Rather Chat has agreed that HBS and Berkley can get full access to GPT code base and make changes and iterations to prompts and chains to enable frequent testing and updating.

Live view of Survey completions

You can access the data via this API
⁠
https://bot-yazi.comparisure.co.za/api/survey/{survey-id}/
⁠
⁠
For example:
https://bot-yazi.comparisure.co.za/api/survey/83/
Will return:
{
"id": "83",
"title": "Project Fur Baby Visual Task Diary",
"created_at": "2023-03-09T10:24:13.234009Z",
"n_started": 13,
"n_completed": 2
}

Summary:

Data
Well structured signup process that collects 10 demographic data points - easy to download demographic
Easy download of survey response data, joined with signup data
Easy download of GPT chat histories
General navigation
All navigation built around the main menu - the dashboard or home page of our bot which makes using the bot very simple for non-tech native participants
Error messages for if required answers aren’t given in multiple choice or text specific question (postcode, cell number etc.)
Starter phrases to jump participants to specific ‘flows’
Logic in surveys or questions to direct people to specific places based on both the previous question and data collected prior to that question. I.e. different paths for participants based on which treatment group the participant is part of
Ability to go back to previous question (if made a mistake in response)
Menu
Surveys or research tasks shown to participants based on specific characteristics in the ’Survey menu'
Malleable bot design to create menus for diary studies or FAQs very easily - with ability to navigate to different flows or places in chat
Personal communication
Template messages or notifications that allows Yazi to nudge participants to use chatbot
Can add variables and reference them in chat or chain into GPT prompts
Incentive tracking on bot menu to show participants how much they’re owed
Full use of WhatsApp new features. The type of data Yazi collects
Multiple choice - Buttons
Multi-select multiple choice - Participants can indicate their selections by providing the associated answer letter (e.g., A, C, F), separating multiple choices with commas (e.g., A, C, F)
Video
Pictures
Voice notes & transcriptions
Webapp to be able to launch new flows or questions to specific audiences immediately
Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.