Yazi Repo 2.0

Explore

Old Retros

Kearney Report

Kearney Debacle

Rough Timeline:

Wholistic Timeline

Wholistic Timeline

⁠

Table

Table

⁠

Planned Timeline (At Start):

6 May: Meeting to discuss next week’s plan, run through the finalized survey questions and answer any questions

9 May: Meeting to discuss input provided on the survey from AI-BO & dry-run survey with the team (invite to be shared)

10 May: Distribute survey to the SME’s (including our ring-fenced SME database)

16-17 May: Receive responses & clean data

18 May: Final dataset (in Excel format) ready for analysis

Rough written failure summary:

It is difficult to accurately separate and define each issue or problem as it was an amalgamation of multiple occurrences throughout the process. A series of unfortunate events in someways. I will try to summarize below, from my perspective, the main issues of failure followed by solutions, and learnings to take forward into the future

Conversation started with Francois, being referred by our previous clients John-Luke. It seemed as though the timeline was short and they wanted the results as quickly as possible. From the time of initial email introduction, through to the finalization of pricing, and then first questions draft being sent by Francois took over 2 weeks.

In some ways this is unavoidable, I believe this process can be sped up significantly in future.It was definitely an initial lag on the campaign.

An expectation, and assertion of ability to gain 350 responses at the given price of R150 per response was agreed upon in the beginning. I believe this caused many issues going forward, (especially upon discovering the confusion requiring every response being registered with the CIPC) the initial agreement was based off having 1000 business owners already in our database [but not comprehending at the time that these were 95% unregistered and informal] therefore would not qualify.

Initial conversation occurred through Tim and Francois, and the expectations of costing and capability was set. Once the agreement was made, Ian took over entirely with the survey creation and interaction with Francois [the client] from this point forward.

Through the week of the survey creation, there are a number of back-and-forth changes with Francois in the question order, question designed addition and lamination of multiple questions. Upon inquiring about all inputs that they desired for improvement on the questions – François explained the specific nature design of the questions [only wanted advice on the variation of phrasing each question]. I gave a number of short suggestions, and encouraged a shortening of some very long questions, this thing extended the period of changing the survey due to the need to check amongst his team before finalizing.

An additional external stakeholder, just before the survey to be launched also made some additional changes. All these additional changes and variations did cause some issues with the logic and accuracy of the Landbot flow. I did ask Tim, Matt and Jayde to attempt the survey to make improvements and check the flow - although partially done I believe there is a significant error in this regard with the checking (In general with all our surveys).

Who desire from Kearney to include a ranking question, was accommodated for and I spent a good portion of a day creating this new type of question using regex formulas within Landbot. The lack of extensive testing, the short timing and nature caused many issues with this ranking question as it is not been thoroughly tested. Particularly the usability and length of time it took participants to attempt the question cause the block in answers.

Currently required veering fence business owners to be separated completely from my database, which in itself was also a new experiment which took a number of testing and bug fixing to get correct. This took up a significant amount of time but also caused complex data analysis influx within the sheets.

Upon launching the survey, certain issues were only discovered after a number of results and attempts by respondents – highlighting the need for multiple test scenarios in their creation phase, also to ensure variation in the sheets. [Number of different responses, to insure the sheets is accurately connected] and a more thorough look into the sheets to check accuracy before official launch..

Upon the realization of the limitation of Whatsapp, the difficulty, and lack of responses – the shift to top form occurred. This although a far easier platform to use, was created in a rush, out of desperation – and caused a multitude of issues and problems for the downline. Including the alignments of information, the missing question, not having the name or requiring the registration number of participants for validation.

The week of recruitment, seemed to be a big fail. After four days of intensive recruiting, it seemed as though we only received around 10 responses from all the platforms used. [Facebook, LinkedIn, Facebook advertising, emails, cold calls etc.]

There was one of many two issues that occurred within Facebook marketing for this demographic, the groups we posted on Waze so large that there was too much spam [and no one believed or into the link as it was overwhelmed with other spam on that Facebook group], proceeding to try post on smaller more intimate and refined groups, were either rejected when attempting to enter the group, or the post was rejected as it was not exactly relevant and seemed to be a scam. The realization that there is a big trust issue, especially with this level of participant [business owner]. Due to the number of scams and fake posts, there is no trust established.

There was also a massive delay with an acceptance of Facebook groups, and with an acceptance of posts. A number of learnings around how to post [and use in the comments rather than in certain link within]

All of the above are some of the major issues I noticed within this campaign. A summary below of bullet pointed issues may help summarize the above.

Failures:

The desire to adapt, change our policies/normal procedures in order to accommodate for Kearneys needs however at the cost of many issues and errors that came from it.

Too many new experiments and variations for such a big and unique client. [ring fence flow, ranking question, type form]

An early expectation and assertion of quantity and price. [without flexibility of change]

Speed of initial discussion to first questions. [Slow and Timely]

Change in management lead [and communication within our team], direct WhatsApp messages, and emails in which both Tim and I were not always included together. Disjointed understanding of timeline, expectation, and regular communications with the client.

A confusion, blurry line of platform versus service understanding by the client.

A long, timely and multitude of changes to the survey questions – caused number of issues

The change in WhatsApp to type form and rush in doing so.

Business owner recruiting delay and bad performance. [Facebook groups, emails, code calls]

Learnings:

The firm in our capabilities/policies/procedures. Create rules, strongly suggested formats [length of characters within multi select answers and ranking questions]

Business owners, especially registered ones, I found more difficult to acquire then urgently anticipated due to trust and other issues.

Separate business owners entirely from our WhatsApp flow, look into using top flow in the future for business owners.

Ensure testing of experiments before new clients extensively.

Creating a system, procedure in which either internally, or within the large Delta [at least three or four people] to thoroughly check the survey usability and general flow before launching the survey.

When using Facebook groups, it's far more effective on smaller and private groups when trying to access business owners.

To adapt, and accommodate for the delay and lag with him being accepted within a group and making a post, with the restrictions etc.

Clearly define the difference between a platform and insights service.

Potentially create document a sign, or just agreed to an email of the invoice price being based on disability to achieve, as well as the assumptions made.

Being able to change the price, and responses during the campaign after being prayed upon by the client – if the incident rates or some other event occurs changing the initial assumptions.

With every client to ensure entire team is aware of all interaction and communications and expectations. Avoid personal WhatsApp communications unless it's on a group, and always cc in each other with an emails.

Short summary of learnings from Tim

Sales doc with T&Cs to accommodate for changes or not achieving respondent goals during survey

Character limits per question and MCQ answer

Test survey with group or create testing ritual

Treat bus owners separate to B2C

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.