Explore

GoHigh Level Learning Retention PRD

1. Goal

Implement a systemic solution - a feature where users can write key takeaways/learning after every course module, enhancing learning retention of the content within the GoHighLevel course platform.

2. Context

Currently in LJ v6, the user journey of finishing a deck is as follows:

User starts a topic/deck → share some feedback in between → user finishes reading a topic/deck → rates the content → marks it done → moves on to next topic/deck

As of now, there's no systemic mechanism for long term retention of content though users can make their own notes for the retention and revision purposes.

There are 3 types of user persona in terms of making notes for content retention:

Table 1

Table 1

User Types

Want to make notes

Actually make notes

Note taking tool used

Reason for not making notes

% of users(estimation)

Yes

3rd party

Yes

cumbersome

There are no rows in this table

⁠

Users who fall in persona Type A would be far better off than users in persona Types B & C in terms of long term retention of the key concepts from a specific topic [backed by research on effective learning → refer section 4]

3. Problem

3.1 User problem:

The upon feels less confident and more anxious

If a user did not make any notes or jotted down some key learnings while learning then after finishing a few decks, there are very low chances of recalling concepts from the previous decks

The existing remedy is time & efforts consuming

If a Type B and C user doesn't revise the concepts, then he/she performs poorly in assessments and eventually in interviews → poor performance If Type B and C user chooses to revise the concepts, then he/she again needs to devote good enough time to finish the topic → time-consuming & extra efforts

3.2 Business problem

Such a user either gets no placement. or takes too much time to get placed

If a user is not performing well in assessments then his/her confidence sees a dip

Such a user is highly unlikely to perform well in placements

Hence, the outcome gets impacted negatively either directly (no placement) or indirectly (consuming extra resources-time & effort)

Learning → Retention → Application

4. Research on the relevant problem statement

Let's try to understand the difference between 'How we learn' vs 'How we think we learn’

Robert Bjork is a distinguished Professor in the Dept of Psychology, UCLA. Не has done extensive research on ways of effective learning. Subtleties of the way learning actually occur are far different from the way we think how it works

Some key insights [reference links are given at the end of the section]

When DO not out to learn something, we typically have a goal in mind - that goal is to perform better in later instances, for ex - hours of net practice is done to perform better in an actual match

But there are some ways, if Integrated within the practice regime, can result into long term retention and hence, a better performance

It's called introducing some sort of hurdles, known as 'desirable difficulties in the learning process. These are called difficulties because they make the learning process somewhat difficult & slower and are called desirable as they will make you retain things for a longer duration and hence, make you perform better. These are 4 in number:

Varying - Varying the conditions of learning than keeping them constant for example, changing the place where you chose to practice or learn - it could be the library some days, it could be your dorm room other days. Just like other desirable difficulties, this one also goes against the traditionally given advice to students to learn better

Spacing - Space our or introduce a time interval between the repeated study sessions in order to recall them later in a better manner

Practice retrieving than re-studying - Every time we retrieve, we modify our memory. Hence, retrieving information, i.e., writing tests or key takeaways is always better than revising or restudying the same material

Interleaving than blocking - If a subject matter is inter-leaved instead of finishing it in a block, it serves better in terms of long-term retention, For example - planning to finish Math in Week 1 and Science in Week 2 is going to be less effective than planning to finish Math and Science both simultaneously in the span of 2 Weeks

Robert Bjork's research suggests 3 benefits of information retrieval (which is usually implemented via tests or writing summaries in your own words)

It modifies memory → hence consolidates the information (which is easier to retrieve when required) in a much better manner

It provides feedback → you get to know how much information has actually been ingrained into your memory

It makes subsequent study for more effective If you get few or no correct answers + Even errors in tests or retrieval process help in cementing that piece of information in terms of long-term retention

Reference

Lecture by Robert Bjork on

How we learn vs How we think we learn⁠

[65 min]

Ted Talk by Elizabeth & Robert Bjork on

How to maximize performance⁠

[15 min]

5. Why to solve the problem right now:

Commonly demanded feature by the users

As discussed in section 2, 50% of the users and up not making any notes as the process is cumbersome → such users have raised demand of such features

Problem would be far worse in the future when we have large cohorts

Building systems now for the Long term Retention is necessary to increase the chances of positive outcomes in future

6. Possible solution exploration

6.1 Solution

We are going to solve the problem by introducing one of the 'desirable difficulties - practice retrieval than revision' as proposed by Robert Bjork, in our learning system → adding a section at the end of each topic which makes users jot down key takeaways from the deck they have just finished.

Users recalling or retrieving information in order to write key takeaways would help in:

long-term retention of the piece of info they would be able to recall → consolidation

figuring out the piece of information they struggled a lot to recall → revise it asap → holistic

6.2 User Journey

Current user journey

User starts a module → 2) share feedback in between → 3) user finishes module→

rates the content → 5) marks topic as done → 6) moves on to next module

Expected user journey Same as above with just one additional step in between steps 3) & 4):

User starts a module→ 2) share feedback in between → 3) user finishes module 4) writes/submits key takeaways → 5) rates the content ب 6) marks the module as done →

moves on to next module

6.3 User needs

Table 2

Table 2

Write and submit key takeaways

Low

High

Delete key takeaways

??? Not in v1

Edit key takeaways at a later stage

High

Med

??? v1

Access key takeaways at a later stage

High

Med

Receiving key takeaways on email [as revision]

High

Med

Share it with friends on social media - LinkedIn, Twitter

High

Receiving something in return of submitting key takeaways like key takeaways written by course experts

High

Low

There are no rows in this table

⁠

7. Implementation

7.1 Project Details

7.1.1 Scope - small

This falls in the larger ambit of building systemic solutions for achieving the objective of long term retention of the content, key learnings - Enhancing learning Retention

As of now, в larger vision is to have functionality for the users to create notes, underline or highlight key points in the decks - all while going through the decks only!

In the context of this specific project — scope is narrow → we need to just focus on adding a feature which will ask users to write key takeaways from the deck just after finishing the deck

7.1.2 Timeline - quick

Plan is to handoff to the engineering team by 3rd week of October

7.1.3 Quality - high

Quality should be high as its continuous usage is dependent on the experience → bad experience will also be counterproductive from the lens of entire learning experience

All the possible edge cases have to be taken care of

Need to think of all the constraints

7.2 Final Requirements

There should be 3 core functionalities in v1:

[1] Adding a ‘feature’ to write ‘key takeaways’ within the LMS

[2] Saving the key takeaways in the database & sending the email to the user

[3] Keeping a count of individual # of key takeaways being written in a sequential manner

7.2.1 Functionalities

For [1] adding a "feature" to write ‘key takeaways’ within the LMS

This will be launched for new users only and not for the existing users

It is to avoid user dissatisfaction amongst the existing users. For them, it would be an additional work

For a new user, this would be the only reality in his learning journey or LMS

User should be able to find this feature of writing key takeaways in every topic page → tied to every learning topic (google slides deck) in LMS exception are topics from program tasks modules)

User should be able to open it and close it even without writing key takeaways

User should be able to write anywhere between 1 - 10 [including the lower and upper limit] takeaways every time pertaining to each topic

Once written, user should be able to submit those takeaways

Once submitted, user should not be able to delete OF edit takeaways → this will be explored in v2

For [2] saving the key takeaways in database & sending the email to user

Once submitted, user should be able to revisit key takeaways in future n number of times

Once submitted, following data should get stored in database:

user name, 2) email id, 3) text of actual takeaways submitted,

module & 6) topic pertaining to the takeaways submitted, 7) total count of takeaways user has written till now.

Once submitted, user should get an email. Details of the emails are as follows:

Table 3

Table 3

Email Attribute

Format

Example

Sender email id

{user_firstname}

learning@gohighlevel.com

⁠

Jay@gohighlevel.com

Receiver email id

learning@gohighlevel.com

Email Subject line

{user_firstname}

There are no rows in this table

⁠

For [3] Keeping a count of individual $ of takeaways

It is required to keep a track of individual number of key takeaways submitted by user

Once submitted, user should see a screen every time congratulating the behavior and showing the updated individual number of takeaways he has written till now

7.2.2 Constraints

Maximum a of takeaways for a topic - 10

Maximum character limit for each takeaway- ~280 (this will be a soft limit not strict or mandatory) in v1 and once a user surpasses 200 character, we should show him that he should wrap it up to keep it concise)

Minimum character limit for each takeaway-1 (по need to put min char limit restraints as user can anyway game this system, rather we should encourage the behavior of writing)

It is mandatory to write takeaways pertaining to all the decks .. user can mark a topic done only after submitting key takeaways. Till then the CTA of 'mark as done" should not be enabled and should have a prompt asking to submit takeaways

Copy functionality should be enabled but paste functionality should be disabled

Pasting the info from own notes or slides won't justify the objective this feature is having → writing is compulsory

It might happen that user would have to do double efforts for the 1st time → once making notes in A word doc and then writing same points in takeaways → frustration

But it's okay as the user is supposed to go through this 'pain’ of writing - will be a learned behavior afterward

No font related functionality is required in v1 → There should be a single default font size, font type (appropriate for web and mobile screens respectively)

7.2.3 Scenarios

User writing takeaways for the 1st time

There should be an additional screen showing the reason or importance of this feature → this screen should show up for 1st 3 times users will De writing takeaways

User hasn't read the deck but wants to mark the topic as done

Not possible → mark 20 done button should be enabled only after writing & submitting takeaways. Till then it should have a prompt asking to submit takeaways

User finishes a deck, doesn't mark topic as done and leaves or closes the window

No new action required-> current usual flow should take place as user would be seeing tasks as pending in his task lists → this will be a good enough reason/cue/trigger to submit takeaways, as of now in v1

User starts writing takeaways but closes the window without saving them

Prompt should come → please save else progress will be lost → though user should have both option - 1) ignore the prompt and close the window or 2) save the takeaways by submitting

If user chooses option 1) then current usual flow should continue, i.e. task will be shown as pending

User starts writing takeaways and wants to make or use a line break in one single bullet

Should be allowed as many line breaks as he wants [same max character limit rule will be applied here]

User starts writing takeaways → writes 1 bullet, 2nd bullet, 3rd bullet → takes cursor to 2nd bullet and deletes it

It should be allowed → but now effectively he is having 2 takeaways → 3rd bullet should automatically becomes 2nd and so on

7.2.4 Things which have to be kept in mind for future scope/versions

[Out of scope for v1]

User should be able to edit takeaways after a time limit

User should be able to access all the takeaways 20 a single global place in LMS → grouped as modules

User should be able to share the takeaways written on social media → 1) each individual bullet in a topic takeaways as Individual text or 2) each individual bullet in a topic takeaways as an image or 31 entire topic takeaways as a text block or 4) entire topic takeaways as an image

User should get something once he submits the key takeaways

7.2.5 Wireframes describing details spec’d out above

7.2.6 Events

Table 4

Table 4

topic_page

url

4 August

For Review

user clicks on “add takeaways” button on the attachment

attempted to initiate takeaways

attachment_name

{attachment_name}

topic_name

{topic_name}

module_name

{module_name}

user clicks on “view takeaway” button on the attachment

attempted_to_view_takeaways

attachment_name

topic_name

module_name

user clicked on the “X” on the top of the modal

dismissed_takeaway_popup

takeaways_state

before_writing, after_writing, after_submitting

total_takeaways_added

0, 1,2,3,…,10

user presses “write key takeaways” CTA on the onboarding flow of the feature

finished_takeaway_onboarding

user presses the “save key takeaways” CTA button

attempted_to_add_takeaways

attachment_name

{attachment_name}

topic_name

{topic_name}

module_name

{module_name}

total_takeaways_added

1, 2, 3, …, 10

There are no rows in this table

⁠

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.