Research and Analytics Team Crit at Slack

How We Run Crit in Research and Analytics at Slack

Explore

How We Run Crit in Research and Analytics at Slack
How We Run Crit in Research and Analytics at Slack

How we're keeping our team connected in a distributed world, building a feedback culture, and cross-pollinating between data science and research using small group Crit pods.

How we're keeping our team connected in a distributed world, building a feedback culture, and cross-pollinating between data science and research using small group Crit pods.

Austin Wilt

☣️ The motivation for Crit ☣️

When the pandemic forced us into distributed work in Spring 2020, we found new challenges for team culture and for our work. By Summer 2020, once the novelty and connection stemming from the shared experience at the outset of the pandemic wore off, group members reported feeling disconnected from the Research & Analytics (R&A) team at large, and increasingly removed from work going on outside their immediate area.

This presented a huge problem for our effectiveness as a team, and a huge concern for our team culture. There were 3 key superpowers of our org that felt compromised.

🦸‍♀️ Superpower 1: We bring broad, wholistic perspective to our product partners

One of our superpowers as a team is bringing a wholistic perspective to our product partners, and we can’t do that without broad exposure and context. As casual interactions were all but eliminated in a fully remote world, everyone was losing context on other parts of the organization, other work happening, and outside thoughts and ideas.

🦸‍♀️ Superpower 2: Trust supercharges our impact

We are advisors to the Product org, and our impact is predicated on trust, which means consistent work quality is paramount. We were finding that our team size, structure, and resourcing plan meant that often team members were working fairly independently on projects. We didn’t have any formal process for internal feedback, and without co-location we weren’t incorporating nearly enough feedback into our project lifecycle.

🦸‍♀️ Superpower 3: Pride permeates our team

Our team thrives in part because of a strong sense of pride that permeates the R&A org. A big key to powering that pride is a deep sense of connection to our teammates and our organization. Each member of the team matters to our culture and our collective success, and in order for each person to deeply feel that they matter they need to feel connected to and seen by their teammates.

💡 The advent of Crit Pods 💡

We could have addressed these challenges any number of ways. We chose to group them together because their solutions are complementary. Sharing work to build awareness presents a great opportunity for feedback. Giving and receiving feedback is done best in an environment where trust abounds, and trust is easier to build in a small group focused on creating a safe space. Before I tell you exactly how we run crit today, here’s a bit about the process we used to get here. Nothing is perfect at the outset so I don’t want to skip from problem to perfect solution. Plus, a process like this is very repeatable so think of it like a bonus how-to guide. Keep in mind the appropriate amount of initial structure will vary depending on the project.

🥅 Decide on the objectives

We got clear on the purpose of crit, which is:

🔗 To build connections with R&A team members

🤝 To give and receive quality feedback on R&A work

🧠 To increase awareness of work outside your area

We aligned on that mandate as an R&A leadership team so we’d all be on the same page about what we were meant to accomplish.

🕒 Create the physical space

We decided as an R&A leadership team that this was a priority, and we’d all commit to 1 hour per week to this practice. We communicated that expectation to the broader team, and encouraged our leads to lead by example in prioritizing Crit. Everyone is busy, and we can’t succeed at something if we don’t commit the time needed to get it off the ground. This trick works for so many things at work and in your life - if you’re struggling to make something happen, assess whether you’ve carved out the physical space needed.

🤔 See what works

We weren’t sure exactly what form factor would be most effective, so we treated this a little bit like a

mutli-armed bandit⁠

experiment. We created groups with leads at the helm (remember, those leads had a clear understanding of the goals), and set them off and running. Leads had a lot of latitude to decide how to run their Crit pods, because we wanted a diverse set of ideas and methods at the outset.

As expected, we found some things that were successful and some that weren’t. Several Crit pods fizzled out quickly. We sought feedback informally and formally and shared it back so everyone could benefit from understanding what was working best, and as we developed best practices we turned them into guidelines.

⛰️ Embrace Continuous Improvement

Once we saw the benefit of Crit and decided to make this an ongoing and prioritized team ritual, we knew we needed to ensure we were making the most of this practice.

Early on we checked in regularly with the leads and a selection of ICs to understand what was working and what wasn’t, and to know when crit pods needed some redirection. And we regularly shared our thoughts back with the team to keep crit top of mind and to keep the practice on a path to improvement.

We continue to solicit feedback regularly in an informal way (either broad calls for qual feedback, or direct questions in 1:1s), and we ran a survey 6 months in to quantify how we were doing and what to focus on improving. At this point Crit feels fairly self-sustaining and we’re investing less in feedback and improvement, but I don’t think that practice will ever end entirely.

🍎 Crit Best Practices 🍎

Here are our guidelines for successful Crit.

✋ Ownership

Crit needs an owner at the team level. This person is responsible for the overall program - setup, program health, maintenance, and continuous improvement. They don’t have to do all these things, but someone needs to feel ownership over Crit and responsible for its success.

🐬 Creating Pods

The makeup of the groups is important. Here’s what we’ve found to be a good set of guidelines. It’s not always possible to meet all of them and we make compromises when we need to.

Groups of 5-6. 3-4 feels minimum viable for a meeting so there’s wiggle room if people miss. More than 6 starts to feel too large for the intimacy we’re trying to foster.

No manager/IC pairs in the same group.

Diversify groups across data science and research, and across areas of the business to promote relationships that won’t naturally happen in day to day work.

Consider mentorship opportunities within a crit pod. Pair team members with someone who they can learn from.

Two “leads” (willing senior folks that others look up to) per group responsible for coordination and setting the tone.

New hires are put in crit pods immediately.

We’re currently planning to mix up the groups every 12 months to give groups a chance to gel and form a bond, but mix it up every year

🤝 Fostering Trust

We’ve found that trust is paramount in these groups, and that we can create a safe space by intentionally fostering vulnerability and trust. We do this in 2 key ways.

Safe space clause. Crit pods have a safe space clause, where content and discussions have an expectation that they won’t leave the group.

Groups do a trust building exercise for their first 1-hour meeting, and repeat periodically if new folks have joined, or as it feels necessary. There is an infinite wealth of these ice breaker type get to know you exercises available, and it can be nice for leads to choose their own. We’ve found activities that focus on sharing personal reflections, and preferences around feedback are helpful. Here are some examples:

Have everyone write a

personal operating manual⁠

and share with the group

Have everyone write the answers to 2-3 questions and then share back. What are 2-3 skills you’re working on right now? How can the group be most effective in giving you feedback? What types of feedback do you find least helpful?

🖨️ Format and Work Sharing

Find a good balance between social time and work time. Connection is key in this group, so spend time at the beginning of each meeting catching up and checking in.

We’ve found that a no-signup, informal work sharing arrangement works best. Ask everyone what they’ve brought to share, and see what feels most pressing, or most interesting to the group. Depending on the size of each topic we’ve found we get through 1-3 topics per meetings. Keep an eye out for air time overall to ensure everyone is taking turns in sharing.

Encourage sharing work at any stage. Early work sharing tends to foster great discussion, so encourage folks to bring early ideas, thoughts, questions, or outlines. They can have an output to share like notes or a deck, or nothing at all. Sharing later in the lifecycle of a project is great too, and sharing finished or nearly finished work is also encouraged.

Give the leads some prompts to put in their back pocket in case nobody has work to share. We always find there are team topics that would make good discussions, and we equip our leads with a short list of topics. If there’s dead time in a Crit meeting they can fill it with a meaningful discussion.

How do you think we’re doing at working on the most important/impactful things? How are you feeling about our ability to triage requests and say no to less important items? How are we doing at ensuring our recommendations are having impact on the product? How did the most recent planning cycle go, and is there anything we can do to improve next time?

☮️ Feedback

This has been the hardest part. Giving and receiving feedback isn’t easy. Here’s what we’ve found success with.

Have the presenter say up front where they’re at in the project, and what type of feedback will be most helpful. Maybe they’re just starting out and they want feedback on the overall direction. Or maybe they’re 95% done and they just want feedback on their polish and presentation. Or anywhere in between. Let them say what they’re looking for.

Encourage the leads to model giving and receiving feedback. For us, the battle has always been around people feeling uncomfortable giving feedback, so we’re always trying to figure out how to encourage more feedback. Feedback should be thoughtful, tangible, actionable, and factual. It should be a description of how to make the work better rather than its shortcomings.

Take hard conversations offline if necessary. Maybe there’s a junior team member that presents a project which appears to be very off track, and you know that person doesn’t do well with tough feedback in a public forum, even in the small Crit Pod. It’s ok to take the feedback offline and followup with them after the meeting so as not to compromise the safety of the Crit group if necessary.

👪 Reflections 👪

👑 Trust and Safety is 👑

You may be wondering at this point why there are so many emojis in this post. They’re here because that’s how we write at and communicate at Slack. We use the apple as our internal emoji for Crit because Crit is our classroom, our safe space for feedback and learning. I think if I had to do it over again I would choose an emoji for family, because while I thought at the outset Crit was going to be about feedback and learning, it’s actually fundamentally about connection and safety.

🎵 Leads set the tone

We’ve found that leads really set the tone for Crit. Some of the biggest challenges we’ve had around the success of Crit have been in getting regular attendance, having enough content at meetings, striking a balance between social and work topics, and getting people comfortable with giving meaningful feedback. In all of these cases, asking the leads to model the behavior we’re looking for has gone a long way.

🦸‍♀️ Superpower 4: We have a spectrum of research and data science tools to draw from

We’ve found that there’s a 4th superpower that Crit has furthered within our team, which is cross-pollination among researchers and data scientists. Previously folks only worked together on specific projects, and crit has given everyone much more exposure to skillsets and tools within research and data science. Hearing how a researcher goes after a problem. Diving into the details of data analysis. Critiquing a survey. This exposure is incredibly valuable to everyone on our team building a broad understanding of all the tools we have at our disposal, making each person a more valuable partner to the Product org.

❤️ Good luck!

We often hear from team members that Crit is their favorite meeting of the week. Or that it’s the single thing they love the most about our team, or it’s the things that makes them feel most connected to R&A. It’s an incredibly powerful practice, and we hope you’ll give it a try.

Copy this doc

⁠

and check out the

“Launching your own crit practice”⁠

page to get started!

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.

How We Run Crit in Research and Analytics at SlackHow We Run Crit in Research and Analytics at Slack