Share
Explore

icon picker
Interview load balancing: How Coda’s engineering team scales fast

Making interviewing fair, efficient, and flexible
Oliver Heckmann
Running a great interviewing process has been a critical success factor for organizations throughout my career, from academia, through building multiple large teams at Google (Gmail, YouTube, Google Shopping, Google Travel), to leading the engineering team at a Silicon Valley startup.
In an ever-growing company, the amount of time employees spend interviewing potential candidates can be pretty high. The Coda engineering team has designed our interview process to distribute the lift across the team fairly, efficiently, and flexibly. We achieve that by designing for efficiency and flexibility, and by load balancing at two stages.
If you want to use our process and adapt it to your needs, we are happy to help you set up your own Interview Load Tracker in Coda. Contact our team and me here
Contact Coda
⁠
⁠

⁠

Designing for efficiency

In a high-capacity interview process, it’s incredibly important that interviewing is efficient. We believe that training new interviewers should be easy and fast, interviewers should be well calibrated for their interview so that their feedback has maximum impact, and that the overall process should be as short as possible to make a decision.
Given these needs, we use a standardized process for hiring engineers where every candidate for the same role gets the same interview questions, and interviewers specialize in a single interview question. This strategy creates higher quality interview feedback, as the pool of interviewers is smaller, and individual interviewers see more candidates solve the same problem. We also invest some thought into fitting the interviewer to the right interview type. And of course, if an interviewer gets tired of their interview, we rotate them to a different one.
Each interview question has a scoring rubric, a person who handles training new interviewers, plus a Slack channel for the interviewers to calibrate in situations not covered by the rubric and training. Also, each hiring decision maker is assigned one of the questions. When they read all the interview feedback, they pay extra attention to the quality of the feedback for their question and provide (positive or negative) feedback to the interviewers as needed. Here’s an example of what that rubric looks like:
⁠
Screenshot 2022-08-10 at 17.36.22.png
⁠
⁠

⁠

Designing for flexibility

In addition to being efficient, the process also needs to be flexible. It might sound fair for everyone to spend the same number of hours interviewing each week, but that would be too rigid in many circumstances.
First, while we have a target interview load for all interviewers, we can reduce the load for individual interviewers to factor in all kinds of circumstances, ranging from their project load, working part-time, or anything that impacts how much of their time they should dedicate to interviewing.
Second, to encourage a reflection of real-world circumstances, we balance our interview load over a rather long period of 90 days. By expanding the time scale past weekly or monthly, we give our engineers the freedom to take leaves for illness, childcare, or vacation, without this creating havoc to their interview load afterward.

⁠

Stage one balancing

We use a Coda doc to manage the entire interview process and all interview information. That doc includes a table with all interviewers and the question they are assigned. In one table, we can control interview load from 0 to 100% to account for long-term circumstances why someone’s load should be reduced. In another, we track new hires and their start date and can add them as interviewers and assign their interview question. As a nice feature, the new hire table will flag if they still need to be trained, so the person running interview training can set up that training.
⁠
Screenshot 2022-08-10 at 17.39.10.png
⁠
⁠
The table to manage interviewers
⁠
Screenshot 2022-08-10 at 17.40.25.png
⁠
⁠
A table to add new hires to the interview process.
A key question at this stage is how many interviewers to assign/train for each interview question. This is where the first stage of load balancing comes in (Interviewer Assignment).
⁠
Screenshot 2022-08-22 at 16.17.42.png
⁠
⁠
We can play with hiring targets and their implications for our interviewers on this page. Critical input data here is how many interviews of each type are needed for a successful hire (as determined by the interview funnel and offer acceptance rate). Even if you do not have these numbers at hand, Coda has integrations with tools like
Greenhouse
⁠
that can help.
Factoring in the different efforts (and duration) of the interviews, our doc calculates the optimal number of interviewers for each interview question. Then, it compares the optimal number of interviewers to the current pool of interviewers to show gaps in coverage. This function lets us know how many people to train for which question.
The outcome of stage one balancing is that each pool of interviewers is the right size so that interviewers spend roughly the same time interviewing no matter which pool they are in.

⁠

Stage two balancing

In the second stage of load balancing, we need to assign the right person to the next interview that is being scheduled, taking into account who is trained for the interview and who has the least load. For that, it’s necessary to pull in accurate snapshots of our interviewers' schedules.
Our doc is doing that by using the
Google Calendar Pack
⁠
, which allows us to pull in all scheduled interviews from our interview calendar and match them to our interviewers. Like many Coda projects, this was easy to build, yet powerful. When our recruiters go to schedule an interview, they see the table below, always in sync with Google Calendar:
⁠
Screenshot 2022-08-10 at 18.50.38.png
⁠
⁠
The doc shows all interviewers grouped by interview question and sorted by who should do the next interview. Who “should” be the next interviewer is determined by their expected interview load, as well as how many interviews everyone has done in the last 90 days and how many they are scheduled to do in the next four weeks.
When a recruiter needs to schedule an interview, they invite the interviewer at the top of the list for the interview type. If that person is not available on the day needed, the recruiter picks the next interviewer on the list. This ensures load balancing over a sliding window of 90 days (factoring in future scheduled interviews). The tool also marks if someone has a massive number of interviews in the next four weeks or the last 2, allowing the recruiter to reach out before setting up more interviews.

⁠

We could have used several out-of-the-box tools to run our interview process, but we found that the flexibility of our Coda doc allowed us to build a tool that uniquely fit the team and our need for transparency, flexibility, and efficiency. If you are interested in optimizing your interviewing process, the team and I are very passionate about this area and are happy to share our doc and help you with it. Do not hesitate to reach out
Contact Coda
⁠
⁠

Want to print your doc?
This is not the way.
Try clicking the ⋯ next to your doc name or using a keyboard shortcut (
CtrlP
) instead.