Over the last year, I’ve been iterating my interview process as a QE manager at The Zebra. This has been my first time hiring consistently as a manager, so it’s the first time I’m really reflecting on my practices as a hiring manager and evaluating my process. I’ve written about a few things in their own blog posts where I offer more detail, but I also wanted to share a summary of the various changes I’ve made, as well as the intended mpact behind those changes.

Rejection emails

Before:

• Our job reqs include information that we don’t offer visa sponsorships, but we still get a chunk of applications indicating visa sponsorship as a need.
• The rejection email was fairly generic, and indicated that the candidate’s skills and experience were not a match for the role requirements.

After:

• I asked our recruiting team if we could create a custom template for this rejection category.
• Recruiting checked with our compliance/legal team to make sure it was acceptable to put that information in writing.
• Our recruiting team updated the email template to say that the application was rejected because the candidate indicated a need for visa sponsorship and our roles are not eligible for that. It also encourages the candidate to reply back if they checked that option by mistake.

Impact:

• Updating the template this way gives candidates honest context around the reason for being rejected.
• Often the candidate skills were a match for the role, and sending the generic email could leave a bad impression or make it seem like we don’t know what we’re doing.
• I also hope that it’s better for candidate morale, especially if they’re in the middle of a job search. Specifying that it’s because of visa requirements means we’re not adding to the “not a good fit” narrative that candidates may be hearing from other companies.

Hiring manager screens

Before:

• Purpose of these calls was to gauge whether the candidate should move forward in our hiring process.
• The format was similar to regular behavioral panel interviews where I’d ask a bunch of “tell me about a time” questions.
• I was checking off questions but didn’t feel like I was getting the meaningful signals that I wanted around candidate and role match.

After:

• I clarified my purpose and adjusted the converation format to match: to checking for a fit between role needs and candidate needs.
• On the call, I describe role requirements in more detail, share context around current and upcoming goals, and explain the criteria for success in the role.
• I ask about the candidate’s career goals, what they like about working in QE, and what they want to get out of changing jobs.

Impact:

• I have a better understanding around whether the candidate is a good fit for the role: Do they have the skills or potential to be successful in this role? Do they have the potential to be successful at this company? Do they bring perspectives or experience that the team needs?
• I have a better understanding around whether the role is a good fit for the candidate: Can I support their career growth the way they want? Can I be the kind of manager they need? Does the company offer an environment that the candidate can succeed in?

Number of interview sessions

Before:

• Recruiter screen
• Hiring manager screen
• Two behavioral panels
• Technical panel
• Wrap-up w/hiring manager and department director

After:

• Recruiter screen
• Hiring manager screen
• One behavioral panel
• Technical panel
• Wrap-up w/hiring manager and department director

Impact:

• Ease of process can be a competitive advantage in hiring these days!
• It’s less of a burden for the candidate’s schedule and asier for internal scheduling
• We’re still getting good insights on candidates without that additional behavioral panel. It hasn’t reduced the quality of feedback, and we’re able to make faster decisions.

Take-home assessment

Before:

• From the evaluation side, our rubric wasn’t great. There was some overlap and duplication between criteria, and overall it wasn’t really a good reflection of the things we actually wanted to assess.
• There was no direction around choosing a framework or technology for the assessment, which sometimes meant it was difficult for the interviewers to run through the candidate’s code and evaluate it meaningfully.

After:

• I updated the rubric to reflect the skills and experience we wanted to assess the candidate on.
• The email to candidates now included the high-level evaluation criteria in our email to the candidate.
• It also explained that the intended goal for the take-home was to see their current skills, and encouraged them to select a framework that they were already proficient in.
• We tell the candidate how interviewers will be running their test suite.

Impact:

• A clear rubric allowed us to evaluate the candidate on the skills that we defined as important for the role.
• Evaluating the assessment is easier for because the candidate can create the test suite with interviewers’ environment in mind.
• Candidates are set up for success because they know what’s important to us and what our expectations for the assessment are.
• I share additional details around this in a previous blog post: Updating Our Technical Assessment.

Introducing our panel process

Before:

• We sent a fairly generic email with the dates and times, and the names of the interviewers.
• We also included a “best practices” doc with information about the STAR method, links to research our company, and a reminder/reassurance that we’re here to get to know them, not try and trick them.

After:

• In addition to the “best practices” information, the email also introduces our overall interviewing process.
• The email offers additional context for the behavioral and technical panels.
• It also includes the question list for the candidate’s behavioral panel.

Impact:

• We’re creating a more collaborative interviewing process by setting expectations and sharing context ahead of time.
• It gives candidates the opportunity to prep and hopefully feel more comfortable participating in our process.
• I share a deeper dive around this topic in a previous blog post: More Inclusive, Less Stress: Updating Panel Interviews.

Whiteboarding interview

Before:

• We recently transitioned from take-home assessments to in-person technical interviews.
• When our interview runbook was created, we ran into many of the same challenges that I’d addressed with the take-home updates: the rubric wasn’t clear and didn’t address the skills we wanted to evaluate, and we didn’t set up enough context for candidates.

After:

• Updated the rubric to clearly ask interviewers to evaluate on critical skills.
• We made sure the flow of the interview gave candidates the opportunity to address the evaluation criteria.
• The email introducing the technical panel includes a link to the collaboration tool we use in the interview so they can get familiar with it beforehand.
• The email sets context around the whiteboarding session format and flow.
• It also includes an FAQ section to explain our goals behind it the technical interview, what we’re assessing them on, and how they can prepare for it.

Impact:

• A clear rubric means we’re assessing candidates on the criteria we care about, and also means we’re able to be consistent in our evaluation across candidates.
• “Whiteboarding” has a pretty negative connotation in the industry - it’s often assumed to mean pressurized leetcode traps. Sharing the context behind our technical interview can help reduce that pressure.
• The FAQ section sets a really collaborative tone, with the goal of showing the candidate that we want them to succeed and explaining what success looks like.

Creating change

Many of these updates are not big, sweeping changes - they’re iterations that have been made over time as I’ve learned how to do better. Some changes needed extra layers of approval, some called for working with colleagues to update, and some I decided on my own and shared with my recruiting partner to implement.

The running themes throughout all of these changes are transparency, inclusion, communication. Hiring often feels like a broken process, but it doesn’t have to be that way. Give candidates a better experience than you’ve had. Be thoughtful about your process, and reflect on how it could be improved before you need it - by that time, you’ll be too busy trying to hire to think about the process of it all!

What are some good hiring experiences you’ve had as a candidate? Hiring managers, how have you improved the hiring process at your company?