interviewing.io logo interviewing.io blog
better interviewing through data

No engineer has ever sued a company because of constructive post-interview feedback. So why don’t employers do it?

Introduction

Aline Lerner

Aline Lerner

Aline is the CEO and founder of interviewing.io.


LATEST POSTS

We ran the numbers, and there really is a pipeline problem in eng hiring. 03rd December, 2019

3 exercises to craft the kind of employer brand that actually makes engineers want to work for you 14th May, 2019

Navigation

Uncategorized

No engineer has ever sued a company because of constructive post-interview feedback. So why don’t employers do it?

Posted on .

One of the things that sucks most about technical interviews is that they’re a black box—candidates (usually) get told whether they made it to the next round, but they’re rarely told why they got the outcome that they did. Lack of feedback, or feedback that doesn’t come right away, isn’t just frustrating to candidates. It’s bad for business. We did a whole study on this. It turns out that candidates chronically underrate and overrate their technical interview performance, like so:

Where this finding starts to get actionable is that there’s a statistically significant relationship between whether people think they did well in an interview and whether they’d want to work with you. In other words, in every interview cycle, some portion of interviewees are losing interest in joining your company just because they don’t think they did well, even when they actually did. It makes sense… when you suspect you might not have done well, you’re prone to embark on a painful bout of self-flagellation, and to make it stop, you’ll rationalize away the job by telling yourself that you totally didn’t want to work there anyway.

Practically speaking, giving instant feedback to successful candidates can do wonders for increasing your close rate.

In addition to making candidates you want today more likely to join your team, feedback is crucial for the candidates you might want down the road. Technical interview outcomes are highly non-deterministic. According to our data, only about 25% of candidates perform consistently from interview to interview. Why does this matter? If interview outcomes are erratic, it means that the same candidate you reject this time might be someone you want to hire in 6 months. It’s in your interest to forge a good relationship with them now and be cognizant of and humble about the flaws in your hiring process.

I thought this tweet captured my sentiments particularly well.

Danny Trinh's tweet

So, despite the benefits, why do most companies persist in giving slow feedback or none at all? I surveyed founders, hiring managers, recruiters and labor lawyers (and also put out some questions to the Twitterverse) to understand why anyone who’s ever gone through interviewer training has been told in no uncertain terms to not give feedback.

As it turns out, feedback is discouraged primarily because companies are scared of getting sued… and because interviewers fear defensive candidate backlash. In some cases, giving feedback is avoided just because companies view it as a no-upside hassle.

The sad truth is that hiring practices have not caught up with market realities. Many of the hiring practices we take for granted today originated in a world where there was a surplus of candidates and a shortage of jobs. This extends to everything from painfully long take-home assignments to poorly written job descriptions. And post-interview feedback is no exception. As Gayle Laakmann McDowell, author of Cracking the Coding Interview, explains on Quora:

“Companies are not trying to create the most perfect process for you. They are trying to hire—ideally efficiently, cheaper, and effectively. This is about their goals, not yours. Maybe when it’s easy they’ll help you too, but really this whole process is about them… Companies do not believe it helps them to give candidates feedback. Frankly, all they see is downside.”

Look, I’m guilty of this, too. Here’s a rejection email I wrote when I was head of technical recruiting at TrialPay. This email makes me want to go back in time and punch myself in the face and then wish myself the best in my future endeavors to not get punched in the face.

Bad rejection email

These kinds of form letter rejections (which I guess is better than just leaving the person hanging) make a lot of sense when you have a revolving door of disposable candidates. They are completely irrational in this brave new world where candidates have more leverage than companies. But, because HR is fundamentally a cost center tasked with risk mitigation (rather than a profit center tasked with, you know, making stuff better), and because engineers on the ground only have so many cognitive cycles to tackle hard stuff outside their job descriptions, we continue to march forward on autopilot, perpetuating outdated and harmful practices like this one.

In this hiring climate, companies should move toward practices that give candidates a better interview experience. Is fear of litigation and discomfort legit enough to keep companies from giving feedback? Does optimizing for fear and a few bad actors in lieu of candidate experience make sense in the midst of a severe engineering shortage? Let’s break it down.

Does the fear of getting sued even make sense?

While researching this piece, I spoke to a few labor lawyers and ran some Lexis Nexis searches to see just how often a company’s constructive feedback (i.e. not “durrrr we didn’t hire you because you’re a woman”) to a rejected eng candidate has resulted in litigation.

Hey, guess what? IT’S ZERO! THIS HAS NEVER HAPPENED. EVER.1

As some of my lawyer contacts pointed out, a lot of cases get settled out of court, and that data is much harder to get. But in this market, creating poor candidate experience to hedge against something that is highly unlikely seems… irrational at best and destructive at worst.

What about candidates getting defensive?

At some point, I stopped writing trite rejection emails like the one above, but I was still beholden to my employer’s rules about written feedback.2 As an experiment, I tried giving candidates verbal feedback over the phone.

For context, I had a unique, hybrid role at TrialPay. Though my title was Head of Technical Recruiting, which meant I was accountable for normal recruiter stuff like branding and sourcing and interview process logistics, my role had one unique component. Because I had previously been a software engineer, to take the heat off the long-suffering eng team, I was often the first line of defense for technical interviews and conducted something like 500 of them that year.

After doing a lot of interviews day in and day out, I became less shy about ending them early when it was clear that a candidate wasn’t qualified (e.g. they couldn’t get through the brute force solution to the problem, let alone optimize). Did ending interviews early cause candidates to fly off the handle or feel particularly awkward, as many people suspect?

Defensive candidates tweet

In my experience, cutting things off and saying nothing about why is a lot more awkward and leads to more defensiveness than letting candidates know what the deal is. Some candidates will get defensive (at which point you can politely end the call), but if you offer constructive feedback—let them know what went wrong, make some recommendations about books to read, point them to problem repositories like Leetcode3, etc.—most will be grateful. My personal experience with giving feedback has been overwhelmingly positive. I used to love mailing books to candidates, and I formed lasting relationships with many. Some became early interviewing.io users a few years later.

Anyway, the way to avoid negative reactions and defensiveness from candidates is to practice giving feedback in a way that’s constructive. We’ll cover this next.

So if giving feedback isn’t actually risky and has real upsides, how does one do it?

When I started interviewing.io, it was the culmination of what I had started experimenting with at TrialPay. It was clear to me that feedback is a Good Thing and that candidates liked it… which in this market means it’s also good for companies. But, we still had to grapple with prospective customers’ (pretty irrational) fears about the market being flooded with defensive candidates with a lawyer on speed dial.

For context, interviewing.io is a hiring marketplace. Before talking to companies, engineers can practice technical interviewing anonymously, and if things go well, unlock our jobs portal, where they can bypass the usual top-of-funnel cruft (applying online, talking to recruiters or “talent managers,” finding friends who can refer them) and book real technical interviews with companies like Microsoft, Twitter, Coinbase, Twitch, and many others… often as early as the next day.

The cool thing is that both practice and real interviews with companies take place within the interviewing.io ecosystem, and wrt feedback, you’ll see why this matters in a moment.

Before we started working with employers, we spent some time building out our practice platform and getting the mechanics right. For practice interviews, our post-interview feedback forms looked like this:

The feedback form that an interviewer fills out

The feedback form that an interviewer fills out

After each practice interview, interviewers fill out the form above. Candidates fill out a similar form rating their interviewer. When both parties fill out their forms, they can see each other’s responses.

If you’re curious, you can watch people practicing and read real feedback that they got in our public showcase. Here’s a snapshot:

An interviewer's rubric after an interviewing.io interview

Check out our showcase. It’s cool.

When we started letting employers hire on our platform, we just recycled this post-interview feedback format, told them they should leave feedback to help us calibrate and because it’s good for candidate experience, and fervently hoped that they wouldn’t have an issue with it.

To our surprise and delight, employers were eminently willing to leave feedback. On our platform, candidates were able to see whether they passed or not and exactly how they did, just a few minutes after the interview was over, stopping the rising tide of post-interview anxiety and self-flagellation in its tracks, and, as we’ve said, increasing the likelihood that a great candidate will accept an offer.

A real, successful company interview on interviewing.io

A real, successful company interview on interviewing.io

And if a candidate failed an interview, they got to see exactly why they failed and what they needed to work on, probably for the first time ever in interview history.

A real, failed company interview on interviewing.io

A real, failed company interview on interviewing.io

Anonymity makes it easier to give feedback

On interviewing.io, interviews are anonymous: an employer knows literally nothing about the candidate before and during the interview (employers can even enable our real-time voice masking feature). Candidates’ identities are only revealed after a successful interview, i.e. after the employer has already submitted feedback.

We insist on anonymity because about 40% of our top-performing candidates are non-traditional, and we don’t want lack of pedigree or an unusual background to open them up to bias. Because interviews are anonymous, it’s often impossible to discriminate on the basis of age or gender or background. Therefore, feedback has to be constructive, by design, because the only info the interviewer has to go on is how well the candidate is performing in the interview. In addition to helping candidates get a fair shot, this anonymity provides something of a safety net for employers—it’s harder to build a discrimination case out of the feedback when the employer doesn’t know the identity of the candidate.

In many other contexts, anonymity can be destructive because of reduced accountability. But in the interview process, we’ve discovered over and over that anonymity sets free the better angels of our nature and creates a kinder, more inclusive interview experience for candidates and employers alike.

Building post-interview feedback into your eng hiring process

So, how can you fold these learnings into your process? Of course, the easiest way to get your feet wet is to start using interviewing.io. We’ll get you candidates you wouldn’t be sourcing without us, and we’ll empower you to give them the best interview experience you can.

But even if you don’t use us, based on how unlikely it is that you’re going to get sued or deal with angry candidates, we strongly recommend having your interviewers provide constructive feedback (like in the examples above) after every interview for all candidates, whether they pass or not, over email.

Here are a few strategies for delivering constructive feedback:

  1. Be clear that it’s a no-go. Ambiguity is psychologically difficult in a stressful situation. For instance: Thank you for interviewing with us. Unfortunately, you didn’t pass the interview.
  2. After you make it clear that it’s a no-go, tell them something nice. Find something about their performance—an answer they gave, or the way they thought through a problem, or how they asked the right questions—and share it with them. They’ll be more receptive to the rest of your feedback once they know that you’re on their side. For instance: Despite the fact that it didn’t work out this time, you did {x,y,z} really well, and I think that you can do much better in the future. Here are some suggestions for what you can work on.
  3. When you give suggestions, be specific and constructive. Don’t tell them that they royally screwed the whole operation and need to rethink their line of work. Instead, focus on specific things they can work on. Or, to put it another way, “Hey, familiarize yourself with big O notation. It’s not as scary as it sounds bc it comes up a lot in these kinds of interviews.”4 doesn’t say “you’re dumb and your work experience is dumb and you should feel bad” or “you seem like an asshole.” It says you should familiarize yourself with big O notation.
  4. Make recommendations. Is there a book they could read? If they’re promising but just lack knowledge, it’s a really nice gesture to ship said book to them.
  5. If you think the candidate is on their way to becoming a great engineer (especially if they take your recommendations and advice!), let them know that they can contact you again in a few months. You’ll build goodwill with someone who, even if they don’t work with you in the future, will talk about you to others. And when they do improve, you’ll be in a better position to bring them onto your team.
1If you know of such a case, please tell me, and I’ll update this post/associated content accordingly ASAP.
2This is a rule pretty much every company has, ever. It’s not just TrialPay, which was a great place to work and whose defensive HR policies, like every other company’s, were in no way indicative of their workplace culture.
3Whether algorithmic problems of the type you’d find on Leetcode are the best way to interview is a question worth asking… and I’ve since come to feel pretty strongly that many of them are not. But that’s out of scope for this piece.
4I recently discovered an amazing online book that makes big O approachable, practical, and not scary (all without talking down to the reader): Grokking Algorithms by Aditya Bhargava.

Aline Lerner

Aline Lerner

Aline is the CEO and founder of interviewing.io.

Comments
  • user

    AUTHOR Jeff massy

    Posted on 1:26 pm February 8, 2020.
    Reply

    No engineer has ever sued because for the most part no company ever gives post interview feedback.

    There is nothing but risk for the company giving feedback.

  • user

    AUTHOR Anonymous

    Posted on 11:09 am February 7, 2020.
    Reply

    Interesting post. The only company where I had a really good Post interview experience was at Facebook. The recruiter gave me itemised feedback on each session of the in-person round. He spent close to 45 mins summarizing what went wrong. He started by clearly saying that I did not pass that interview and added that the feedback could be areas I have to improve on or could just be perception issues or communication issues. He ended the conversation with a positive note explaining I can re-apply 6 months later and How I can be better prepared for it. He offered to guide me if I reach out to him after 6 months or so. I felt like this guy was on my side. And That was really good. I would recommend that process to any team that is hiring.

  • user

    AUTHOR Noah Silbert

    Posted on 11:01 am February 7, 2020.
    Reply

    This is a great post. Having recently gotten part way through a number of (data science) interviews, only to be left guessing what had gone wrong, it makes me happy to see people working to change the standard lack of feedback. Only one of the companies I interviewed with gave me direct (and constructive) feedback, and it made a huge difference to me, both with respect to how I felt (and continue to feel) about the company and with respect to the steps I have taken to improve my skillset since then.

  • user

    AUTHOR Kavita

    Posted on 9:06 pm February 6, 2020.
    Reply

    Very true. I faced this very several times recently. I felt the problem lies when the interviewer does not share the expectations for the roles that have been hiring for. So, I started asking them to share their expectations so that we can come to the same page and speak sharing the same goals. In addition, I asked several interviewers that please be open and share the feedback about where I do lack so that I can prepare for the next interviews. Unfortunately, they wish me the future endeavours.

    This is the sad truth of the interview process.

  • user

    AUTHOR Charlie Hayes

    Posted on 4:57 pm February 6, 2020.
    Reply

    I’ve received feedback similar the feedback your suggesting. In a few cases I found the feedback patronizing. I realize it’s anecdotal but I think it’s worth saying.

    Once I received feedback suggesting I read more programming books but there wasn’t a single question during the interview that was programming related. This was very upsetting. Was I supposed to ‘read the room’ differently and speak more technically? Was my feedback mixed up with someone else? What recourse do I have?

    Another time an interviewer force-fed me a solution as I was trying to explain my own. This effectively derailed my train of thought and I failed the interview. The failure had nothing to do with my skill or experience, but the feedback that I wasn’t able to solve the problem was incorrect and infuriating at the time.

    I appreciate the suggestion to be unambiguous, however I think in many cases a discussion that could result in a followup interview might make a lot more sense. For example “how do you think the interview went?” possibly with a response “I thought it went really well but this one question caught me off guard because it’s something I’ve done 15 times in front of a computer but doing it on the phone with a stranger during awkward silence didn’t work out well”. Working through a technical problem is a great exercise but it’s a two way street; maybe the fault is with the interviewer.

    At the point a candidate is rejected, there may have already been significant resources spent which could be recouped if only a candidate is given a second chance. The second chance may also go a long way to foster a positive work relationship and elicit referrals.

  • user

    AUTHOR F L

    Posted on 4:14 pm February 6, 2020.
    Reply

    Because it’s a pain in the butt to put together something that’s useful. I interviewed someone the other day, and I could tell them that “I didn’t hire you because you didn’t have the faintest idea how the internet works” but would that be helpful? Maybe. Maybe not.

  • user

    AUTHOR David

    Posted on 3:23 pm February 6, 2020.
    Reply

    After a quick search, I found this opinion from a lawsuit that was filed partly because: “[a] Google recruiter told [her] that she needed to put her dates of graduation on her resume ‘so the interviewers [could] see how old [she was].'”

    https://www.leagle.com/decision/inadvfdco170904000864

  • user

    AUTHOR Rajesh

    Posted on 3:02 pm February 6, 2020.
    Reply

    That’s nice case study. Employers must provide feedback for sure.

    Also, a place central place where candidate/interviewer can track his/her progress, HR can track progress from start to end for all conversations, questions, answers, and feedback.

    That would put pressure on making right decisions from both side.

  • user

    AUTHOR corey math

    Posted on 2:21 pm February 6, 2020.
    Reply

    Your statement contradict itself. If companies don’t do it then how can people she them? Maybe, if companies started doing it then we might actually see cases where people due sue. You can’t premeditated sue.

  • user

    AUTHOR Guy

    Posted on 10:06 am February 6, 2020.
    Reply

    In my opinion, the reasons companies don’t give feedback has nothing to do with the worry of being sued. That’s just the easy excuse. The main reason has to do primarily with HR/Recruiting. Recruiters, like salespersons, are compensated by their offer ratio. The more candidates who sign, the more compensated they are. If they were compensated for “Candidate experience” things would have been different. In such a case, the recruiter would proactively ask the interview panel to send them constructive feedback so that they can communicate it back to the candidate. This is even more acute with large and very sought after companies, where they deliberately hire many recruiters to engage with as many candidates as possible to bring into the funnel, with the expectation that only a few of those candidates will make it into the offer stage. Recruiters in these companies have zero incentives to provide feedback, as that candidate is no longer a viable option and will not be for at least a year or more.

  • user

    AUTHOR Agnieszka

    Posted on 9:24 am February 6, 2020.
    Reply

    The title is not true. I know an interviewer that was sued. It was in Poland, not US, but it can happen

  • View Comments (17) ...