Why AI can’t do hiring

The recent exciting and somewhat horrifying inflection point in AI capability tipped me into writing this blog post.

I simply don't believe that AI can do hiring. My argument isn't about bias (though bias is a real problem) or that it's technologically impossible. It's just that the training data simply isn't available.

Most people believe that if you can somehow combine what's available on LinkedIn, GitHub, and the social graph (who follows whom on Twitter etc.), you'll be able to find the good engineers who are actively looking and also figure out what they want. This is wrong. None of those 3 sources are actually useful. At the end of the day, you can’t use AI for hiring if you don’t have the data. And if you have the data, then you don’t strictly need AI.

We have the best technical interviewers on the market. Here's how we do it.

interviewing.io is an anonymous mock interview platform and eng hiring marketplace. We make money in two ways: engineers pay us for mock interviews, and employers pay us for access to the best performers. This means that we live and die by the quality of our interviewers in a way that no single employer does – if we don’t have really well-calibrated interviewers, who also create great candidate experience, we don’t get paid.

In a recent post, we shared how, over time, we came up with two metrics that, together, tell a complete and compelling story about interviewer quality: the candidate experience metric and the calibration metric. In this post, we’ll talk about how to apply our learnings about interviewer quality to your own process. We’ve made a bunch of mistakes so you don’t have to! It boils down to choosing the right people, tracking those 2 metrics diligently, rewarding good behavior, and committing to providing feedback to your candidates.

Why giving feedback (whether it’s good or bad) will help you hire

Giving feedback will not only make candidates you want today more likely to join your team, but it’s also crucial to hiring the candidates you might want down the road. Technical interview outcomes are erratic, and according to our data, only about 25% of candidates perform consistently from interview to interview.

Our business depends on having the best interviewers, so we built an interviewer rating system. And you can too.

interviewing.io is an anonymous mock interview platform and eng hiring marketplace. Engineers use us for mock interviews, and we use the data from those interviews to surface top performers, in a much fairer and more predictive way than a resume. If you’re a top performer on interviewing.io, we fast-track you at the world’s best companies.

We make money in two ways: engineers pay us for mock interviews, and employers pay us for access to the best performers. To keep our engineer customers happy, we have to make sure that our interviewers deliver value to them by conducting realistic mock interviews and giving useful, actionable feedback afterwards. To keep our employer customers happy, we have to make sure that the engineers we send them are way better than the ones they’re getting without us. Otherwise, it’s just not worth it for them.

This means that we live and die by the quality of our interviewers, in a way that no single employer does, no matter how much they say they care about people analytics or interviewer metrics or training. If we don’t have really well-calibrated interviewers, who also create great candidate experience, we don’t get paid.

In this post, we’ll explain exactly how we compute and use these metrics to get the best work out of our interviewers.

Hamtips, or why I still run the Technical Phone Screen as the Hiring Manager

“Hamtips” stands for “Hiring Manager Technical Phone Screen.” This combines two calls: the Technical Phone Screen (TPS), which is a coding exercise that usually happens before the onsite, and the HMS call, which is a call with the Hiring Manager. By combining these two steps you shorten the intro-to-offer by ~1 week and reduce candidate dropoff by 5-10%. It’s also a lot less work for recruiters playing scheduling battleship. Finally, Hiring Managers will, on average, be better at selling working at the company – it’s kind of their job.

How to write (actually) good job descriptions

When you start writing a job description, the first question you should ask yourself is, Am I trying to attract the right people, or am I trying to keep the wrong people out? Then, once you answer it, write for that audience deliberately, because it’s really hard to write for both…

The technical interview practice gap, and how it keeps underrepresented groups out of software engineering

I’ve been hiring engineers in some capacity for the past decade, and five years ago I founded interviewing.io, a technical recruiting marketplace that provides engineers with anonymous mock interviews and then fast-tracks top performers—regardless of who they are or how they look on paper—at top companies. We’ve hosted close to 100K technical interviews on our platform and have helped thousands of engineers find jobs. Since last year, we’ve also been running a Fellowship program specifically for engineers from underrepresented backgrounds. All that is to say that even though I have strong opinions about “diversity hiring” initiatives, I’ve acquired them the honest way, through laboratory experience.

Technical phone screen superforecasters

“The new VP wants us to double engineering’s headcount in the next six months. If we have a chance in hell to hit the hiring target, you seriously need to reconsider how fussy you’ve become.”

It’s never good to have a recruiter ask engineers to lower their hiring bar, but he had a point. It can take upwards of 100 engineering hours to hire a single candidate, and we had over 50 engineers to hire. Even with the majority of the team chipping in, engineers would often spend multiple hours a week in interviews. Folks began to complain about interview burnout. Also, fewer people were actually getting offers; the onsite pass rate had fallen by almost a third, from ~40% to under 30%. This meant we needed even more interviews for every hire. Visnu and I were early engineers bothered most by the state of our hiring process. We dug in. Within a few months, the onsite pass rate went back up, and interviewing burnout receded. We didn’t lower the hiring bar, though. There was a better way.

No engineer has ever sued a company because of constructive post-interview feedback. So why don’t employers do it?

One of the things that sucks most about technical interviews is that they’re a black box—candidates (usually) get told whether they made it to the next round, but they’re rarely told why they got the outcome that they did. Lack of feedback, or feedback that doesn’t come right away, isn’t just frustrating to candidates. It’s bad for business. We did a whole study on this. It turns out that candidates chronically underrate and overrate their technical interview performance, like so: Where this finding starts to get actionable is that there’s a statistically significant relationship between whether people think they did well in an interview and whether they’d want to work with you. In other words, …

3 exercises to craft the kind of employer brand that actually makes engineers want to work for you

If I’m honest, I’ve wanted to write something about employer brand for a long time. One of the things that really gets my goat is when companies build employer brand by over-indexing on banalities (“look we have a ping pong table!”, “look we’re a startup so you’ll have a huge impact”, etc.) instead of focusing on the narratives that make them special. Hiring engineers is really hard. It’s hard for tech giants, and it’s hard for small companies… but it’s especially hard for small companies people haven’t quite heard of, and they can use all the help they can get because talking about impact and ping pong tables just doesn’t cut it anymore. At interviewing.io, …

You probably don’t factor in engineering time when calculating cost per hire. Here’s why you really should.

Whether you’re a recruiter yourself or an engineer who’s involved in hiring, you’ve probably heard of the following two recruiting-related metrics: time to hire and cost per hire. Indeed, these are THE two metrics that any self-respecting recruiting team will track. Time to hire is important because it lets you plan — if a given role has historically taken 3 months to fill, you’re going to act differently when you need to fill it again than if it takes 2 weeks. And, traditionally, cost per hire has been a planning tool as well — if you’re setting recruiting budgets for next year and have a headcount in mind, seeing what recruiting spent last year is …

What do the best interviewers have in common? We looked at thousands of real interviews to find out.

At interviewing.io, we’ve analyzed and written at some depth about what makes for a good interview from the perspective of an interviewee. However, despite the inherent power imbalance, interviewing is a two-way street. I wrote a while ago about how, in this market, recruiting isn’t about vetting as much as it is about selling, and not engaging candidates in the course of talking to them for an hour is a woefully missed opportunity. But, just like solving interview questions is a learned skill that takes time and practice, so, too, is the other side of the table. Being a good interviewer takes time and effort and a fundamental willingness to get out of autopilot and …

People are still bad at gauging their own interview performance. Here’s the data.

interviewing.io is an anonymous technical interviewing platform. We started it because resumes suck and because we believe that anyone, regardless of how they look on paper, should have the opportunity to prove their mettle. In the past few months, we’ve amassed over 600 technical interviews along with their associated data and metadata. Interview questions tend to fall into the category of what you’d encounter at a phone screen for a back-end software engineering role at a top company, and interviewers typically come from a mix of larger companies like Google, Facebook, and Twitter, as well as engineering-focused startups like Asana, Mattermark, KeepSafe, and more. Over the course of the next few posts, we’ll be sharing some …

A founder’s guide to making your first recruiting hire

Recently, a number of founder friends have asked me about how to approach their first recruiting hire, and I’ve found myself repeating the same stuff over and over again. Below are some of my most salient thoughts on the subject. Note that I’ll be talking a lot about engineering hiring because that’s what I know, but I expect a lot of this applies to other fields as well, especially ones where the demand for labor outstrips supply. Don’t get caught up by flashy employment history; hustle trumps brands At first glance, hiring someone who’s done recruiting for highly successful tech giants seems like a no-brainer. Google and Facebook are good at hiring great engineers, right? …

Engineers can't gauge their own interview performance. And that makes them harder to hire.

interviewing.io is an anonymous technical interviewing platform. We started it because resumes suck and because we believe that anyone regardless of how they look on paper, should have the opportunity to prove their mettle. In the past few months, we’ve amassed over 600 technical interviews along with their associated data and metadata. Interview questions tend to fall into the category of what you’d encounter at a phone screen for a back-end software engineering role at a top company, and interviewers typically come from a mix...