What do the best interviewers have in common? We looked at thousands of real interviews to find out.

At interviewing.io, we’ve analyzed and written at some depth about what makes for a good interview from the perspective of an interviewee. However, despite the inherent power imbalance, interviewing is a two-way street. I wrote a while ago about how, in this market, recruiting isn’t about vetting as much as it is about selling, and not engaging candidates in the course of talking to them for an hour is a woefully missed opportunity. But, just like solving interview questions is a learned skill that takes time and practice, so, too, is the other side of the table. Being a good interviewer takes time and effort and a fundamental willingness to get out of autopilot and engage meaningfully with the other person.

Of course, everyone and their uncle has strong opinions about what makes someone a good interviewer, so instead of waxing philosophical, we’ll present some data and focus on analytically answering questions like… Does it matter how strong of an engineering brand your company has, for instance? Do the questions you ask actually help get candidates excited? How important is it to give good hints to your candidate? How much should you talk about yourself? And is it true that, at the end of the day, what you say is way less important than how you make people feel?¹ And so on.

Before I delve into our findings, I’ll say a few words about interviewing.io and the data we collect.

The setup

interviewing.io is an anonymous technical interviewing platform. On interviewing.io, people can practice technical interviewing anonymously, and if things go well, unlock real (still anonymous) interviews with companies like Lyft, Twitch, Quora, and more.

The cool thing is that both practice and real interviews with companies take place within the interviewing.io ecosystem. As a result, we’re able to collect quite a bit of interview data and analyze it to better understand technical interviewing. One of the most important pieces of data we collect is feedback from both the interviewer and interviewee about how they thought the interview went and what they thought of each other. If you’re curious, you can watch a real interview on our recordings page, and see what the feedback forms for interviewers and interviewees look like below — in addition to one direct yes/no question, we also ask about a few different aspects of interview performance using a 1-4 scale. We also ask interviewees some extra questions that we don’t share with their interviewers, one of which is their own take on how they thought they did.

In this post, we’ll be analyzing feedback and outcomes of thousands of real interviews with companies to figure out what traits the best interviewers have in common.

Before we get into the nitty-gritty of individual interviewer behaviors, let’s first put the value of a good interviewer in context by looking at the impact of a company’s brand on the outcome. After all, if brand matters a lot, then maybe being a good interviewer isn’t as important as we might think.

Brand strength

So, does brand really matter for interview outcomes? One quick caveat before we get into the data: every interview on the platform is user-initiated. In other words, once you unlock our jobs portal (you have to do really well in practice interviews to do so), you decide who you talk to. So, candidates talking to companies on our platform will be predisposed to move forward because they’ve chosen the company in the first place. And, as should come as no surprise to anyone, companies with a very strong brand have an easier time pulling candidates (on our platform and out in the world at large) than their lesser-known counterparts. Moreover, many of the companies we work with do have a pretty strong brand, so our pool isn’t representative of the entire branding landscape. However, all is not lost — in addition to working with very recognizable brands, we work with a number of small, up-and-coming startups, so we hope that if you, the reader, are coming from a company that’s doing cool stuff but that hasn’t yet become a household name, our findings likely apply to you. And, as you’ll see, getting candidates in the door isn’t the same as keeping them.

To try to quantify brand strength, we used three different measures: the company’s Klout Score (yes, that still exists), its Mattermark Mindshare Score, and its score on Glassdoor (under general reviews).²

When we looked at interview outcomes relative to brand strength, its impact was not statistically significant. In other words, we found that brand strength didn’t matter at all when it came to either whether the candidate wanted to move forward or how excited the candidate was to work at the company.

This was a bit surprising, so I decided to dig deeper. Maybe brand strength doesn’t matter overall but matters when the interviewer or the questions they asked aren’t highly rated? In other words, can brand buttress less-than-stellar interviewers? Not so, according to our data. Brand didn’t matter even when you corrected for interviewer quality. In fact, of the top 10 best-rated companies on our platform, half have no brand to speak of, 3 are mid-sized YC companies that command respect in Bay Area circles but are definitely not universally recognizable, and only 2 have anything approaching household name status.

So, what’s the takeaway here? Maybe the most realistic thing we can say is that while brand likely matters a lot for getting candidates in the door, once they’re in, no matter how well-branded you are, they’re yours to lose.

Choosing the question

If brand doesn’t matter once you’ve actually gotten a candidate in the door, then what does? Turns out, the questions you ask matter a TON. As you recall, feedback on interviewing.io is symmetric, which means that in addition to the interviewer rating the candidate, the candidate also rates the interviewer, and one of the things we ask candidates is how good the question(s) they got asked were.

Question quality was extremely significant (p < 0.002 with an effect size of 1.25) when it came to whether the candidate wanted to move forward with the company. This held both when candidates did well and when they did poorly.

While we obviously can’t share the best questions (these are company interviews, after all), we can look at what candidates had to say about the best and worst-rated questions on the platform.

The good

Always nice to get questions that are more than just plain algorithms.

Really good asking of a classic question, opened my mind up to edge cases and considerations that I never contemplated the couple of times I’ve been exposed to the internals of this data structure.

This was the longest interviewing.io interview I have ever done, and it is also the most enjoyable one! I really like how we started with a simple data structure and implemented algorithms on top of it. It felt like working on a simple small-scale project and was fun.

He chose an interesting and challenging interview problem that made me feel like I was learning while I was solving it. I can’t think of any improvements. He would be great to work with.

I liked the question — it takes a relatively simple algorithms problem (build and traverse a tree) and adds some depth. I also liked that the interviewer connected the problem to a real product at [Redacted] which made it feel like less like a toy problem and more like a pared-down version of a real problem.

This is my favorite question that I’ve encountered on this site. it was one of the only ones that seem like it had actual real-life applicability and was drawn from a real (or potentially real) business challenge. And it also nicely wove in challenges like complexity, efficiency, and blocking.

The bad

Question wasn’t straightforward and it required a lot of thinking/understanding since functions/data structures weren’t defined until a lot later. [Redacted] is definitely a cool company to work for, but some form of structure in interviews would have been a lot more helpful. Spent a long time figuring out what the question is even asking, and interviewer was not language-agnostic.

I was expecting a more technical/design question that showcases the ability to think about a problem. Having a domain-specific question (regex) limits the ability to show one’s problem-solving skills. I am sure with enough research one could come up with a beautiful regex expression but unless this is something one does often, I don’t think it [makes for] a very good assessment.

This is not a good general interview question. A good interview question should have more than one solution with simplified constraints.

Anatomy of a good interview question

1. Layer complexity (including asking a warmup)
2. No trivia
3. Real-world components/relevance to the work the company is doing are preferable to textbook algorithmic problems
4. If you’re asking a classic algorithmic question, that’s ok, but you ought to bring some nuance and depth to the table, and if you can teach the interviewee something interesting in the process, even better!

Asking the question

One of the other things we ask candidates after their interviews is how helpful their interviewer was in guiding them to the solution. Providing your candidate with well-timed hints that get them out of the weeds without giving away too much is a delicate art that takes a lot of practice (and a lot of repetition), but how much does it matter?

As it turns out, being able to do this well matters a ton. Being good at providing hints was extremely significant (p < 0.00001 with an effect size of 2.95) when it came to whether the candidate wanted to move forward with the company (as before, we corrected for whether the interview went well).

You can see for yourself what candidates thought of their interviewers when it came to their helpfulness and engagement below. Though this attribute is a bit harder to quantify, it seems that hint quality is actually a specific instance of something bigger, namely the notion of turning something inherently adversarial into a collaborative exercise that leaves both people in a better place than where they started.³

And if you can’t do that every time, then at the very least, be present and engaged during the interview. And no matter what the devil on your shoulder tells you, no good will ever come of opening Reddit in another tab.⁴

One of the most memorable, pithy conversations I ever had about interviewing was with a seasoned engineer who had spent years as a very senior software architect at a huge tech company before going back to what he’d always liked in the first place, writing code. He’d conducted a lot of interviews over a career spanning several decades, and after trying out a number of different interview styles, what he settled on was elegant, simple, and satisfying. According to him, the purpose of any interview is to “see if we can be smart together.” I like that so much, and it’s advice I repeat whenever anyone will listen.

The good

I liked that you laid out the structure of the interview at the outset and mentioned that the first question did not have any tricks. That helped set the pace of the interview so I didn’t spend an inordinate amount of time on the first one.

The interview wasn’t easy, but it was really fun. It felt more like making a design discussion with a colleague than an interview. I think the question was designed/prepared to fill the 45 minute slot perfectly.

I’m impressed by how quickly he identified the issue (typo) in my hash computation code and how gently he led me to locating it myself with two very high-level hints (“what other tests cases would you try?” and “would your code always work if you look for the the pattern that’s just there at the beginning of the string?”). Great job!

He never corrected me, instead asked questions and for me to elaborate in areas where I was incorrect – I very much appreciate this.

The question seemed very overwhelming at first but the interviewer was good at helping to break it down into smaller problems and suggest we focus on one of those first.

The bad

[It] was a little nerve-wracking hearing you yawn while I was coding.

What I found much more difficult about this interview was the lack of back and forth as I went along, even if it was simple affirmation that “yes, that code you just wrote looks good”. There were times when it seemed like I was the only one who had talked in the past five minutes (I’m sure that’s an exaggeration). This made it feel much more like a performance than like a collaboration, and my heart was racing at the end as a result.

While the question was very straightforward, and [he] was likely looking for me to blow through it with no prompting whatsoever in order to consider moving forward in an interview process, it would have been helpful to get a discussion or even mild hinting from him when I was obviously stuck thinking about an approach to solve the the problem. While I did get to the answer in the end, having a conversation about it would have made it feel more like a journey and learning experience. That would have also been a strong demonstration of the collaborative culture that exists while working with teams of people at a tech company, and would have sold me more vis-a-vis my excitement level.

If an interview is set to 45 minutes, the questions should fit this time frame, because people plan accordingly. I think that if you plan to have a longer interview you should notify the interviewee beforehand, so he can be ready for it.

One issue I had with the question though is what exactly he was trying to evaluate from me with the question. At points we talking about very nitty-gritty details about python linked list or array iteration, but it was unclear at any point if that was what he was judging me on. I think in the future he could outline at the beginning what exactly he was looking for with the problem in order to keep the conversation focused and ensure he is well calibrated judging candidates.

Try to be more familiar with all the possible solutions to the problem you choose to pose to the candidate. Try to work on communicating more clearly with the candidate.

Anatomy of a good interview

1. Set expectations, and control timing/pacing
2. Be engaged!
3. Familiarity with the problem and its associated rabbit holes/garden paths
4. Good balance of hints and letting candidate think
5. Turn the interview into a collaborative exercise where both people are free to be smart together

The art of storytelling… and the importance of being human

Beyond choosing and crafting good questions and being engaged (but not overbearing) during the interview, what else do top-rated interviewers have in common?

The pervasive common thread I noticed among the best interviewers on our platform is, as above, a bit hard to quantify but dovetails well with the notion of being engaged and creating a collaborative experience. It’s taking a dehumanizing process and elevating it to an organic experience between two capable, thinking humans. Many times, that translates into revealing something real about yourself and telling a story. It can be sharing a bit about the company you work at and why, out of all the places you could have landed, you ended up there. Or some aspect of the company’s mission that resonated with you specifically. Or how the projects you’ve worked on tie into your own, personal goals.

The good

I like the interview format, in particular how it was primarily a discussion about cool tech, as well as an honest description of the company… the discussion section was valuable, and may be a better gauge of fit anyway. It’s nice to see a company which places value on that 🙂

The interviewer was helpful throughout the interview. He didn’t mind any questions on their company’s internal technology decisions, or how it’s structured. I liked that the interviewer gave me a good insight of how the company functions.

Extremely kind and very generous with explaining everything they do at [redacted]. Really interested in the technical challenges they’re working on. Great!

Interesting questions but the most valuable and interesting thing were the insights he gave me about [redacted]. He sounded very passionate about engineering in general, particularly about the challenges they are facing at [redacted]. Would love to work with him.

The bad

[A] little bit of friendly banter (even if it’s just “how are you doing”?) at the very beginning of the interview would probably help a bit with keeping the candidate calm and comfortable.

I thought the interview was very impersonal, [and] I could not get a good read on the goal or mission of the company.

And, as we wrote about in a previous post, one of the most genuine, human things of all is giving people immediate, actionable feedback. As you recall, during the feedback step that happens after each interview, we ask interviewees if they’d want to work with their interviewer. As it turns out, there’s a very statistically significant relationship (p < 0.00005)⁵ between whether people think they did well and whether they’d want to work with the interviewer. This means that when people think they did poorly, they may be a lot less likely to want to work with you. And by extension, it means that in every interview cycle, some portion of interviewees are losing interest in joining your company just because they didn’t think they did well, despite the fact that they actually did.

How can one mitigate these losses? Give positive, actionable feedback immediately (or as soon as possible)! This way people don’t have time to go through the self-flagellation gauntlet that happens after a perceived poor performance, followed by the inevitable rationalization that they totally didn’t want to work there anyway.

How to be human

1. Talk about what your company does… and what specifically about it appealed to you and made you want to join
2. Talk about what you’re currently working on and how that fits in with what you’re passionate about
3. When you like a candidate, give positive feedback as quickly as you can to save them from the self-flagellation that they’ll likely go through otherwise… and which might make them rationalize away wanting to work with you
4. And, you know, be friendly. A little bit of warmth can go a long way.

Becoming a better interviewer

Interviewing people is hard. It’s hard to come up with good questions, it’s hard to give a good interview, and it’s especially hard to be human in the face of conducting a never-ending parade of interviews. But, being a good interviewer is massively important. As we saw, while your company’s brand will get people in the door, once they’ve reached the technical interview, the playing field is effectively level, and you can no longer use your brand as a crutch to mask poor questions or a lack of engagement. And in this market, where the best candidates have a ton of options, when wielded properly, a good interview that elevates a potentially cold, transactional interaction into something real and genuine can become the selling point that gets great engineers to work for you, whether you’re a household name or a startup that just got its first users.

Given how important it is to do interviews well, what are some things you can do to get better right away? One thing I found incredibly useful for coming up with good, original questions is to start a shared doc with your team where every time someone solves a problem they think is interesting, no matter how small, they jot down a quick note. These notes don’t have to be fleshed out at all, but they can be the seeds for unique interview questions that give candidates insight into the day-to-day at your company. Turning these disjointed seeds into interview questions takes thought and effort — you have to prune out a lot of the details and distill the essence of the problem into something it doesn’t take the candidate a lot of work/setup to grok, and you’ll likely have to iterate on the question a few times before you get it right — but they payoff can be huge.

Another thing you can do to get actionable feedback like the kind you saw in this post (and then immediately level up) is to get on interviewing.io as an interviewer. If you interview people in our double-blind practice pool, no one will know who you are or which company you represent, which means that you get a truly unbiased take on your interviewing ability, which includes your question quality, how excited people would be to work with you, and how good you are at helping people along without giving away too much. It’s also a great way to go beyond your team, which can be pretty awkward, and try out new questions on a very engaged, high-quality user base. You’ll also get to keep replays of your interviews so you can revisit crucial moments and figure out exactly what you need to do to get better next time.

Want to hone your skills as an interviewer? Want to help new interviewers at your company warm up before they officially get added to your interview loops? You can sign up to our platform as an interviewer, or (especially for groups) ping us at interviewers@interviewing.io.