interviewing.io logo interviewing.io blog
better interviewing through data

We built voice modulation to mask gender in technical interviews. Here’s what happened.

Introduction

Aline Lerner

Aline Lerner

Aline is the CEO and founder of interviewing.io.


LATEST POSTS

We analyzed thousands of technical interviews on everything from language to code style. Here’s what we found. 13th June, 2017

LinkedIn endorsements are dumb. Here’s the data. 27th February, 2017

Navigation

Uncategorized

We built voice modulation to mask gender in technical interviews. Here’s what happened.

Posted on .

interviewing.io is a platform where people can practice technical interviewing anonymously and, in the process, find jobs based on their interview performance rather than their resumes. Since we started, we’ve amassed data from thousands of technical interviews, and in this blog, we routinely share some of the surprising stuff we’ve learned. In this post, I’ll talk about what happened when we built real-time voice masking to investigate the magnitude of bias against women in technical interviews. In short, we made men sound like women and women sound like men and looked at how that affected their interview performance. We also looked at what happened when women did poorly in interviews, how drastically that differed from men’s behavior, and why that difference matters for the thorny issue of the gender gap in tech.

The setup

When an interviewer and an interviewee match on our platform, they meet in a collaborative coding environment with voice, text chat, and a whiteboard and jump right into a technical question. Interview questions on the platform tend to fall into the category of what you’d encounter at a phone screen for a back-end software engineering role, and interviewers typically come from a mix of large companies like Google, Facebook, Twitch, and Yelp, as well as engineering-focused startups like Asana, Mattermark, and others.

After every interview, interviewers rate interviewees on a few different dimensions.

Feedback form for interviewers

Feedback form for interviewers

As you can see, we ask the interviewer if they would advance their interviewee to the next round. We also ask about a few different aspects of interview performance using a 1-4 scale. On our platform, a score of 3 or above is generally considered good.

Women historically haven’t performed as well as men…

One of the big motivators to think about voice masking was the increasingly uncomfortable disparity in interview performance on the platform between men and women1. At that time, we had amassed over a thousand interviews with enough data to do some comparisons and were surprised to discover that women really were doing worse. Specifically, men were getting advanced to the next round 1.4 times more often than women. Interviewee technical score wasn’t faring that well either — men on the platform had an average technical score of 3 out of 4, as compared to a 2.5 out of 4 for women.

Despite these numbers, it was really difficult for me to believe that women were just somehow worse at computers, so when some of our customers asked us to build voice masking to see if that would make a difference in the conversion rates of female candidates, we didn’t need much convincing.

… so we built voice masking

Since we started working on interviewing.io, in order to achieve true interviewee anonymity, we knew that hiding gender would be something we’d have to deal with eventually but put it off for a while because it wasn’t technically trivial to build a real-time voice modulator. Some early ideas included sending female users a Bane mask.

Early voice masking prototype

Early voice masking prototype (drawing by Marcin Kanclerz)

When the Bane mask thing didn’t work out, we decided we ought to build something within the app, and if you play the videos below, you can get an idea of what voice masking on interviewing.io sounds like. In the first one, I’m talking in my normal voice.

And in the second one, I’m modulated to sound like a man.2

Armed with the ability to hide gender during technical interviews, we were eager to see what the hell was going on and get some insight into why women were consistently underperforming.

The experiment

The setup for our experiment was simple. Every Tuesday evening at 7 PM Pacific, interviewing.io hosts what we call practice rounds. In these practice rounds, anyone with an account can show up, get matched with an interviewer, and go to town. And during a few of these rounds, we decided to see what would happen to interviewees’ performance when we started messing with their perceived genders.

In the spirit of not giving away what we were doing and potentially compromising the experiment, we told both interviewees and interviewers that we were slowly rolling out our new voice masking feature and that they could opt in or out of helping us test it out. Most people opted in, and we informed interviewees that their voice might be masked during a given round and asked them to refrain from sharing their gender with their interviewers. For interviewers, we simply told them that interviewee voices might sound a bit processed.

We ended up with 234 total interviews (roughly 2/3 male and 1/3 female interviewees), which fell into one of three categories:

  • Completely unmodulated (useful as a baseline)
  • Modulated without pitch change
  • Modulated with pitch change

You might ask why we included the second condition, i.e. modulated interviews that didn’t change the interviewee’s pitch. As you probably noticed, if you played the videos above, the modulated one sounds fairly processed. The last thing we wanted was for interviewers to assume that any processed-sounding interviewee must summarily have been the opposite gender of what they sounded like. So we threw that condition in as a further control.

The results

After running the experiment, we ended up with some rather surprising results. Contrary to what we expected (and probably contrary to what you expected as well!), masking gender had no effect on interview performance with respect to any of the scoring criteria (would advance to next round, technical ability, problem solving ability). If anything, we started to notice some trends in the opposite direction of what we expected: for technical ability, it appeared that men who were modulated to sound like women did a bit better than unmodulated men and that women who were modulated to sound like men did a bit worse than unmodulated women. Though these trends weren’t statistically significant, I am mentioning them because they were unexpected and definitely something to watch for as we collect more data.

On the subject of sample size, we have no delusions that this is the be-all and end-all of pronouncements on the subject of gender and interview performance. We’ll continue to monitor the data as we collect more of it, and it’s very possible that as we do, everything we’ve found will be overturned. I will say, though, that had there been any staggering gender bias on the platform, with a few hundred data points, we would have gotten some kind of result. So that, at least, was encouraging.

So if there’s no systemic bias, why are women performing worse?

After the experiment was over, I was left scratching my head. If the issue wasn’t interviewer bias, what could it be? I went back and looked at the seniority levels of men vs. women on the platform as well as the kind of work they were doing in their current jobs, and neither of those factors seemed to differ significantly between groups. But there was one nagging thing in the back of my mind. I spend a lot of my time poring over interview data, and I had noticed something peculiar when observing the behavior of female interviewees. Anecdotally, it seemed like women were leaving the platform a lot more often than men. So I ran the numbers.

What I learned was pretty shocking. As it happens, women leave interviewing.io roughly 7 times as often as men after they do badly in an interview. And the numbers for two bad interviews aren’t much better. You can see the breakdown of attrition by gender below (the differences between men and women are indeed statistically significant with P < 0.00001).

Also note that as much as possible, I corrected for people leaving the platform because they found a job (practicing interviewing isn’t that fun after all, so you’re probably only going to do it if you’re still looking), were just trying out the platform out of curiosity, or they didn’t like something else about their interviewing.io experience.

A totally speculative thought experiment

So, if these are the kinds of behaviors that happen in the interviewing.io microcosm, how much is applicable to the broader world of software engineering? Please bear with me as I wax hypothetical and try to extrapolate what we’ve seen here to our industry at large. And also, please know that what follows is very speculative, based on not that much data, and could be totally wrong… but you gotta start somewhere.

If you consider the attrition data points above, you might want to do what any reasonable person would do in the face of an existential or moral quandary, i.e. fit the data to a curve. An exponential decay curve seemed reasonable for attrition behavior, and you can see what I came up with below. The x-axis is the number of what I like to call “attrition events”, namely things that might happen to you over the course of your computer science studies and subsequent career that might make you want to quit. The y-axis is what portion of people are left after each attrition event. The red curve denotes women, and the blue curve denotes men.

Now, as I said, this is pretty speculative, but it really got me thinking about what these curves might mean in the broader context of women in computer science. How many “attrition events” does one encounter between primary and secondary education and entering a collegiate program in CS and then starting to embark on a career? So, I don’t know, let’s say there are 8 of these events between getting into programming and looking around for a job. If that’s true, then we need 3 times as many women studying computer science than men to get to the same number in our pipelines. Note that that’s 3 times more than men, not 3 times more than there are now. If we think about how many there are now, which, depending on your source, is between 1/3 and a 1/4 of the number of men, to get to pipeline parity, we actually have to increase the number of women studying computer science by an entire order of magnitude.

Prior art, or why maybe this isn’t so nuts after all

Since gathering these findings and starting to talk about them a bit in the community, I began to realize that there was some supremely interesting academic work being done on gender differences around self-perception, confidence, and performance. Some of the work below found slightly different trends than we did, but it’s clear that anyone attempting to answer the question of the gender gap in tech would be remiss in not considering the effects of confidence and self-perception in addition to the more salient matter of bias.

In a study investigating the effects of perceived performance to likelihood of subsequent engagement, Dunning (of Dunning-Kruger fame) and Ehrlinger administered a scientific reasoning test to male and female undergrads and then asked them how they did. Not surprisingly, though there was no difference in performance between genders, women underrated their own performance more often than men. Afterwards, participants were asked whether they’d like to enter a Science Jeopardy contest on campus in which they could win cash prizes. Again, women were significantly less likely to participate, with participation likelihood being directly correlated with self-perception rather than actual performance.3

In a different study, sociologists followed a number of male and female STEM students over the course of their college careers via diary entries authored by the students. One prevailing trend that emerged immediately was the difference between how men and women handled the “discovery of their [place in the] pecking order of talent, an initiation that is typical of socialization across the professions.” For women, realizing that they may no longer be at the top of the class and that there were others who were performing better, “the experience [triggered] a more fundamental doubt about their abilities to master the technical constructs of engineering expertise [than men].”

And of course, what survey of gender difference research would be complete without an allusion to the wretched annals of dating? When I told the interviewing.io team about the disparity in attrition between genders, the resounding response was along the lines of, “Well, yeah. Just think about dating from a man’s perspective.” Indeed, a study published in the Archives of Sexual Behavior confirms that men treat rejection in dating very differently than women, even going so far as to say that men “reported they would experience a more positive than negative affective response after… being sexually rejected.”

Maybe tying coding to sex is a bit tenuous, but, as they say, programming is like sex — one mistake and you have to support it for the rest of your life.

Why I’m not depressed by our results and why you shouldn’t be either

Prior art aside, I would like to leave off on a high note. I mentioned earlier that men are doing a lot better on the platform than women, but here’s the startling thing. Once you factor out interview data from both men and women who quit after one or two bad interviews, the disparity goes away entirely. So while the attrition numbers aren’t great, I’m massively encouraged by the fact that at least in these findings, it’s not about systemic bias against women or women being bad at computers or whatever. Rather, it’s about women being bad at dusting themselves off after failing, which, despite everything, is probably a lot easier to fix.

1Roughly 15% of our users are female. We want way more, but it’s a start.

2If you want to hear more examples of voice modulation or are just generously down to indulge me in some shameless bragging, we got to demo it on NPR and in Fast Company.

3In addition to asking interviewers how interviewees did, we also ask interviewees to rate themselves. After reading the Dunning and Ehrlinger study, we went back and checked to see what role self-perception played in attrition. In our case, the answer is, I’m afraid, TBD, as we’re going to need more self-ratings to say anything conclusive.

Aline Lerner

Aline Lerner

Aline is the CEO and founder of interviewing.io.

Comments
  • user

    AUTHOR Darwin

    Posted on 1:36 pm May 23, 2017.
    Reply

    Men and women are of course not “identical”… you have a mean I.Q difference of about 5 I.Q points…

    Egalitarism is probably the most powerfull ideology in the West today.

    And if you mask names and voices, you will still have a large mean performance difference between let’s say europeans and africans… no little marxists, not because of “racism and stigmatisation”, just because africans, for genetic reasons, have a smaller brain (on a mean) with a thinner and less circonvoluted cortex, and their mean I.Q is lower.

    Oooh… big surprise… everybody is not identical ??? ^^

    • user

      AUTHOR Lucia

      Posted on 11:42 pm August 8, 2017.
      Reply

      Actually the new studies by IQ expert Lynn show women to be outscoring men on IQ test now, soooo…you can scratch that one “Darwin.”

  • user

    AUTHOR Derp

    Posted on 5:00 am August 18, 2016.
    Reply

    Study is flawed, you didn’t gender neutralize the names. all you succeeded in doing was making everyone who was interviewing uncomfortable that they were talking to a boy named sue.

    • user

      AUTHOR Balthasar

      Posted on 10:15 am November 30, 2016.
      Reply

      Interviews are fully anonymous, voice was the only thing to identify person.

    • user

      AUTHOR Saintly Scombridae

      Posted on 6:26 am August 11, 2017.
      Reply

      Incorrect.

  • user

    AUTHOR AnnieV

    Posted on 11:24 am July 22, 2016.
    Reply

    Have you studied the transcripts? Do women’s speech patterns, despite voice modulation, have less authoritative sentence structure, more hedging, less definitive word choices? You can give the same answer, but do it in ways that range from sounding like you’re guessing to sounding like you think the questioner was stupid for even posing such an unworthy query for your talents.

    • user

      AUTHOR Needtoheariy

      Posted on 8:02 pm August 14, 2016.
      Reply

      Why is it so offensive or unacceptable that women are just not as interested in programming. They arent raised to be less curious in the US. When i was a teen in the 90’s lots of guys were taking pc’s apart and making little apps to do annoying things on aol. Because thats how we are. Women used aol just as much to socialize but never met one that was pirating softeare or learning to hack or taking apart hardware and our fathers didnt encoursge it since it wad totally alien to them! If anything they saw it as “faggy” and too intellectual but to us it was the same as them working on a car and enjoying it. Mem are often actually more skilled at development because it requires massive amounts of time and sacrifice, isolation, and the interest to do it as a hobby at first. And its not a matter of intelligence, its a matter of interest leading to practice leading to skill. There are some amazing females in code and they often agree that to allow unskilled people in the field because of gender gap is insulting to us all and the intense pride we have in our work. The reason tech is at the forefront of the job market is because the ppl involved are beyond passionate and obsessed with the work. If you bring identity politics into it, you will corrupt the best thing the modern world has going for it and it will hurt many other political goals by disenfranchising those who spent endless sleepless nights doing this stuff and could have written encrypted chat for revolutionary groups or whatever. This needs to remain a merit based field. We do not need more women in any way shape or form. If she has the skills, let her in. No other way is acceptable and the backlash will be huge. Calling us devout coders sexist wont justify a quota. And unless someone wants to do this difficult and stressful work, do not encourage them to. You might ruin their lives by forcing them into a field that consumes your entire mind at times and not just a few hours while clocked in. I think the only sexist thing i can see is wanting more of your tribe in a respected field for no reason but their gender. Its tribal and stupid and diversity is only helpful intellectually. Females nor males can add anything because of their genitals. Let coders code and dont piss off a very passionate subculture which insiders only can even understand anyway! If you dont write code, do not talk about code or what the dev world needs. We need to be left to ourselves to keep making innovative software and connecting the world. If we need something well let you know

      • user

        AUTHOR RJ Farnor

        Posted on 1:07 pm October 28, 2016.
        Reply

        Before there were home PCs there were approximately equal numbers of male and female hackers. Home PCs were marketed to boys because toy and game manufacturers always marketed separately to boys and girls. Within 5 years male undergraduates were coming to college with more skills and leaving female students unable to compete on a level playing field. The discrepancy dates from sexist decisions made 25 years age and not inherently different skills.

        http://www.npr.org/sections/money/2014/10/21/357629765/when-women-stopped-coding

        • user

          AUTHOR Maverick

          Posted on 7:10 pm January 15, 2017.
          Reply

          You just compared women who attend college for computer science for programming in the 1980s (including introductory courses, in which MANY people tend to drop out, many males included), to being “hackers.”

          There were more women who attended computer science classes in the 80s, back when it was the advent of programming careers and when personal computers were new.

          There generally were no biases against women about the matter of programming jobs, because it wasn’t a wide interest. Computer jobs were un-sexy, compared to today’s geek-chic era, and it was all a new frontier, with relatively fewer people even qualified for tech back then, compared to today’s rife “interest” in tech.

          One of the most prominent computer scientists even back then was a woman–a seasoned pioneer by the name of Grace Hopper, who worked for the military and helped advance computer programming (also coined the term “bug”).

          What happened is that as the industry emerged, and the tech industries started to see more male pioneers who spent years searching for success (you might have heard of companies like Microsoft…took them 12 years to finally become a success), and it became clear that this thing was taking off fast, and would need people who can invest the most energy and interest in the industry, males proved to be most of those who remained with it.

          Creations like Microsoft and Apple weren’t created to spite women. They were created by curious and pioneering men, which inspired tons of other males. Same’s true with gaming. It wasn’t established to spite women. It was generated by those who created and contributed–and they tended to be men.

          Men tended learned this stuff as self-motivated, curious teen boys, who spent much of their youth creating primitive homebrew D&D computer games and hacking minor systems for cheap thrills, often getting years of early experience by the time they entered college.

          Most women, on the other hand, tended to get their first taste of computer science while in college, being young and looking at the prospects of computer science, and you saw waves of young women entering introductory courses. But soon, many found out that this industry wasn’t for them-particularly during its unsexy infancy.

          Per every introductory computer science course, you’ll find plenty of men and women alike dropping out of the introductory course after a while, realize that it’s just not for them, or finding something else of stronger interest to them. The majority of those who stay tend to be male, and often the ones who got a head start as kids.

          When I was 13, I had already learned 2 languages, including the “difficult” C++. By the time I was an adult, I was already well-ahead of those just getting their first taste in college. My dad’s a programmer, but I learned on my own accord, since he was rarely home enough to teach me. Just like Bill Gates and Steve Woz, I got started young, based on my own draw to the field by passion–and NOT as some look at what could be a good job.

          Women waiting around to be “inspired” to work in this field are waiting in vain. Whether male or female, if you’re not personally drawn to the work, it WILL be a chore for you, or you WILL become one of those job-for-job’s-sake programmers that nobody likes to work with because your work’s noticeably lazier.

          It just turns out that, for whatever general reasons, males seem to exhibit earlier passion for this stuff. I find this is a VERY common back story for most guys my age, in my line of work.

          You’ll be much harder pressed to find young women with similar backgrounds–though, I’ve met a few. Just a few, though. Even then, they tended to be second-generation programmers who likewise took around computers and did homebrew, like myself.

      • user

        AUTHOR Garrett

        Posted on 12:00 am November 28, 2016.
        Reply

        I counter your anecdotes with my own: Growing up I had a number of female friends who were eager software pirates, several who built their own PCs, and one in particular who was an absolute genius with a TI-85 calculator and had a postdoc-level grasp of advanced mathematics and information theory while she was still in high school.

        That girl in particular went to CalTech, then burned out for a while and went into child development, then turned a 180 again and became a driver-level network optimization engineer for Facebook, and her career has moved up from there. This is not because of the happy and welcoming cadre of brogrammers that preceded her … but more IN SPITE of their obsessive and overeager attention due to the fact that she was female.

        If you think nothing needs to change in that environment for ALL real computer geeks to participate, you have your head in the sand.

      • user

        AUTHOR RationalistFaith

        Posted on 9:37 am November 30, 2016.
        Reply

        How dare you use logic and suggest that women’s 1-2 weeks PMS’ing and their periods that affect their brain from thinking logically is an ISSUE?!!! /s

      • user

        AUTHOR Robmar

        Posted on 10:23 am August 10, 2017.
        Reply

        Well put, bravo 👏 .

  • user

    AUTHOR Protest Manager

    Posted on 6:28 pm July 7, 2016.
    Reply

    Hmm.
    “Specifically, men were getting advanced to the next round 1.4 times more often than women. Interviewee technical score wasn’t faring that well either — men on the platform had an average technical score of 3 out of 4, as compared to a 2.5 out of 4 for women.”

    “masking gender had no effect on interview performance with respect to any of the scoring criteria (would advance to next round, technical ability, problem solving ability). If anything, we started to notice some trends in the opposite direction of what we expected: for technical ability, it appeared that men who were modulated to sound like women did a bit better than unmodulated men and that women who were modulated to sound like men did a bit worse than unmodulated women. Though these trends weren’t statistically significant, I am mentioning them because they were unexpected and definitely something to watch for as we collect more data.”

    “As it happens, women leave interviewing.io roughly 7 times as often as men after they do badly in an interview.”

    “Once you factor out interview data from both men and women who quit after one or two bad interviews, the disparity goes away entirely”

    But we’re told that women drop out at 7x the rate of men after a poor interview. So to translate your sentence into something realistic: “After we got rid of a significant number of poorer performing women, and almost none of the poorer performing men, what was left was a pool of people who performed roughly the same.”

    Let me express it for you in mathematical terms:

    The standard deviation between women is less than the standard deviation between men. This means that in any “pool” where entry requires above average performance on characteristics whereon average men do at least as well as women do (and intelligence is such a case), you will find more men at the positive extremes than women. So in any environment where honest evaluation takes place, men will do better than women, and will succeed more often than women.

    Now, you might want to claim that it’s better to have more “women in tech”, even if that means having dumber, less qualified programmers who write worse code. And if you want to make that argument, I’ll be happy to listen and respond.

    But can we please drop the fantasy that there aren’t “enough” women in STEM because of sexism?

    • user

      AUTHOR Jprusakova

      Posted on 9:59 am July 10, 2016.
      Reply

      There seems to be a flaw in your analysis. The people dropping out after one unsuccessful interview aren’t “the poorer performing people”. This is a group that drops out most easily – which may or may not have any correlation with their quality as engineers, professionals, or even their ability to write decent code.

      • user

        AUTHOR Ghoon

        Posted on 11:08 am July 27, 2016.
        Reply

        One might claim that perseverance is an important trait of a good software engineer
        One might also suspect that perseverance is correlated with intelligence

        • user

          AUTHOR AnnieV

          Posted on 3:54 pm July 29, 2016.
          Reply

          The definition of insanity is doing the same thing over and over again, but expecting different results. – Albert Einstein.

          • user

            AUTHOR karen straughan

            Posted on 4:12 pm July 29, 2016.

            “The definition of insanity is doing the same thing over and over again, but expecting different results. – Albert Einstein.”

            I lifted weights for a whole 15 minutes, and didn’t look like Jason Statham by the end of it! What a rip off!

            Correct me if I’m wrong, but people coming to this site do so because they’re applying for (or soon will be) jobs in their field. They want to get feedback on their performance in a typical interview *before* they risk poisoning any major wells.

            Practice is not insanity. The whole point of this exercise is to find out what you are and are not doing right, and work on improving your performance. The point is NOT to keep doing the same thing that doesn’t get you hired, but to find out what you need to improve on.

            Women are more likely to give up on the process after 1 or 2 bad performances than men are. Men are more willing to stick with the process of improving their performance in preparation for real life interviews.

            That’s not insanity. That’s how you get a job.

          • user

            AUTHOR AnnieV

            Posted on 8:08 pm July 29, 2016.

            Karen, my reply was very specifically directed at the comment “One might also suspect that perseverance is correlated with intelligence,” which insinuates since women don’t persevere in this exercise, that they are not intelligent.

            Of course interview practice can help get you a job. Asserting that a measure of how many times someone comes *here* to practice (perseverance) is a substitute measure of intelligence is not a warranted assumption.

          • user

            AUTHOR Ryan

            Posted on 4:08 pm November 16, 2016.

            That’s actually not remotely the definition of insanity and Einstein never said it. That is all.

  • user

    AUTHOR Curious

    Posted on 11:15 pm July 6, 2016.
    Reply

    Did you factor in the interviewees’ experience with interviewing.io, whether for the women it was a ‘last resort’ in their job search versus a ‘first resort’ for the men? That is, for the female interviewees, was this interview among the last in a long line of interviews, whereas for the male interviewees, was it among the first, if not the first, interview? I wondered if the tendency to use such a platform has this phenomenon as a difference between men and women in the job search process.

  • user

    AUTHOR Bethany

    Posted on 8:52 am July 6, 2016.
    Reply

    One factor that you might look into, especially when it comes to attrition events, is how people respond to rejection (or perceived rejection), and also, how you inoculate (i.e. prepare) people for technical interviews. I’ve never held a purely technical position, but when I did a technical interview, I thought that I completely bombed it. What I learned later was that it was an interview designed to test my limits and how I react when I don’t know something, rather than an interview designed to test my knowledge. I later learned that the interviewer was *really* impressed with my interview, to the extent that she offered to recommend me for other similar jobs in the future (turns out I was overqualified for the position).
    I’m guessing that by telling people you might be masking/manipulating voices, people probably eliminated any bias that would normally be there. Did you compare the data from your study to your normal gender breakdown/data?

    • user

      AUTHOR Dustin Wehr

      Posted on 7:15 pm May 21, 2017.
      Reply

      I like the idea of inoculating interviewees against rejection. If interviewing.io isn’t already doing that, you should suggest it! Or if they’re not doing it well, suggest how they could improve.

    • user

      AUTHOR Dustin Wehr

      Posted on 7:19 pm May 21, 2017.
      Reply

      The author of the April 2017 Atlantic article should have read and cited this.

  • user

    AUTHOR John D.

    Posted on 3:47 am July 4, 2016.
    Reply

    I really don’t care for this question man vs woman. Because if I cared it would probably meant that I am biasing for a side, which I don’t.
    I know some women that kick some butts on development. They are awesome and they are widely recognize by their abilities.
    But for every woman like this I know at least 5 men with same ability or better.

    This is just my experience over 10 years.

    • user

      AUTHOR Bethany

      Posted on 9:02 am July 6, 2016.
      Reply

      Caring about gender disparities really isn’t about bias, it’s about being aware of privilege. Power isn’t good or bad, but it’s important to be aware of. When white men speak in a meeting, more people listen, and listen more closely. It’s unfair and it’s something we as a society should work to change, but right now, it is what it is. Being aware of your privilege and speaking up for people who don’t enjoy the same privileges you do is a key way for you to demonstrate that you care about people, and understand society’s bias.
      This article actually had a really good point about attrition events too – a lot of women face A LOT of discrimination in STEM fields – in school, in recruitment (venues and processes), in interviews, in jobs, etc. Knowing WHY you know that many men with the same ability or better – and hopefully employing strategies to mitigate that disparity – is really the core question.

      • user

        AUTHOR Df82

        Posted on 6:54 am July 7, 2016.
        Reply

        “It’s about privilege. Women are soooo discriminated against, even though controlled experiments prove otherwise.”

        Lol. You’re like a caricature.

        • user

          AUTHOR Garrett

          Posted on 12:21 am November 28, 2016.
          Reply

          Among men in the workforce, being a man is a privilege, because you are judged along the same axes as “everyone else”. Being a woman in a male-dominated workforce is one hell of a lot harder, especially because of men who tell themselves they are not biased, and then refuse to examine their own biases when they come into play. Like you, perhaps.

          In my 20+ years in this industry, the most common examples I’ve seen are from mid-level male developers who overreact to criticism from female developers as though their manhood were on the line, and then fail to communicate with them like they do with other co-developers because they are afraid of how it makes them look to the other men, thus isolating the female developer from the team and forcing her to work twice as hard to get things done.

          The ugliest examples I’ve seen are from men who would decorate their open-floor-plan workspace with figurines of busty, barely-dressed anime chix, men who would talk about how hot-or-not the female developers are, on the clock at work, and my personal favorite, a man in a group meeting who turned to his co-worker, just after she’d made a forceful point about keeping unstable code on side-branches, and said “you’re cute when you’re angry.”

          You try dealing with that shit for your whole career, and tell me it’s not a _privilege_ to never ever worry about it.

      • user

        AUTHOR gmmay

        Posted on 7:10 am July 7, 2016.
        Reply

        When attempting to perform legitimate research, addressing bias is crucial to achieving some sort of realistic understanding. Dubious pseudo-sociological hypotheses about ‘privilege’ have no place.

  • user

    AUTHOR Yann G

    Posted on 10:30 pm July 2, 2016.
    Reply

    “Contrary to what we expected (and probably contrary to what you expected as well!), masking gender had no effect on interview performanc”

    why is that the first assumption ? Im happy that you came around realizing that its not about inherent bias, but how the fuck did we get to a world where this is the basic assumption ?

    Imagine I had two race horses, with one consistently, for years, beating the other on a 500m run by around 10 senconds.
    would your primary assumption be:
    1- the losing horse is just not as fast even with the same raining regime, maybe they are built differently from birth.
    2- the horse racing society has conditionned me to think that the losing horse is wining when actually both horses have the same time and im probably a bigot because the wining horse is like green while the other is cyan or something.

    • user

      AUTHOR agilemind

      Posted on 12:02 pm July 17, 2016.
      Reply

      Or 3- the losing horse has been taught from birth not to run too fast because it’s undignified, and has been made to lose over and over in training because it wasn’t expected to become a champion, and has been pushed to spend hours and hours being groomed because people like to look at how “pretty” it is rather than spend time practicing running fast….

      • user

        AUTHOR Garrett

        Posted on 12:24 am November 28, 2016.
        Reply

        Then 3b, the winning horse feels sad that he doesn’t ALSO get the grooming and attention of the losing horse, and comforts himself by becoming a brony 😀

  • user

    AUTHOR Werkof Rodann

    Posted on 9:11 pm July 2, 2016.
    Reply

    Love all the people flailing their arms desperately looking for other ways discrimination was actually present. It’s got to be that the female-male modulator was worse quality! Or that speech patterns between men and women are different and this is where the discrimination happens!

    Look ladies, if you have to keep coming up with more and more difficult to disprove means for discrimination each time evidence comes out supporting the opposite, you’re probably on the wrong side of the argument.

  • user

    AUTHOR borabosna

    Posted on 3:47 pm July 2, 2016.
    Reply

    “to get to pipeline parity”

    This is sexism against men. Nobody should try to get any kind of parity in anything. That’s social engineering. Why isn’t anyone concerned with parity for men in nursing, veterinary, dance etc.? Because those don’t pay much. People only care about trying to force women into higher paying jobs.

    • user

      AUTHOR Garrett

      Posted on 12:34 am November 28, 2016.
      Reply

      You are arguing from the wrong premise.

      “People” are not concerned with the STEM imbalance. Politicians, CEOs, and hiring managers are, because they need more and better STEM workers. If dancers and veterinarians were in similar demand, you would see the same attention paid to them.

      This is a hugely growing field. If women and men are being rejected on the same terms, but the women are more discouraged by rejection, then politicians, CEOs, and hiring managers would like to find a way to turn that around, because it means they are losing perfectly good candidates.

      What would be helpful here is another landscape to compare it to: Is there a job that equal numbers of men and women apply for, that men drop out of more quickly? If there is, what accounts for that? If there ISN’T… If there is no job anywhere on Earth that men drop out of competing for sooner… Well, that would be quite interesting. But it would also totally invalidate the findings of the study above.

  • user

    AUTHOR Anna

    Posted on 11:23 am July 2, 2016.
    Reply

    The fact is that voice pitch is not the only thing that helps us to identify gender. I’m working in a company which deals with different voice conversions: high-low pitch, male-female, all this stuff. Every time I hear a recording with female voice converted to male (voice sound is changed but the intonations and the whole manner of speaking stay the same), it feels very weird. I hear a male voice but I also feel that men don’t speak like this. Never. Well, maybe stereotype gay men in some movies. And it seems to me that this manner of speaking matters for us much more than the sound itself.

  • user

    AUTHOR Susan Corwin

    Posted on 7:55 am July 2, 2016.
    Reply

    This article, while having good information,
    I view that it is strategically void.

    The human race is wildly successful… for 13,000 years ago.
    The historic selection doesn’t, however, get us to Alpha Centauri,
    …… as it inhibits 1/2 the population.

    There is the historical selection results:
    ..1. physical size differential
    ……testosterone is a much stronger muscular steroid than estrogen.
    ..2. the hormonal response to environmental conditions
    ……testosterone causing “elevation of mood”…. if a situation is “won”
    …… “take the hill” and “winning is the only thing” are mood response driven
    ..3. the early neural net programming of “blue blanket” vs “pink blanket”
    ……that has a reality based stress response basis.
    ..4. the mating behaviors that are wildly successful
    along with “modern” brainwashing.

    So, is there a “confidence” issue?
    Sure, for those who “winning is the only thing” getting a “high” are far more likely to do
    …”just about anything” to win.

  • user

    AUTHOR Zkauba

    Posted on 6:51 am July 2, 2016.
    Reply

    There are, in fact, differences in the ability distributions between the sexes that translate into a massive male overrepresentation at the upper ranks of ability, as Science vs. Feminism ably documents: http://www.sciencevsfeminism.com/the-myth-of-equality/sex-differences-general-intelligence/

    These cognitive differences explain why males dominate the top levels of achievement in all fields as well as the upper strata of all known human societies. Claims to the contrary rest on tests that are heavily sex-normalized and fail to account for the differing developmental trajectories of males and females.

  • user

    AUTHOR Dean Esmay

    Posted on 4:37 am July 2, 2016.
    Reply

    I would absolutely have predicted this result, because contrary to feminist doctrine, women are if anything favored in the IT field and generally treated a little better than the rest of us. The strange thing is that anyone should think it’s depressing if women don’t want to be in IT as much. For goodness sakes, that’s not a moral failing.

  • user

    AUTHOR air

    Posted on 10:46 pm July 1, 2016.
    Reply

    I spoke to a few music teachers who told me that girls can not take criticism, or pressure.
    Boys can handle pressure and criticism without a complete breakdown.

  • user

    AUTHOR jpv

    Posted on 3:23 pm July 1, 2016.
    Reply

    Interesting… always impressed when scientists show results that don’t match the hypothesis.

    I would love to see more studies like this, would need to:
    –Be triple-blinded (researchers, interviewers, and interviewees)
    –Larger sample size (esp. female)
    –Mitigate the possible attrition issue by not showing the results to the interviewee until X interviews are completed

  • user

    AUTHOR Dave

    Posted on 2:34 pm July 1, 2016.
    Reply

    So many comments from people who missed the key takeaway (which was conveniently bolded for readers):

    “Once you factor out interview data from both men and women who quit after one or two bad interviews, the disparity goes away entirely.”

    I’m sure there may be underlying cultural elements at work that result in more women quitting after bad interviews, but it gives this recruiting platform a clear way to help: reach out to anyone (male or female) who’s had a poorly-rated interview and encourage them to continue. Sure, it may come off as self-serving (“Hey, keep using our product!”), but encouraging this kind of resiliency could make a positive impact.

  • user

    AUTHOR Randy

    Posted on 7:42 am July 1, 2016.
    Reply

    Did you think to analyze the text of the interviews to see if there was an indication of gender in the way someone spoke. There could have been a positive bias towards a more male textual representation.

    • user

      AUTHOR karen straughan

      Posted on 1:54 pm July 2, 2016.
      Reply

      @Randy

      So what about the men who did slightly better with their voices modulated to sound female than when unmodulated?

      That was the finding. Women made to sound like men did worse than when unmodulated. Men made to sound like women did better than when unmodulated. The difference was slight, but there.

      There’s plenty of evidence out there regarding an innate general preference for female voices. This video cites a lot of the relevant research (in this case, regarding the voices of digital personal assistants). https://www.youtube.com/watch?v=NExaO5NTCz8

  • user

    AUTHOR cameron

    Posted on 6:19 am July 1, 2016.
    Reply

    I’m wondering if some of the initial attrition rates between the sexes may be caused not by gender but by societal norms of gender in a particular region?

    So I’m wondering if you would add a regional question into your analysis if it would show anything.

    I’m also wondering if previous life experiences may also have a factor. Such as, what other activities do the interviewees participate in? Sports, Art, Music.

    I’m not sure if this is bias on my part but where I am from it is more common for males to participate in athletics. I’m wondering if that may also be a factor as well, I say this because in athletics many are faced with failing but are commonly taught not to give up.

    But this is also something that is taught in many extra curricular activities during someone’s education.

  • user

    AUTHOR Isaac

    Posted on 5:46 am July 1, 2016.
    Reply

    Carol Dweck’s Mindsets or it’s more academic counterpart Self-Theories might be worth a read.

    She’s done a lot of research into how we react to failure.

    • user

      AUTHOR Isaac

      Posted on 5:48 am July 1, 2016.
      Reply

      Including looking into gender disparity in reacting to failure.

  • user

    AUTHOR Computers must be gender-biased

    Posted on 1:31 am July 1, 2016.
    Reply

    So let me see: you set up an elaborate project that was supposed to prevent the alleged discrimination and the whole thing blew up in your face, proving that there isn’t any gender discrimination in the first place! Ha ha … Astonishing that the assumptions of your leftist ideology did not match with the reality. Maybe the computers themselves somehow became gender-biased, hm? You have to investigate that next.

  • user

    AUTHOR Russ

    Posted on 1:31 am July 1, 2016.
    Reply

    The modulated voice sounded rather artificial/”roboty” and lower quality to me.

    So it seems plausible that interviewers are simply more likely to hire someone whose voice sounds natural and clear and is thus easier to understand, than someone whose voice sounds electronically processed and less clear.

    • user

      AUTHOR Carwyn

      Posted on 6:35 am July 1, 2016.
      Reply

      In the article it says that some people were made to sound processed/modulated but without changing gender as a control to eliminate this possibility.

  • user

    AUTHOR The Goldfinch

    Posted on 10:18 pm June 30, 2016.
    Reply

    So when a person who uses vocal fry (like Aline), which is a linguistic trait highly associated with women, or other phrases, pauses, rhythm, and word choices that express femininity, has her voice modulated to sound like a man, she does slightly more poorly than other women? A person who sounds like a man but speaks like a woman does more poorly than a person who sounds like a woman and speaks like a woman.

    But when a person has a voice that sounds female but his interests, phrasing, tone, rhythm, etc dog whistle to being male, he is rated slightly higher than a male voice that sounds like a man?

    It seems to me that your test is showing a preference for male gender-based values, independent of pitch and biological sex. John Smith’s lamentable example above from the architecture and construction world is a great example of this bias. He has no value for the skills the women brought to the table in communication, making sure everyone’s needs were met, collaboration, sensitivity to the building’s use in it’s design and implementation, and so on. He judged the women’s social collaboration as making men do the hard decision making, with consensus being something he opposed as effeminate and therefore unprofessional. Women having emotions were deemed intentionally manipulative, conniving even. “Macho men,” in other words those with traditionally male behavior, however, were unable to “help themselves” in yielding to the women’s femininity. So he sympathizes with men responding in a traditionally male way while demonizing women acting in a traditionally female way and believes the workforce suffered from a positive integration of the feminine and masculine traits.

    Would John Smith consider a poll of clients to see their satisfaction levels on projects with women or without women to see if end level satisfaction was higher, even if there were more corrections needed, when the communication and collaboration women brought to the table was a factor? He’d undoubtedly dismiss it as men unable to help themselves in condescendingly overrating the conniving women in order to avoid an emotional meltdown from expressing displeasure, as there couldn’t be an inherent value in something not in his own checklist of personality, traits, values, or skills.

    I imagine if John Smith were one of your interviewers and he heard a woman who dog whistled to him through aggressive, egoistic, traditionally male phraseology, which is a male biological trait independent of culturalization, then he would take note and increase his measure of esteem for that woman who talks like a man (independent of pitch) above his general poor esteem for feminine traits in the workforce. He’s looking for someone to make the hard decisions by aggressive fiat, not someone who will go around consulting stakeholders to make sure consensus is reached. And, a person he perceived as male would likewise drop in his estimation when the man had the right pitch but talked like a woman, or like Aline with the vocal fry who “totally” did not sound “like a dude.”

    Add in the wonderful point you uncovered about sensitivity leading to higher attrition among women, along with many of the great points others have made above (including, especially, the discouragement women get from being told they’re likely to face bias or fail–which there are studies to support both regarding gender and race), and the dynamics become multivariable and complex again. I agree that there’s a lot more that needs to be done.

  • user

    AUTHOR Anthony

    Posted on 8:36 pm June 30, 2016.
    Reply

    I’ve recently read an article that studied how women will not stay as often in a team where they are significantly outnumbered. Perhaps you should modulate the voices of the interviewers, to see if that affects the attrition rate.

    • user

      AUTHOR karen straughan

      Posted on 9:07 am July 1, 2016.
      Reply

      @Anthony

      So what you’re saying is women are sexist.

      • user

        AUTHOR Tiffany M

        Posted on 6:01 pm July 1, 2016.
        Reply

        @karen

        I’d wager that most western women are sexist, yes. We have feminism to blame for that. Just look at the presumption on which this study was done.

        Despite the fact that we have more legal rights and arguably more social favor than men, most western women have a strong distrust for the opposite sex. When feminists see less women in STEM, they don’t think what a rational person would think at first- that is, “maybe less women are interested in joining STEM fields, in the same way less men are interested in teaching children or becoming nurses”- they automatically assume that there’s some conspiracy being perpetrated by men to keep women down. Unfortunately they’ve failed to realize that this ideology is not “empowering” in the least, but quite the opposite.

        I have to give the author credit for at least accepting the statistics of the study, though- even though they go against the agenda of what they were looking for. Most feminists will never accept statistics. Just look at some of the ones in the comments here.

        • user

          AUTHOR karen straughan

          Posted on 2:24 pm July 2, 2016.
          Reply

          @Tiffany

          Off topic, but OMG, I hate this commenting app. (Must be a patriarchal plot to keep women like me in our place!) Why not go with Disqus? Everyone can manage Disqus, even rubes like me.

          As for your comment, yes, I agree. I’m very pleased with the fact that the researchers were honest in presenting their findings, and I don’t even have a problem regarding their speculation as to what the results mean (higher attrition rates for women). I disagree with their basic premise that the lower representation of women in certain fields is a “problem” that must be solved, but at least they’re not fudging their numbers to fit the narrative.

          I think, given the comments of MANY people here, there is no consideration given to the possibility (and I’m not saying this is a certainty, only a possibility) that women’s higher attrition rates in the face of difficulties or barriers might not be 100% about culture. I’ve seen many people suggest that perhaps more women drop out at this stage than men because they’ve had more “attrition-causing events” in the past. No one seems to be willing to consider that perhaps men and women have different thresholds regarding what they would see as an “attrition-causing event”.

          We know women apologize more than men do largely because they take personal offence over smaller slights than men. When a woman bumps you accidentally and apologizes, it’s because she would expect you to apologize if the situation were reversed and you’d accidentally bumped her. When a man bumps someone accidentally, he doesn’t apologize because he would not expect an apology if the situation were reversed. It’s kind of like both men and women adhere to the Golden Rule, but they have different standards as to how they would have others to “do unto them”.

          It’s entirely likely that men actually face more “barriers” and “obstacles” in this field than women do, and the results of this study certainly don’t refute that possibility. But we don’t raise our boys to ask for those obstacles to be removed, do we? We expect them to deal. Man up. Grow a pair. In the words of Gunny Highway: improvise, adapt and overcome, you sissies.

          This points to a very different set of criteria in terms of how men and women expect to be treated, with women having higher expectations of courtesy, politeness, kindness, friendliness, forgiveness, tolerance of failure and the rest, than men. And given the roles of sex hormones, I can’t imagine they don’t play a part in this.

          Can culture or socialization change this? Certainly, it can affect it, by either reinforcing it or working against it. But I can’t get behind the idea that the culture must change in order to make it so that women experience fewer of these “attrition-causing events”, because this only reinforces their sensitivity to them. It tells them that their feelings (taking criticism personally and being discouraged) are appropriate and should be indulged, which only lowers the bar as to what kinds of events they will find discouraging or upsetting. Then all you’re left with is a cascade effect of women becoming more and more sensitive to ever more trivial difficulties.

          I mean, honestly, my grandmother was a career woman ending up as manager of the general store and post-mistress of her town, all with a grade 3 education after being born into rural poverty in 1909. She’d tell these milquetoast women complaining about “subtle biases” and “microaggressions” these days to grow up, rub some dirt in it and go do something useful.

          It’s overcoming obstacles that builds fortitude and persistence, not having them removed for you. My grandmother would be rolling over in her grave if she knew the state of the modern western woman, who is the Princess and the Pea come to life.

          • user

            AUTHOR Garrett

            Posted on 1:02 am November 28, 2016.

            A very interesting line of thought.

            I have a wrinkle to add to it: Perhaps it’s too reductionist to say that higher female attrition rates in STEM fields are a call for the workplace to be “sissified”, for one obvious reason: Men and women fall on a broad spectrum, and STEM fields are geeky fields where intellectual rigor and emotional perceptiveness go hand-in-hand. Men and women alike need to learn how to work harmoniously with people who can be a good standard deviation above the sensitivity level of the general public. We’re not just talking about “apologize if you bump into them in the hall”, we’re on the level of “make sure you choose exactly the right words in your feedback to a comment on a pull request so you don’t accidentally invoke a jihad over code formatting between your lead programmer and your PM.”

            If a lot more women are dropping out of the field, this may be a legitimate sign, but of something else: The field is overpopulated with men (and some women) in middle-management positions who don’t know how to build a high-functioning team in general, and favor in others the skills that got them into management.

            Nothing is more refreshing on a team than working with someone who admits their mistakes, owns them, and works to fix and prevent them, humbly recruiting others as needed. How does that ability come across in the average job interview? It barely comes across at all.

      • user

        AUTHOR Taka

        Posted on 10:52 pm July 1, 2016.
        Reply

        Oh my gosh, it’s you! 😀

        Is it too much to hope for a video on the subject?

        • user

          AUTHOR karen straughan

          Posted on 2:35 pm July 2, 2016.
          Reply

          @Taka Maybe. Will have to wait until after I get back from London, tho.

      • user

        AUTHOR Estelle

        Posted on 3:30 am July 2, 2016.
        Reply

        I’m not sure that’s a fair summary of the statement and we tend be very liberal with the term “sexism” these days. We all have inherent biases which sometimes exist on a subconscious level. People often feel more comfortable around people they can identify with and relate to. Things like gender and race can play a significant role in one’s confidence and we all know how important confidence is in an interview.

        It is a suggestion for an experiment which may well have equally surprising results. The original experiment was to test bias in the interviewers. It would be remiss not to test for biases in the interviewees, especially considering it only requires a slight modification to the original test.

        In fact I just discussed the idea with my girlfriend who is also in tech and she said a female interviewer would definitely make her feel more at ease.

        • user

          AUTHOR karen straughan

          Posted on 3:03 pm July 2, 2016.
          Reply

          @Estelle

          There’s a very interesting cognitive difference between men and women when it comes to gender identity. There was a study done back in 2004 called “Gender differences in automatic in-group bias: why do women like women more than men like men?”

          http://www.ncbi.nlm.nih.gov/pubmed/15491274

          It could, more reasonably, have been subtitled, “why does everyone like women more than men like men?” In 4 of 4 experiments designed to detect positive attitudes and feelings of affiliation toward men and women, women strongly favored women. In 3 of 4 experiments, men also strongly favored women.

          This study, done by a feminist grad student at the University of Waterloo, seems to replicate this: https://uwspace.uwaterloo.ca/bitstream/handle/10012/6958/Yeung_Amy.pdf

          In this set of experiments, the subjects (men and women) were shown scenes of men and women interacting, then were asked to answer questions that would indicate hostile sexism on the part of the man, such as “how likely would this man be to abuse his wife?” or “how likely would this man be to deny a woman a promotion based on her sex?”.

          In situations where men treated women better than they treated men, they were judged to be not hostilely sexist. In situations *where they treated women equally compared to how they would treat men* they were judged by the subjects to be hostilely sexist. The only mitigating factor was if it was made clear to the subjects that the man was treating the woman equally in order to promote women’s equality (you know, for a benevolently sexist reason), at which point, he was seen as less hostilely sexist than the second guy, but still more hostilely sexist than the first guy.

          Now, I’m not going to tell you that sexism doesn’t exist, or that people in general (it’s almost never just men) have different expectations of men and women. They absolutely do.

          But men seem to lack a psychological mechanism for automatic own-group gender preference (makes perfect sense in terms of evolution), which makes the idea that men automatically favor men over women in everyday situations or contexts a highly dubious proposition. In fact, it throws the entire “patriarchy” narrative into question. A given woman seems able to make the leap from “I am a good person” to “women in general are good people”, while men simply don’t make that leap. Men do not tend to say, “I am a good person, therefore men in general are good people.”

          “In fact I just discussed the idea with my girlfriend who is also in tech and she said a female interviewer would definitely make her feel more at ease.”

          So what you’re saying is that women are sexist. You’ve only proved my point regarding the fact that both men and women have more positive feelings about women.

          One interesting thing, though. There’s also been research done on mentoring and gender in academia (I don’t have it to hand, but could dig it up), and it shows that even though your friend would feel more at ease with a female interviewer, she’d probably get a poorer score. It is MEN in academia who are more likely to positively mentor female grad students and underlings by talking them up, giving them credit, etc. Female academics give less positive support to female grad students and underlings on average.

          What is even more interesting is research done into photos and resumes. HR is mostly women, and if you’re an attractive man, putting your photo on a resume will make it more likely to get past HR than if you had no photo or were unattractive, but if you’re an *attractive* woman and include your photo, it will be more likely to be binned before you get to the interview stage than if you had no photo, or were unattractive.

          If your friend thinks she’s going to get a fairer shake from a woman than a man, she’s mistaken.

          • user

            AUTHOR Estelle

            Posted on 6:33 am July 13, 2016.

            I think you misunderstand my dispute. I believe what you are saying is largely correct and well researched. However I don’t think “women are sexist” is the conclusion to be drawn from my statement.

            For example, whenever I meet someone from the same country as me, we tend to click more easily because we’ve often shared similar experiences growing up. I wouldn’t consider myself racist for having more in common with people from my country. I would be just as willing to speak to anyone else and still try just as hard to connect with them.

            Being able to relate to someone more easily because they share similarities to you is different from being prejudiced against those who don’t share that similarity. Whether or not she would be better off being interviewed by a man or a woman is a different debate. I only argue that it’s not sexist to worry about being in a male dominated work place. Not knowing if you’d fit in or if you’d be treated differently. Sometimes just a tiny bit of positive reinforcement makes a huge difference.

            Human beings aren’t as rational as we like to think. We worry too much about things we ought not to and worry too little about the things we should do. We draw conclusions from insufficient evidence and disregard things that have overwhelming evidence in their favour. There is often a large gap between the truth and what we perceive to be the truth. I think a more accurate summary would be “sometimes, some people worry about things”. As you’ve meticulously pointed out, some of these things aren’t worth worrying about when you look at the data but that doesn’t change the fact that some people do sometimes worry about such things, especially when they haven’t seen the data.

            This doesn’t affect the validity of your statements. I only make a fuss over it because I’ve noticed that people are becoming desensitised to terms like “sexism” and “feminism”. If we keep using the word “sexism” where it isn’t due it will only increase the stigma against feminism.

          • user

            AUTHOR karen straughan

            Posted on 10:00 am July 13, 2016.

            @Estelle We are all sexist to some degree in that we perceive and treat men and women differently. In my view, these are inherent biases that are biological in origin and will therefore never be completely eliminated. I don’t think that’s necessarily a bad thing–I think it’s a morally neutral aspect of the human condition.

            My bone to pick here, as it were, is the currently fashionable idea that sexism stems from some mythical “patriarchy” that seeks to benefit men as a class at the expense of women as a class. This idea should be relegated to the same obscure corner of society where 9/11 trutherism and birtherism exist. It is little more than a conspiracy theory.

            In nearly every single metric we would use to demonstrate systemic cultural and institutional bias against blacks compared to whites in the US, men fare more poorly than women. From early childhood onward, society shows much greater concern for the safety, health and wellbeing of women and girls than it does for men and boys. Simply put, we treat our women less harshly than we do our men, and this is generally true of all cultures, currently and historically.

            I mean, think about it. If you were to look at the statistics on violence, men and boys are victimized at least 3 times as often as women. If you look at family violence in particular, men and women are roughly equally likely to be victims. If you look at children, parents and caregivers hit boys 2 to 3 times as often as they hit girls, starting before age 1. While more girls are sexually abused than boys, more boys suffer the most extreme forms of sexual abuse than girls. No matter the gender of the perpetrator, violence is more likely to target men and boys than women and girls. And perhaps most disturbingly, children are more likely to be victimized by violence than adult women.

            Yet our only demographically specific violence laws and policies address violence against women–who are already the safest demographic in society.

            At the same time, we have cultural narrative that violence against women is normalized (or even required) by society, that it’s a patriarchal mechanism designed to subjugate women for men’s benefit. This culture that has created a special set of federal laws and policies to protect women from violence even over and above protecting children, is, we are told, inherently and pervasively misogynistic and supportive of gender-based violence against women.

            Looking at the situation as a whole, if one were to apply Occam’s Razor, the assumption would be that society is not only not misogynistic, but that whatever biases do exist exist in the direction of society caring more about women than men, or even children. Our conclusion would be that society favors and elevates women, not that it hates and oppresses them while privileging men.

            But that’s not the general consensus, is it? Ironically, in my opinion, I believe that sexism is the root of this contradiction. We feel as if we cannot ever favor women enough, therefore we *must* be disfavoring them. That, Estelle, is part of the sexism I’m talking about.

            It is so deeply ingrained in our psychological wetware, that we will go to great lengths to deny the mere suggestion that we treat women better than we do men in many, if not most areas, that we are biased in favor of women rather than men, and that until every difficulty and every bit of suffering has been removed from the life of every woman in society, that society is not merely misogynist, but is based on misogyny.

            Look at some of the comments here, trying to find some explanation, ANY explanation for the unexpected results of this experiment. We have people suggesting that gendered speech patterns “tipped off” the interviewers and skewed the results, even though women who sounded like women did better than women who were made to sound like men, and men who sounded like women did better than men who sounded like men. In other words, female voices did better compared to male voices whether there was a man or a woman behind that voice.

            We have people suggesting that women have probably suffered more “attrition-causing events” by the time they made it to this service, and that must be why they were more willing to give up.

            We have people claiming men are privileged in tech and should be sensitive to the needs of women, even though the results of this experiment would suggest that whatever small bias may exist in interviewing exists is in women’s favor (which means that men are actually disprivileged in this situation).

            We have people scrambling to explain how systemic discrimination against women in tech really does exist even in the face of evidence suggesting not only that it does not, but that whatever discrimination does exist is in women’s favor.

            I’m constantly flabbergasted by this tendency toward mental gymnastics people engage in based on the premise that all roads lead to misogyny. If that is not sexism, then what is?

      • user

        AUTHOR Happosai 八宝斎

        Posted on 4:57 am July 2, 2016.
        Reply

        @gww Indeed! The patriarchy has grown so devious as to employ women as it’s primary footsoldiers now. The poor dears!

  • user

    AUTHOR Renat

    Posted on 3:30 pm June 30, 2016.
    Reply

    So this experiment shows there is a ‘women bonus’ in tech. Men should demand equal treatment under the law. Sexism has no place in the industry.

  • user

    AUTHOR FriedBrain

    Posted on 1:09 pm June 30, 2016.
    Reply

    The commentary here is enough to fry the brains of all genders. In any country.

  • user

    AUTHOR James

    Posted on 12:53 pm June 30, 2016.
    Reply

    >> Why I’m not depressed by our results and why you shouldn’t be either

    Here’s a crazy hypothesis. Maybe, just maybe, men and women are different? Maybe they have different strengths and aptitudes? Crazy right? I know!!!

    • user

      AUTHOR Mad beans

      Posted on 4:05 pm June 30, 2016.
      Reply

      If you read the last section you would have noticed that the difference is not aptitude but attitude

      • user

        AUTHOR Parson James

        Posted on 5:50 pm June 30, 2016.
        Reply

        The curves extrapolated from two data points each……the stupid – it burrrnssss!

      • user

        AUTHOR karen straughan

        Posted on 2:29 pm July 1, 2016.
        Reply

        @Mad beans.

        It’s a strength to be willing or able to continue even when discouraged by a negative result. (It can also be a weakness, but still…)

        There are plenty of aggregate differences in men’s and women’s personalities that can explain wildly different attrition rates despite similar levels of ability.

        Those differences in tech seem to persist even in the face of the “tech is for girls! Grrl power!” messaging that’s been going on for a decade or more, and despite the fact that girls have been getting more positive feedback in school (in terms of teacher/student interactions and in terms of grades) than boys since the mid 1990s (this is a well-guarded finding on the part of the UUAW in 1994, no less, that has never been publicized by them–both boys and girls agreed that boys got more attention from teachers in class, but both boys and girls ALSO agreed that the entire surplus and then some was negative attention, and that girls got much more praise and encouragement than boys).

        On top of this, several studies now in the UK and Canada, as well as the US, have discovered that female teachers in primary and middle school grade boys’ work down compared to the assessment of gender-blinded evaluators. By the time you get to high school and university, the trend includes male teachers, and extends to giving girls bonus points compared to evaluators who are gender blinded, rather than just docking males. More than this, by halfway through primary school, boys can detect the bias from female teachers–when asked to bet on what grade they’d get on a paper, they estimated much lower if they knew that the evaluator would be female and would know they were a boy than if they believed the evaluator was a man or would not be told they were a boy, and in primary school, some 85% or more of teachers are women.

        This has been going on for some time, with little to no fanfare, all while male drop-out rates have been increasing relative to girls, and their likelihood of entering college goes down.

        So what we seem to have is a conundrum. It seems that there is a significant amount of systemic bias against boys in school–everything from how likely you are to be scolded by a teacher to being docked marks just for being a boy. (I can direct you to the work of Dr. James Brown, who’s worked in education from kindergarten to university for over 50 years, and he could tell you things that would make your hair turn white).

        And yet somehow the environment that does all this creates men who are much less likely to give up after a single bad interview than their female peers.

        How can this be, if there is no difference in aptitude. Perhaps the difference in aptitude is seated within a personality trait, rather than a technical ability?

        • user

          AUTHOR Taka

          Posted on 10:58 pm July 1, 2016.
          Reply

          Absolutely not disagreeing with you here, but would you mind dropping some links – even if just to another link aggregation? This sounds like a goldmine of evidence that I’d like to have on hand for the future.

          • user

            AUTHOR karen straughan

            Posted on 3:39 pm July 2, 2016.

            You can google: venus mars measuring global sex differences in personality

            It will almost certainly take you to a really cool study.

            As for the stuff on bias in education, I can ask James Brown for his links (mine were all lost when my old computer got spilled on).

          • user

            AUTHOR Taka

            Posted on 5:24 pm July 2, 2016.

            Ah yes, I’ve seen that study. It is very interesting.

            As far as the education links go, if it’s not too much trouble to get ahold of them, I’d very much appreciate them, thanks!

        • user

          AUTHOR Garrett

          Posted on 1:42 am November 28, 2016.
          Reply

          In my last two interviews (both of which landed me jobs) I used the following line: “I believe three personality traits make me a good programmer. I am lazy, stubborn, and suspicious.”

          Then I waited a beat, and said (more or less) “I’m lazy so I’m constantly looking for a way to automate things. I’m stubborn, so when I hit a bizarre bug, I throw everything at it. And I’m suspicious, so I insert logging hooks, and write tests, and check configurations.”

          It gets a laugh and it’s a good humble-brag line at the same time. But the reason I bring it up is, you’ve touched on the greater level of external encouragement that women receive, and the greater reaction they have to its absence, and that stands in contrast to my admission of stubbornness as a positive trait. I was self-taught, and pushed ahead against an uncooperative and vexing machine to learn my trade, in a complete absence of positive feedback from any living soul. Sheer bloody-mindedness, as Sir Pratchett would call it, was at the very center of this pursuit and career from the beginning and to the present day.

          This is really not a fulfilling career for anyone who is easily discouraged.

          Nevertheless, I have met many good female developers, and they all showed that same bloody-mindedness. I’m glad they’re around. I would never say that workplace sexism is neutral for them, let alone works in their favor, though. It’s a lot harder for them to get their jobs done when some hard-to-isolate group of co-workers is keeping a minimum distance out of fear, and another hard-to-isolate group is constantly distracting themselves by asking “is she flirting with me?” over and over in their heads every time they stand over the same screen.

  • user

    AUTHOR David

    Posted on 11:36 am June 30, 2016.
    Reply

    Great blog! I really enjoy it and this post was no exception. I recounted it to my wife, who is very non-tech and also quite uncompetitive, and she was not very surprised by it. She is supremely uninterested in all things tech, and I think we all tend to forget that much (most?) of humanity shares her views.

    Slightly apropos, perhaps, and definitely only anecdotal, but here is a story from a doctor I happened to have a longer conversation with. He said he had twins – boy and girl – and when they were young he coached both their soccer teams. On the differences between genders, he described the boys as follows:

    If you noticed in a game that they weren’t passing enough, at half time if you said ‘team, you need to pass the ball! I want you to pass the ball more!’ each one of them would be convinced that he was talking to someone else. To them it wasn’t criticism, and it had nothing to do with them.

    The girls, he said, were exactly the opposite: each was convinced that he was talking to them and them alone. And, importantly, they would perceive it as criticism and reacted quite strongly.

    I suspect this is a complicated area, and I really enjoy the exploration.

  • user

    AUTHOR JASMINE

    Posted on 11:09 am June 30, 2016.
    Reply

    I think it has more to do with culture. People in the nerdy world tend to fail at social cohesion with people who have interests outside of the nerdy world. This is true of any culture, but I think it’s worse in any profession historically male dominated. People who don’t feel they stand together with their peers tend to feel rejected. I think in combination with women being more sensitive to rejection and women tending to be the social butterfly it’s natural to want to find a career where it feels right. Which usually isn’t one where you need to be good at fending off doubts. My self identity is rebellious to expectations outside of my own and proud of my geeky-ness. I don’t struggle with that. I totally see it with other women. That feeling of rejection is less a bias and more a cultural gap that further feeds into a feeling of rejection.

  • user

    AUTHOR aurel

    Posted on 6:35 am June 30, 2016.
    Reply

    “Why I’m not depressed by our results and why you shouldn’t be either”

    And IF results would prove that women are generally worse at computer science, why in heaven’s name would that make you depressed?

    I am a female programmer and I don’t care… I am also a runner and I don’t care that man in general are faster. Should we make some very elaborate study, that would prove that mabe they are faster, but that’s only because women are more shy? To hell with that!

    • user

      AUTHOR zxq9

      Posted on 12:20 am July 1, 2016.
      Reply

      Steady, there! You can’t just come in here throwing the narrative for a loop!

      That said… I’ve been working in tech (mostly software and hardware development) for several years, mostly in hiring. I have hired both men and women, but never had men quit or elect to work lower responsibility (which tends to correlate to fewer hours or personal commitments to deadlines) just because they had kids or something. I have had numerous female employees ask to reduce work hours or simply not return from maternity leave — only to pop up as contributors on the open-source side later on.

      Technical ability? I don’t know. I’ve met remarkably talented people of both genders (and in-between genders and genders I never knew) and never really gave it a second thought… but I have certainly observed that women tend to opt-out of the high responsibility roles more than men. When men opt-out it is usually because they are actually burned-out (which sucks a lot worse, imo — some people never recover from that).

    • user

      AUTHOR Taka

      Posted on 10:54 pm July 1, 2016.
      Reply

      <3

  • user

    AUTHOR Steve

    Posted on 6:29 am June 30, 2016.
    Reply

    A thought on your initial results of “no apparent change”. Maybe your assumption that voice modulation is a gender mask is incorrect? There might be subtle cues in language choice or communication style that are not affected by the voice modulation.

    You might be able to determine effectiveness by asking the interviewer to guess interviewee gender at the end of the interview.

    • user

      AUTHOR karen straughan

      Posted on 2:44 pm July 1, 2016.
      Reply

      Okay. Maybe there are give-aways. But since there was a slightly higher score for women without the disguise than women disguised as men, and for men who were disguised as women over undisguised men (as in, whether man or woman, the test subject did slightly better if they *sounded* like a woman than if they *sounded* like a man)… well, I don’t know whether the interviewers were detecting the gender of the applicant from other subtle cues. Why wouldn’t they have detected those subtle cues from men modulated to sound like women, and given them the same slightly lower scores as unmodulated males? Why would natural sounding women get slightly higher scores than women modulated to sound like men?

      The slight increase was consistent: whoever sounded like a woman did better than they did when they sounded like a man.

      • user

        AUTHOR Garrett

        Posted on 1:47 am November 28, 2016.
        Reply

        The slight increase was also not statistically significant, given that we’re looking at a mere 238 people. So let’s not start writing any theses just yet.

  • user

    AUTHOR T

    Posted on 6:15 am June 30, 2016.
    Reply

    “Rather, it’s about women being bad at dusting themselves off after failing.” Ooor it’s about a society which teaches girls and women they are never good enough?

    • user

      AUTHOR Peter

      Posted on 11:00 am June 30, 2016.
      Reply

      Or maybe men and women are inherently different and men can be better at some things than women are, and vice versa. Maybe technical jobs are not the strong suit of the majority of women. Maybe biology created gender roles and not society. Maybe it’s okay for women to be different from men but still allowed the same opportunity. Maybe we don’t have to pretend men and women are exactly the same. Maybe we can cherish our differences instead of highlighting them and acting like it’s some massive conspiracy.

    • user

      AUTHOR Kent

      Posted on 11:56 pm June 30, 2016.
      Reply

      Peter, you ever hear of Ada Lovelace? She is considered to be the world’s first programmer. While Charles Babbage worked on building the first calculators, he only ever saw them as calculators; but Ada realized that data itself, *any* data could be represented by states within a machine, and more importantly that instructions could be given to the machine to change it’s operation, in a word, programming. She predicted the very foundation of our technology before it even existed!

      But she was born in 1815, and very few thinkers were willing to take a woman’s ideas seriously. Her theories on computation would have to wait more than a century before they were validated by the work of Alan Turing and others. (fun fact: Joan Clarke was a brilliant cryptanalyst who worked with Turing, but she also has been somewhat neglected by history)

      Think about that for just a moment. You use the internet every day, less than 100 years after Turing’s first machine that we can think of as a modern “computer” What wonders, what technologies could we have reaped if nearly 200 years had passed since Ada Lovelace’s work had started the digital age in her lifetime?

      This is the very real, and deeply tragic cost of sexism in tech. By shutting out an entire section of people; you’re shutting out the ideas and opportunities they bring to the table. True, maybe eventually a few get to contribute; or maybe eventually more homogenous groups will innovate; but in the meantime we’re all losing out by not having access to the ideas and opportunities diversity brings. In the long run, men lose out too when women are pressured to stay away from tech.

      (P.S. One more honorable mention is of Navy Rear Admiral Grace Hopper, who invented the first modern compiler. Ada did invent the theory of computation and describe, though she lacked tools to execute it; but Grace Hopper is the reason your computer has more sophisticated programming then a glorified calculator. Countless other women have contributed to the growth of the digital age, though far too many of them have gone unnoticed and underappreciated.)

      • user

        AUTHOR Mark

        Posted on 6:31 am July 1, 2016.
        Reply

        There’s a big difference between “Group A isn’t as good as Group B in Thing” than “Group A is completely incapable of doing Thing”.

      • user

        AUTHOR karen straughan

        Posted on 8:58 am July 1, 2016.
        Reply

        @Kent, regarding the story of Ada Lovelace.

        You seem to be laboring under the misconception that revolutionary or innovative ideas were immediately accepted by the establishment when men presented them, back then.

        Ignaz Semmelweis was the privileged male who proposed that hand-washing could reduce infection rates in hospitals. At the time, germs had not yet been discovered as a source of disease, and the “nervous excitation” theory was still prevalent. He initiated a program in the hospital where he worked of having staff wash their hands in chlorinated lime between patients. Childbed mortality rates were reduced by 80% in just one year.

        He went to his superiors with his findings. They wouldn’t listen. He showed them the results. They didn’t care. He was fired. Immediately, childbed mortality rates rocketed right back ot to previous levels.

        More than this, he was shunned by the entire medical community, was unable to find steady employment afterward, and ended up dying penniless and friendless in a mental institution.

        Too bad the rampant sexism against men at the time (early 1800s) prevented the establishment from being accepting of his radical ideas. If only the matriarchy had listened, how many women and babies could have been saved? /s

        • user

          AUTHOR Garrett

          Posted on 3:49 am November 28, 2016.
          Reply

          This strikes me as a particularly odious false equivalency, given that Ada Lovelace was of the gender that wasn’t even allowed to vote until 70 years after she was dead and buried. (1850, vs 1930 in England, roughly.)

  • user

    AUTHOR Caroline

    Posted on 5:44 am June 30, 2016.
    Reply

    Honestly, I am quite surprised by the flaws of your method. In your hypothesis, you assume the gender gap stems from interviewers’ bias that might be negative towards women, i.e. at the same level of qualification, women are perceived to be less competent than men. You assume that you can mask gender by altering the interviewee’s voice. However, the voice is only one part of communication, the other being, obviously, your choice of words etc., how much you want to show off, how self-confident you are. In other words: As a woman, you are trained to communicate in a different way than men, this starts as early as pre-K.

    Your hypothesis assumes a sort of crude sexism that might have prevailed a couple of decades ago. Apart from some very old and some very angry men, nobody believes this today. Things are actually more subtle.

    To all the commentors who think that women are just not as into tech as men: Just shut up. Take a look across times and cultures, and you will find that the question of what is a typically female and a typically male occupation changes widely.

    • user

      AUTHOR Debbie

      Posted on 5:28 pm June 30, 2016.
      Reply

      Sorry to disappoint you…. but the participation rate and results show that women aren’t really cut for this. Sure, there are lots of women in the industry… but not as many as men and not as good either. So Facts. If you are good, kudos. Congrats.

    • user

      AUTHOR Jason Voorhees

      Posted on 5:49 pm June 30, 2016.
      Reply

      >To all the commentors who think that women are just not as into tech as men: Just shut up.

      It is a demonstrable fact that this is the case. Yes, it may very well be subject to change, as were a lot of things in the past, but the rock solid fact remains that women, right now, are much less into tech than men are.

      • user

        AUTHOR Jorvial

        Posted on 8:46 am July 1, 2016.
        Reply

        You don’t seem to understand that ideology trumps experiment.

    • user

      AUTHOR Flander

      Posted on 4:46 am July 1, 2016.
      Reply

      Yeah, devil interviewers recognizing a woman’s behaviour behind a man voice and punishing her just for being a woman… *sigh* Accept it, men are generally better in a tech world. In fact, the loudly women that complains a lot about this tend to be the ones with less public work covering their statements

    • user

      AUTHOR karen straughan

      Posted on 2:53 pm July 1, 2016.
      Reply

      “To all the commentors who think that women are just not as into tech as men: Just shut up. Take a look across times and cultures, and you will find that the question of what is a typically female and a typically male occupation changes widely.”

      Yes, and oddly enough, in many cultures that are more patriarchal, you are more likely to find women in computer science, engineering, tech, etc, than in cultures that feminism has had its way with for a hundred years. Isn’t that interesting?

  • user

    AUTHOR Matt

    Posted on 5:43 am June 30, 2016.
    Reply

    The logic of coming to the conclusions from the study are not totally clear to the reader in this article (which is great anyways!). If I got this right, the conclusion for performance in IT job interviews and (more speculatively) in tech jobs in general is this:

    Due to it taking less discouragements for women to drop out than for men, the average woman-in-tech is more of a novice than the average man-in-tech, and because of that (and just that) women perform less well on average.

  • user

    AUTHOR V Y

    Posted on 5:24 am June 30, 2016.
    Reply

    Thank you for posting this. It was an interesting read, and I look forward to seeing any future results as you continue to look at this.

    I wanted to suggest that if we divide people into the (false but useful for these purposes) binary of women and men and look at average starting points on your theoretical attrition curve, we shouldn’t expect them to be in the same place. That is, by the time  women are at the stage of being interviewed for jobs, they have, on average, encountered quite a number more potential attrition events. I am a PhD candidate in a STEM field and I have always had a great love of mathematics. I started getting discouraging feedback even as a child, but there was plenty more to come in university. There are more formal results that suggest this is not an atypical experience for women, and that women encounter this type of negative feedback more often than men do in STEM. So one possibility is that women and men have the same attrition curve, but women are generally much further along it by the time they are at the interview stage. But even if it is not the case that the curve is the same, I would still expect women to be further along their curve. (Assuming, of course, that it even works to have such a curve. I recognise that in that part of the article you were more just musing on the topic and not presenting it as solid data, and I am similarly participating in this type of thinking out loud. It’s quite possible that a single curve like this lacks the right parameters to really model it.) I mean, hell, even just reading these comments there are far more people insisting that women just aren’t as good at tech than there are suggesting that men aren’t as good at tech. The worst position men seem to be put in is of being merely the same as women. What is the cumulative effect of seeing this in the comments of every piece on this topic? Does that push any women further along their attrition curve?

    Another alternative is this:
    If the disparity disappears once you weed out people who quit after one or two interviews, then perhaps it isn’t really about which gender dusts themselves off better but some other parameter that happens to have been correlated with gender for non-direct reasons. It’s possible you happen to have more women than men who are near that “final straw” point when entering the interviews, and that people already considering giving up don’t perform as well in interviews. Especially since for those remaining people the performance seems even across gender.

    Something that isn’t clear to me:
    When you say most people opted in for the the study, does it apply to both interviewers and interviewees? That is, did all of the interviewers opt in or did any choose not to? If there are any interviewers who didn’t participate, I would be concerned about the bias that would introduce. It could, for instance, select for interviewers who don’t see the merit in the experiment because they are fine with the gender gap or accept stereotype-based explanations for it, and maybe these people are more likely to have a negative impact in women’s rankings and performance. Or could introduce bias of some other origin.

    One last bias thought:
    Interviewers who know they are being evaluated might behave differently. I don’t mean they do it deliberately or with some intention to mislead, but that they suddenly have a heightened awareness of the issue and that might slightly shift how they behave because it is present in their thoughts. How did the subset of unmodulated interviewees compare to the modulated groups and to the pre-study numbers you produced? Any difference there?

    To anyone in the comments who hasn’t read it, I highly recommend Cordelia Fine’s book “Delusions of Gender”. It presents, critiques, and synthesizes a fair bit of research around aptitude and gender taken from neuroscience and psychology. Perhaps counterintuitively, I especially wish to recommend the book to those who already support women in STEM, because it will provide you with published references that back up your explanations when talking to those who don’t want us around (who are much less likely to be willing to simply read the book). And there is enough material that even someone well read on the topic has a high likelihood of encountering some new studies or new takes on them. It’s quite accessibly written and also available as an audiobook and I swear I have no connection whatsoever to the author. 🙂

    • user

      AUTHOR ST

      Posted on 3:30 am July 1, 2016.
      Reply

      “Women and men” is not a false binary and asserting otherwise just demonstrates that you’re out of step with reality.

      • user

        AUTHOR V Y

        Posted on 1:56 pm July 2, 2016.
        Reply

        I’m not sure of the nature of your objection here. Is it about labels or about being unaware of the full set of categories?

        Perhaps we do not live in the same country, so to clarify: where I live, the words “women” and “men” generally refer to gender and “male” and “female” refer to biological sex. Admittedly, I don’t know if that is the norm elsewhere.

        Having said that, neither gender nor biological sex is actually binary. Gender is cultural and refers to socially constructed behavioural expectations, so it is by its very nature something that varies depending when and where you are. I am going to assume I don’t need to further elaborate on how gender in particular is not binary, but if you’re still unclear just google for “gender versus sex” or something similar. There are plenty of articles about it, as well as formal definitions at bodies that provide resources and standards like the APA.

        As for biological sex, it simply is not the case that human’s either have XX or XY for sex chromosomes. To be sure, these are the most common two configurations, but there are people with a single X or XXY or XYY or XXYY as well as a number of considerably less common configurations including XXX, XXXX, and others which you can find by googling if you really care.

        In fact, there are even XX males because there are (rare) circumstances during meiosis that can leave the SRY gene (which is normally male) on an X chromosome.

        There is also the issue of genotype and phenotype and people with XY chromosomes who are insensitive to androgens.

        For all that, gender is even more expansive if we start trying to nail down formal boundaries to categories. The complexity of all of this is why I said it wasn’t an unreasonable simplification at this time to artificially binarise things, but when we create a simplified model it is useful to at least acknowledge that we’ve done so.

  • user

    AUTHOR Willy

    Posted on 3:50 am June 30, 2016.
    Reply

    My experience toward attrition and how men and women cope with it.

    Men usually just tell each other straight away they’ve done bullshit, even in harsh word, cause it IS bullshit. No biggie, no hard feelings. My work has been critizied, not me. I can just do it better next time.
    (e.g. Linus’ mailinglist)

    Telling the same thing to a women, and she doubts herself, her abilities. She’s not just brushing it off. (Not true for all women, obviously but that’s what I’ve noticed.)

    In school I have noticed this aswell. Boys’ work was criticized more harshly. Maybe teachers thought (or noticed by experience) they could take it? We had some girls who began to cry when their work was criticized the way boy’s were (Whether this was genuine or they just played the teachers, I dont know). And there were always more boys in class and none of them cried or was really openly upset.
    As a result, teacher’s were generally more lax about the girl’s work.
    (Sure, this is all subjective, but may hold some clues, so sharing it anyway)

    • user

      AUTHOR john smith

      Posted on 1:37 pm June 30, 2016.
      Reply

      My career was electrical construction engineering on the power side. At first female project participation was limited to interior design and a few architects. The number of female architects exploded and they also began working their way into the engineering side. As a rule they believed their degree made them qualified. As we all know, a degree means you are highly trainable and that is all.

      My experience over time with many female designers is they have a much higher error rate (as evidenced by error and omissions change orders), they are very risk averse (leaning upon men for the tough decisions), and react emotionally to criticism or a demand they make the tough decisions themselves.

      I have never seen a man cry in a design meeting or construction progress meeting when confronted with design shortcomings or construction failures. I have seen many women shed tears over this. And invariably, the macho construction men ride to the rescue of the damsel in distress. The women know this works and the men can’t help themselves.

      In essence, much higher failure rates (and the associated costs) are tolerated to accommodate female ineptitude and emotionalism in an industry where such traits were once quickly excised for the good of the project, the team, and the companies involved.

      Hate my post if you want. It is based upon 35 years of experience beginning in 1979 (male only) and ending in 2014 (gynocentric tolerant). I’m sorry, but facts are facts.

      • user

        AUTHOR Garrett

        Posted on 2:05 am November 28, 2016.
        Reply

        So the newcomers to your field weren’t as good at – or as secure in – their work as the seasoned professionals judging them, and all the people with vaginas were newcomers?

        Mighty fine horse you’ve got there, but I don’t suppose you’ve taken into account… It does look suspiciously like a cart… And that thing you’re calling a cart just took a big crap on the road…

  • user

    AUTHOR reddit lurker

    Posted on 3:16 am June 30, 2016.
    Reply

    Upvote for “Tharrrrr be the gender gap”

  • user

    AUTHOR Christopher Allen

    Posted on 3:14 am June 30, 2016.
    Reply

    > As it happens, women leave interviewing.io roughly 7
    > times as often as men after they do badly in an interview.

    If men and women were coming in with equal skill levels—and therefore equally likely to do badly on their first interview—shouldn’t this mean that on average women will do *better* than men, because the male scores will be weighed down by stubbornly persistent incompetent men?

    It would be interesting to see what the average male and female scores are on their first interview vs. average male and female scores on their 2nd, 5th, etc.

    (I suppose it’s possible that there will be no difference if the most competent women are also being removed from the interviewing pool because they’re snapped up by employers keen to hire them.)

    • user

      AUTHOR Kent

      Posted on 7:11 am July 1, 2016.
      Reply

      Those are some really good points. How do we know that the women who seemed to “quit” prematurely, didn’t actually end up getting work through another platform for example? That’s pure speculation of course, but the mere possibility undercuts the author’s entire thesis; that “oh I guess women are rarely successful in tech because they just don’t try hard enough.”

      This “study” simply isn’t rigorous enough or verified enough to draw clear, meaningful conclusions from. It seems it is unfortunately *more* then enough to be used to further entrench prejudices already there, though.

  • user

    AUTHOR Stef Kuypers

    Posted on 1:20 am June 30, 2016.
    Reply

    I wonder whether it matters whether the interviewer is male or female. Would that make a difference? Are names revealed? I’m not familiar with the platform and from the article I got that they want to keep things anonymous so I’m supposing they’re not. But maybe just first names? I so, that might jeopardise the gender masking.
    And last, is there a possibility that we subconsciously pick up on gender in the way people talk? I’m totally going out on a limb here but if we want to get to the bottom of this we need to take all possible factors into account, right?

  • user

    AUTHOR vortex

    Posted on 1:10 am June 30, 2016.
    Reply

    Maybe, just maybe, there is a gap between the sexes in tech because of different interests between the sexes? Just like in chess, mma fighting and vidya. People apparently think that evolution, primarily the evolution of sexual dimorphism in humans, stops at the neck. This assumption leads to unnecessary, or perceived problems that can’t be fixed.

    • user

      AUTHOR gummyworm

      Posted on 7:02 am June 30, 2016.
      Reply

      Even if that’s true, that gender dimorphism doesn’t stop at the neck, the crowd that’s trying to inflate the importance of traditional gender roles certainly aren’t a credible source for any truth on the matter whatsoever.

      • user

        AUTHOR JB

        Posted on 12:33 pm June 30, 2016.
        Reply

        Unlike the “crowd” that is desperately reaching for a narrative to continue seeing women as a victimized class, despite every privilege and opportunity uniquely afforded to women and girls, and their dominance in education, both as teachers and as students?

        The people who insist loudly they aim to disrupt the status quo have the educational system and politics on their side, with feminist organizations like N.O.W. overturning attempts at fairer legislation, feminist politicians like Obama or Trudeau repeating debunked statistics, and even the U.N. producing reports about violence and harassment that quietly need to be retracted, for being sourced from fairy tales. Who lacks credibility here?

  • user

    AUTHOR karen straughan

    Posted on 12:52 am June 30, 2016.
    Reply

    Having been heavily interested in gender differences for many years myself, I think this study might be interesting to you: http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0029265

    It took data from an earlier study that had found sex differences in personality to be very small. By evaluating the data differently (at the univariate level, rather than compiling the data into “the big five” multivariate categories), they found that men and women have about a 10% overlap in personality traits. When they removed the single biggest difference (sensitivity, higher in women), they were still left with only a 24% overlap.

    If sensitivity is the largest single personality trait that differs between men and women, that might explain your attrition levels.

    What the appropriate action to take is, to my mind, NOT to put X times as many women as men through the pipeline. This is actually counterproductive. You already have a talented pool of women. Plugging 10 times more women into the pipeline, only to have them keep on giving up at the first difficulty… well, if role models are valuable, that’s only going to create 10 times as many role models for failure, that other women are going to see.

    What MIGHT just work (and no, I don’t think parity is a reasonable goal, but what might work for the industry and for talented women (and men)), is to just stop. Just stop with all the talk of possible bias in hiring. Stop with all the talk of the sexism in the industry. Just. Stop.

    If women are more easily discouraged than men, then what message are they getting when so much time and attention is being given to convincing everyone that they’re being discriminated against? That not only do they have to deal with all the same difficulties of computer science (it’s not a cakewalk, ffs), but you can add systemic sexism and old boys’ clubs and bias in hiring on top of that.

    You ever wonder if men are more confident (in part) because they’re constantly told they have male privilege? If you’ve been told all your life that you’re probably going to do fine because you’re a man, why would you quit after a first bad interview? For all you know, you could suck at the interview and still be hired into that privileged old boys’ club, just because you have an outie rather than an innie. Or maybe you stick it out because you’ve been told there’s no excuse for not succeeding–all that male privilege, yo. No one to blame there, and nothing to do but suck it up and get better.

    You ever wonder if a woman might have absorbed all of these messages about systemic sexism and anti-woman bias and blah blah blah from the culture, and then the first bad thing happens, that she might think, “OMG, all this sexism. I know they didn’t hire me because I’m a woman, because EVERYONE KNOWS the industry is biased against women. Therefore, because I can’t change the fact that I’m a woman [you know, the way a man might work to change his interview skills], I might as well give up.”

    Don’t you guys think that maybe, just maybe, the pervasive rhetoric around this issue might actually be contributing to it? I’m very pleased that you guys have published the results of this research. When my daughter graduated high school with a partial IB and A’s in almost everything, but with a particular gift for math, chemistry and physics, I TOLD her to get into those fields. The hiring bias is INSANE–in her favor. If she went into mechanical or chemical engineering, she could write her own paychecks. If she went into industrial radiography, she’d be pulling in obscene amounts of money a week out of school. Companies would be throwing offers at her 3 months before graduation.

    Right now, the tech industry can’t get enough competent women. They’re falling over themselves and each other to hire competent women. They’re doing it for all the wrong reasons, in my opinion, but that doesn’t change the fact that they’re doing it. It’s never been easier for a woman to get into a tech field.

    And then a massive percentage of them just throw up their hands after one or two bad interviews? Well, maybe they’ve internalized the more pervasive message, which is the exact opposite of what I’ve told my daughter?

    • user

      AUTHOR Panda

      Posted on 9:10 am June 30, 2016.
      Reply

      Bravo, Karen!

    • user

      AUTHOR Taka

      Posted on 11:05 pm July 1, 2016.
      Reply

      Once again, you are the best – so glad to see you on here.

    • user

      AUTHOR Estelle

      Posted on 3:50 am July 2, 2016.
      Reply

      Well said 👏🏾

    • user

      AUTHOR Garrett

      Posted on 3:05 am November 28, 2016.
      Reply

      I do believe this is what my high-school debate team – if I’d gone to a school classy enough to have one – would have called “special pleading”.

      Look, I agree that sometimes taking about a trauma can magnify the impact of that trauma. War veterans sometimes have to deal with this counterintuitive fact when deciding whether and how to get counseling, for example. But you might want to leaven your above criticism with some understanding of context: The people coming here are already a self-selected group. They’re STEM careerists looking for insights to improve their interviewing skills. If the take-away is that the field is totally unbiased, and that women are merely whiny quitters, isn’t that message _even_worse_ than one that (supposedly wrongly) teaches them to be vigilant of bias, in themselves and others?

      You’ve noted yourself that tech companies are falling over each other to hire women. Programming as a mainstream career choice has been around for about 30 years, give-or-take. That’s barely enough time to grow beyond the generation and the specific culture that spawned it. Perhaps tech CEOs and hiring managers are not being reverse-sexist idiots as you imply, but instead they are aware of how the field is _unnecessarily_ biased towards too-narrow of a culture that happens to favor a given gender, and are trying to leverage an untapped supply of coders by pulling them in and forcing them to advocate for changes that make them more comfortable, and by making those changes, paving the way for an even broader group in their wake.

      How many men found their way into the field by playing video games? How many corporate environments originally formed to cater to them, and now need to reorient themselves? Do you think the era of “booth babes” at trade shows is over? The company party I went to _last_month_ took place in a bar with go-go dancers in bikinis suspended in cages from the ceiling. It made _me_ uncomfortable, and I’m a man! It made my female co-worker furious and she filed a complaint. Good for her.

      How much more reliable could software be, if companies could raise their retention rate, not by simply throwing more money at the problem, but by establishing practices that are convenient for adult women and inspire loyalty, like on-site childcare, in-house charter schooling, flexible leave and vacation schedules, or really good telecommuting infrastructure? One new program that particularly interests me, is a partnership my company has established with a local college. Developers can teach, or assist in teaching, the programming classes at the college as part of their job duties. The program is in pilot and a much greater ratio of women signed up for it than the ratio of women to men in the company. I signed up for it too. Respect for a greater diversity of appearance and behavior improves the situation for unconventional men too.

      How many dev teams would function more smoothly if the macho fist-pounding gorilla types had to learn to hold their behavior in check because making decisions the Man Way is not even theoretically the norm now, and patiently building consensus the Woman Way results in fewer mistakes?

      Pointless and damaging gender and racial bias in the tech industry is NOT over. The transitions and adaptations the industry must make to maximize recruitment and retention are ongoing. We’re not at the point where “just stop talking about it” is the better tactic. If you believe we are, well, you’ve been reading too many scholarly articles and not getting enough ground truth.

  • user

    AUTHOR Joerg

    Posted on 12:24 am June 30, 2016.
    Reply

    First of all: thanks for the data and the article. I wonder if the gender can also be perceived by other aspects of the interaction then the voice, e.g choice of words, rhythm, and just generally the content of the conversation. Thinks “mansplaining” – it is a good clues that the person doing is male. Lack thereof might be a clue as well. Maybe, if interviewers are aware of the voice modulation, they tend to start listening even more for those “interaction clues”.

    Maybe it would be interesting to have the interviewer record what their perceived gender of the candidate is, and see if the rating correlates with that.

    • user

      AUTHOR Michael Knight

      Posted on 4:38 am July 3, 2016.
      Reply

      “Mansplaining” can ONLY be done by men. It’s in the definition. The same dismissive behaviour, when performed by women, is “explaining”

  • user

    AUTHOR Christoph

    Posted on 10:31 pm June 29, 2016.
    Reply

    First off, to make sure that there are no misconceptions about the points I will try to make below: When I talk about “men”, “women”, “boys”, “girls”, I talk about statistical means. I assume that the distributions I refer to are normally distributed, unless specified otherwise. On the other hand, that means that the individual person can be anywhere on that distribution.

    “Specifically, men were getting advanced to the next round 1.4 times more often than women. Interviewee technical score wasn’t faring that well either — men on the platform had an average technical score of 3 out of 4, as compared to a 2.5 out of 4 for women.

    Despite these numbers, it was really difficult for me to believe that women were just somehow worse at computers, so when some of our customers asked us to build voice masking to see if that would make a difference in the conversion rates of female candidates, we didn’t need much convincing.”

    People are different. People have different abilities and interests. Obviously, men and women are also different. Seeing that men and women are already different superficial (i.e. they look different), wouldn’t it be fair to assume they “function” differently, i.e. have different interests and abilities?

    Then why is it hard to accept that men might simply be more apt in computer related topics? Why should that conjecture be specifically excluded? It explains everything, AND it matches observations. Men might simply be more interested in technical stuff than women. Why is that bad? I still fail to see the point of the social engineering idea that instead of having people do stuff they are apt in AND enjoy, to push them into something they might not. Studies have shown that women in rich countries tend to be LESS likely to go for technical jobs, BECAUSE the economic and social security enables them to do what THEY want. So also data proves that women apparently simply do not want to deal with technical issues, if they can freely choose not to. That is sufficient to explain the employment gap in STEM (in western societies). I sincerely ask why we should ignore that huge elephant in the room and try to make up experiments to find evidence for a conjecture which is created specifically from the mindset “but, that cannot be!”. Maybe it can be and is so, and that’s fine.

    Why do we “need” more women in STEM? In my job, I want to have colleagues who know what they’re doing. What gender they have is not important for that. And if it so happens that women are not interested and maybe not equally able do work in tech, what’s the big deal? I think the real issue in western societies is that jobs which are actually supporting the welfare and stability of the society are done to a large extent by women, and are massively undervalued (by pay, which also explains the mythical gender pay gap, which doesn’t account for job fields etc.). That’s the issue that needs fixing. Fighting to gain a foothold in STEM, because apparently wages are higher there misses the point. What women and feminists should fight for, is that jobs women are actually interested in and doing to be valued much higher than they are right now. That would be a fight which goes hand in hand with the interest of women (i.e. the bulk of the distribution), and not the small fraction which is interested and able to work in STEM.

    Fight for better pay and conditions in hospitals, kindergartens, schools, etc! The rich investor can only do his job because there is a huge workforce of underpaid people who keep the real life of his back, so that he can play casino at the stock markets. Show them who’s really running daily life!

  • user

    AUTHOR Emelio Lizardo

    Posted on 10:30 pm June 29, 2016.
    Reply

    Not surprising. For men the career is everything. No career, no life.

    For women, work and career are choices. Mom is the career of choice or last resort. Because men will support her.

  • user

    AUTHOR Yreva

    Posted on 9:36 pm June 29, 2016.
    Reply

    I remember hearing about interviewing.io on NPR. I think the idea is great but the execution is very poor. The pitch of a “male” or a “female” voice isn’t the only distinguishing characteristic. Men and women simply speak differently: inflection, intensity, duration, diction, etc. etc. etc. make it pretty easy to distinguish a male speaker from a female speaker, even when the voice is “masked.” In all three interviewing.io voice-modulated examples I’ve heard, it’s been pretty easy to tell if the speaker is male or female (yes, without knowing the speaker’s gender beforehand). I would be surprised if most, if not all, interviewers didn’t figure out the gender of the interviewee pretty early on in the conversation. I’m wondering if interviewing.io is more a good PR move for employers rather than an effective tool for increasing diversity in the workplace. Has interviewing.io considered this? Have you done any research on it? (Play a few voice-modulated examples for a focus group and ask them to guess the speaker’s gender?)

    • user

      AUTHOR Freeflight

      Posted on 4:28 am June 30, 2016.
      Reply

      Sorry but I do not agree on your thesis that men and women can be “easily” distinguished based on their speaking habits.

      Linguistics is a complex subject and peoples way of speaking gets affected by many more things than just “gender”. Being a native speaker or not, what native language somebody speaks (habits from that translate to English), where somebody grew up and so much more.

      Do you have any studies going into that topic? The only thing I found, being from some “Europe-University Viadrina”, called “Language and Gender”. But that’s neither a study nor does it look in any way useful or scientific, it’s just a random collection of quotes while the author makes a lot of very weird claims, without actually sourcing them in any way.

  • user

    AUTHOR Max P

    Posted on 8:46 pm June 29, 2016.
    Reply

    This got me wondering as to how much the propensity for males to engage in risk taking and reward seeking behaviours as compared to females, may be a factor in the results you are seeing. As well as differences in competitive behaviour. Given that your platform has essentially gamified the process of technical interviewing, it may be that this “game” stimulates male dominated behavioural tendencies more so than female tendencies. It may be that the perceived rewards for continued participation in the process after initial failure are not a great enough motivating factor for females, while the failure and success (risk and reward) mechanics of the “game” provide greater motivation for males to persist in seeking reward in-spite of their initial failures.

    Your platform may be a good analogue for some of the differences seen in the gambling behaviours of males and females. Specifically the relationship between failure and re-engagement.
    Males dominate females in terms of the number of regular gamblers. Males having a tendency to begin gambling at a younger age and being more likely to become problem gamblers. Behaviours that can be beneficial in some circumstances can be detrimental in others.

    What I am suggesting is that this may go deeper than an inability for one to dust oneself off after an initial failure (i.e. personal shortcomings) and may have some ties to instinctual behavioural characteristics. Food for thought.

  • user

    AUTHOR Robert Ancell

    Posted on 6:39 pm June 29, 2016.
    Reply

    Did you do any checks to confirm the modulation worked well? It would be interesting to ask the interviewers afterwards what gender they thought the interviewee was. There may still be some bias based on what gender the interviewer thinks they are talking to.

  • user

    AUTHOR reaVer

    Posted on 6:14 pm June 29, 2016.
    Reply

    So basically, employers are discriminating against men…

  • user

    AUTHOR Rhi

    Posted on 5:33 pm June 29, 2016.
    Reply

    You mean to tell me that women doubt themselves quicker than men… No, how could it be? Women have always been taught to not put themselves down… no, wait, just kidding. Well, look at it this way, at least they were not asked if they were planning to have a baby anytime soon. Always a win-win!

  • user

    AUTHOR IamMyself

    Posted on 5:26 pm June 29, 2016.
    Reply

    Methodology is a little weak – all the participants knew they were involved in a study, and you did provide a sort of gender indicator. Since the gender imbalance issue is widely known, you need to control for the possibility of interviewers consciously attempting to be non-discriminatory. The unexpectedly higher ratings for the higher pitched mask and lower for the lower pitched mask potentially indicated that kind of behavior.

  • user

    AUTHOR John

    Posted on 5:04 pm June 29, 2016.
    Reply

    Nitpick:

    “to get to pipeline parity, we actually have to increase the number of women studying computer science by an entire order of magnitude.”

    Your formulation makes it sound like you’re advocating for this as the solution to the problem. However, as you later mention, fixing the attrition issue is probably a more effective, more realistic solution.

  • user

    AUTHOR Jesse

    Posted on 4:13 pm June 29, 2016.
    Reply

    I think this is a great start but shows that the problem needs to be analyzed on a more fundamental level. From what I see the problem is on the interviewers side not the interviewees! I think if companies focused more on the candidates people skills you will find the gender gap decrease.

    • user

      AUTHOR Freeflight

      Posted on 10:57 pm June 29, 2016.
      Reply

      How is it the inverviewers fault, when they interview for the technical skills required for the job, instead of focusing on “people skills”? A job requires certain talents and skills, some more, some less so.

      These requirements are not made up on the spot, but based on what a company is looking for. When they are looking for tech-adapt people they will weight the tech skill more heavily, when they are looking for PR/sales people (or other positions with a lot of people interacting) they focus on people skills.

      In essence you are arguing for “lowering the bar” by weighting what’s actually important less (technical skills) while weighting not-so important skills (people skills) more heavily.

    • user

      AUTHOR JC

      Posted on 12:07 am June 30, 2016.
      Reply

      How the fuck is this “on the interviewers side” when the voices were modulated so they were all men?

    • user

      AUTHOR Waffle

      Posted on 2:05 am June 30, 2016.
      Reply

      >From what I see the problem is on the interviewers side not the interviewees

      But why? That was the original assumption, hence the voice modulation approach. It did not change a thing, leading to a conclusion totally opposite to yours.

    • user

      AUTHOR John Horst

      Posted on 7:11 am June 30, 2016.
      Reply

      I have had to give about 20 technical interviews over the past 12 months. I am always looking for skills and problem-solving abilities first. Man or woman, I have no room for someone who doesn’t get it. I may be wrong, but I try to level-out my expectations because women (especially Indian women) tend to be less assertive. Sometimes much less. Encouraging someone to speak up is very important when interviewing, because code doesn’t challenge you or embarrass you (or judge you, either). IOW, I don’t care if someone is shy or lacks certain people skills — can they code?

  • user

    AUTHOR TM

    Posted on 4:10 pm June 29, 2016.
    Reply

    As a transwoman who transitioned 20 years into her engineering career, I can fill in some anecdotal information on this (if not broad data.) I’d like to begin with saying that I love real data and analysis going into issues like this. I’m glad this experiment was performed and the data fully presented even though it conflicted with the original hypothesis. I learned a good bit reading this.

    First, real data. There are studies that have found that resumes with female or black sounding names are rejected more frequently, anonymizing the name removes the bias. There is a gap at the very beginning of the pipeline even before interviews taken place.

    Second, during the personal interactions before, during, and after interviewing I find that I am “talked down to” far more frequently. What I mean by that is simple concepts are being explained to me, often in painful detail, before the questions are fully asked. As one example, when someone takes the time to explain how sporting games work as a preamble to design discussions involving information about them, that’s pretty obviously biased, that had never occurred before.

    Third, I have noticed that my appearance and how I dress influence how I am treated. If I wear a bit more makeup, wear nail polish, dress more fashionable, and curl my hair, I am treated far more patronizingly than if I wear the “standard geek wear” of jeans, t-shirt, hair in a ponytail, and virtually no makeup. I have heard the same feedback from all of the genetic woman I have asked about this, it is a common experience. Women know they need to dress down to be treated seriously.

    Fourth, I have noticed that the results for me have dramatically changed. In the past when I interviewed, I received offers for nearly every interview I had, about 7 out of 8. Now, I receive offers in about 2 out of 8. My technical skills have not deteriorated that dramatically in just a couple of years between looking for a new position. I have now received feedback along the lines of, “While your communication skills are excellent, we feel that you aren’t very technical.” I have never received a single comment before even hinting that I’m “not very technical.” I have peer-reviewed technical publications that are referenced by others. I have been a senior principle engineer in the past and architected systems on very large scales (Amazon amongst them.) It is hard for me to reconcile that these differences have nothing to do with gender.

    Fifth, the offers I have received are often for lower titles than I have had in the past, there is a well documented bias that men are promoted based on potential when they haven’t done the job and that women are promoted only after they have done the job for a time. In coming from a principle position, I have literally heard, “well, you haven’t been a principle here before, it’s much harder here, let’s try you out in a senior engineer position.” Seriously? A senior principle position at Amazon responsible for the design of a well known and publicly used product earning billions of dollars is literally much easier than a senior engineer position elsewhere? I doubt it.

    Sixth, rejections based on “not a good culture fit” occur. I’ve never heard that before. Ever. When you hear that from a company with 60 male engineers and 2 female engineers, you know exactly what that means. There’s a wonderful article about Paypal that they had problems finding a woman with a good culture fit. Later in the article it mentioned that they frequently solved design discussions by wrestling in the aisles.

    Seventh, woman are far more prominently represented in testing or quality assurance engineer roles, in full software engineer positions there are fewer women. Women tend to be shunted to the “less technical” positions.

    Eight, I’d never met a female principle software engineer before being hired as one. Everyone around me says things like, “I knew one once. Hmm, I think the fact that I can remember the single one is the problem.”

    Having literally been on both sides of this equation and speaking with other technical transwomen who have experienced the same differential treatment, appearance does matter. With voice only, the technical results may be the same, but the interviewer is free to imagine the appearance of the person being interviewed. It’s not just about perceived gender, it’s about the appearance of the specific person, body language, and other hard to quantify social cues. The treatment in a technical question completely divorced from the social interaction surrounding an interview does not necessarily reflect the questioning and results of real interviews. It also fails to reflect the pipeline, offers, and other aspects of the process.

    It’s also amusing to hear long time male friends hear my experiences and try to mansplain to me how they haven’t happened or aren’t valid in some way.

    This article in no way proves that we can all pat ourselves on the back because there is no bias. It proves that significant in-roads have been made into a single aspect of the process for which I am grateful and it shows that continually investigating the process does have positive results.

    • user

      AUTHOR Rhi

      Posted on 5:38 pm June 29, 2016.
      Reply

      ^^^^^Why I love being my own boss and having my own business. Ole! And if I ever crash and burn, I’d prefer living in a shelter. More humans in a shelter. Heh.

    • user

      AUTHOR Bernd Paysan

      Posted on 6:15 pm June 29, 2016.
      Reply

      I assure you, as male geek/nerd, you have to dress down to be taken seriously; it’s just not makeup and lipstick. It’s T-shirts and shorts instead of suits and ties. I think you know that ;-). It’s just a new experience for you that as female nerd, you have to dress down where you don’t actually wear a dress literally.

      I don’t know if you do hormone therapy, but many trans people do, and the hormones of course do affect you. The therapy has many effects on your brain, see e.g. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2879914/ even though your history as man has left its traces – you can’t switch over to a full “female” brain by taking estrogen, your brain will stay partly male. But due to this therapy, you are not exactly the same person as before, and that’s the point of transgender: you wanted to become a different person. Pretending you are still the same, now in the “correct” body is not honest.

      Let’s phrase it differently: We have a number of challenging fields like medicine or law, which were dominated by males 50 years ago about as much as IT is today. They have shifted to a somewhat female domination, at least in the younger generation. The culture of these male-dominated fields were just like any other male-dominated field, because that’s how men work – I don’t want to say that this is good or desirable, but that it did not prevent women to enter the field gradually to eventually dominate it. Prejudice and culture can change. These fields are difficult, too, but the challenges are different.

      Therefore, we have to continue to investigate the reasons for this particular gender gap, and not use “prejudice” as single reason. There is prejudice, of course, there always is, but as a single explanation (“there’s a gender gap, because men are full of prejudice”) it is just an example of prejudice at work – looking for simple explanations, and wildly extrapolating. And while prejudice alone means that the process of women entering any male-dominated field is going to be gradually (not over night), in the long run, prejudice can be overcome. The non-prejudice parts that cause this gender gap – if they exist, can not be overcome that easily.

      I’ve seen the opposite report of what you say of a transman; he said he told people that his former female self was “his sister”, and people told him while he works in the same field as his sister, his work was considerably better. Maybe it was prejudice, or maybe it really was better, and the hormone therapy was the cause. We have to find out. IMHO it’s to easy to blame prejudice alone.

      I also know a transwoman who I consider does a good job as software engineer, but I think of her more as cross-dresser, because her male features of face and body are too strong (big nose, wide shoulders, voice too deep), and I know she was a man before; she’s open about that, and our community accepts that. I do have the impression that she is considerably less self-confident than men, and that does affect her in-person presentations. It does not affect online, written discussion.

      It could well be that this effect is measured here, because even when your voice sounds male, all the signs about your ego will still go through unfiltered. If you show that you are not very confident about what you present, you’ll be rated down. If you have a female voice (or appearance) and show less confidence, people will adjust for it somewhat, if they know that this reduced self-confidence does not match with the actual skills.

      • user

        AUTHOR TM

        Posted on 2:52 am June 30, 2016.
        Reply

        I do not do hormone therapy, my brain has not changed because of that, and I pass perfectly well without it. I have confirmed with friends that the differences in my behavior is that I am more happy and if anything, my confidence has gone up. My technical performance has improved as well with the reduction of my anxiety.

        Are you transgender? It doesn’t sound like it. If you aren’t please don’t mansplain to me what the “point” of transgender is. Your understanding as expressed in your words is deeply flawed. I am in the correct body and I have never wanted to be a different person. You do not understand, please do not think that your outsider knowledge of me is somehow more comprehensive than my insider knowledge of me.

        Instead of addressing my actual points you tell me how my experiences and opinions are invalid. Your first reaction is to tell me how I am flawed based upon your incomplete knowledge rather than thinking that I might have spent a lot of time throughout my life researching, learning, and observing how the interplay between women and men works. That knowledge has been very important for my actual survival. I bet far more time than you have spent doing the same.

        You even spend time informing me that in the past there were male dominated occupations that are now becoming more female dominated because of the revelation that prejudice and culture have changed. Do you seriously think I don’t know that? Or that your conclusions are somehow profoundly insightful? Please reread that paragraph and try to see how patronizing it sounds if someone took the time to tell you those things?

        Being transgender for my entire life and a scientist with degrees in biology, medicine, mathematics, and computer science with extensive experience designing, running experiments, and analyzing the resulting data I have experience in how to properly determine what is going on and how to gather data. I have done numerous experiments by varying certain aspects of the interview process, my looks, my voice, the perceived gender of my name, etc because I am fascinated at the differences that have cropped up. I don’t just interview to find a job, I have interviewed far more than I need to as part of my own private interest in determining what is going on. It’s all anecdotal because of the difficulties of controls in sociological experiments and the small sample size, but fascinating to me in the possible patterns it exposes.

        It would have been nice if you started to discuss the sociological context of how men and women are raised, societal pressures and expectations, inherent biases in numerous systems, or any number of other possible influences and how that might have any impact in this. Instead your “helpful explanation” is proof positive that you believe you can instruct others about the validity of their own experiences. You don’t even comprehend that you might have a more incomplete view of what is going on than the person experiencing it and that may cause you to perceive things inaccurately. Please read and try to see things from a point of view other than your own, you might learn something because other people do have experience and knowledge you do not.

        As a very personal note to help a trans person in your life, please try to not think of the transwoman as a crossdesser because of your opinions about her, especially her appearance, calling a trans person who has transitioned a cross dresser is deeply insulting on numerous levels because of the connotations it has and the perceived validity of her motivations. I have numerous friends who are cross dressers and there is nothing wrong with cross dressing, but because of historical reasons calling a transwoman a crossdesser is very insulting.

        • user

          AUTHOR Overtaxed

          Posted on 4:55 am June 30, 2016.
          Reply

          TM,

          I’m going to say this the nicest way possible, but.. Most employers don’t want to deal with the constant explanation that comes with having a trans employee. Especially for anything “customer facing” the fact of the matter is that, in much of the world, a trans person is going to have a very hard time “fitting in”. And no, your job shouldn’t be all about conforming, but it does matter. Don’t show up for an interview as a plumber wearing a 3 piece suit. And don’t show up for a banking interview wearing overalls. You have to look/act the part. And, being trans is not acting the part in most jobs (and nearly all IT jobs). It’s akin to me going out and getting a face tattoo and being upset that people won’t hire me for executive positions anymore.

          • user

            AUTHOR TM

            Posted on 12:49 pm June 30, 2016.

            I work in backend system design and do not do customer facing work at all.

            Also, no one knows that I am transgender in the work world. I have kept that side completely segregated and do not reveal or call it out in anyway.

            And before you say that people figure it out on their own. I pass so well that other transgender people believe that I’m a genetic female – I’m not easily read by the people who are the best at detecting it.

            To further put that aspect to rest, I have eventually ran into co-workers who are straight, gay, lesbian, and transgender outside work in social settings and even after years of working with me, every single one told me they never knew or even thought I was transgender. Again, I’m very hard to read.

            Perhaps you have seen or noticed the more easily read transgender people, but you have not detected many of us who easily pass and are currently around you. Again, your experience and perceptions are incomplete, it would be nice if you didn’t assume you’ve seen it all.

            I have had multiple instances of being hit on during interviews. Does that ever happen to men? Not in my experience. Do you think that may influence their opinion of my skills or whether to hire me? Even an obvious wedding ring does not prevent it. That’s another trick women learn, always wear a wedding ring to an interview, even if you aren’t married, it reduces but does not eliminate the occurrence of being hit on. Do men need to take that into consideration? I bet you never even thought that these were things that women had to take into account because you have never experienced it. You have not seen it all or even heard of it all.

            And to forestall the statement that “things are getting better” with regards to the younger generation, I’ve been hit on almost entirely by engineers in the 25-35 year old range. The growing “brogrammer” phenomenon has increased the problem, not reduced it.

            I never wore anything but a suit to engineering interviews before, I spent most of my life and work on the East Coast often in financial positions, it’s expected. I have never worn a suit as a woman to a West Coast interview although I did as a man.

            On the West Coast wearing a demure, completely covering and slightly baggy sweater and a long skirt with your hair down is enough to influence the results – it also increases the chance of being hit on. The “uglier” you make yourself, the better the chances of being hired – wearing natural looking makeup that washes you out and emphasizes the worst features helps your chances.

            Please tell me how this is a shared experience and there is no bias and everything is hunky dory in the women in tech world.

          • user

            AUTHOR Jemma

            Posted on 1:25 pm June 30, 2016.

            You might want to go back and reread everything she wrote, then delete your comment. For starters, “principle software engineer” isn’t exactly a customer service job.

            The rest of your comment is so awful, I dunno where to start with it.

          • user

            AUTHOR Garrett

            Posted on 3:32 am November 28, 2016.

            Wow. You start out with “I’m going to say this the nicest way possible,” and then produce a degrading turd of a lecture like that? Please remove yourself from the internet.

    • user

      AUTHOR V Y

      Posted on 5:33 am June 30, 2016.
      Reply

      I just wanted to say to you, TM, that I really appreciate your thoughtful informative comments and your incredibly tolerant responses here. They were fantastic to read. Thank you so much. You inspire.

    • user

      AUTHOR Taka

      Posted on 11:12 pm July 1, 2016.
      Reply

      Thank you for a view from the other side – this is a good datapoint to have as a counterargument. I appreciate you sharing this with us.

    • user

      AUTHOR Garrett

      Posted on 3:39 am November 28, 2016.
      Reply

      Finally someone in the conversation with some real ground-truth experience! Thank you for being here.
      I implore you to turn your attention to Karen Straughan’s comments here, and give them some much needed pushback on their claims that computer science is inherently “men’s work” and that the best thing we can do for women entering the field is “stop scaring them away by claiming there is bias”.

  • user

    AUTHOR jupitaur

    Posted on 3:42 pm June 29, 2016.
    Reply

    Maybe women are less likely to go through a bunch of work-based rejections because their experience is that rejections are going to just be followed by more rejections and never an acceptance. Maybe women give up because it’s the rational thing to do after doing well in interviews and never getting called or always coming in second place.

    You can’t assume all other conditions being equal for these women because they’re not.

  • user

    AUTHOR Sarah

    Posted on 3:39 pm June 29, 2016.
    Reply

    This is a harsh group of critics! Interesting work for sure and I’m glad you included the last bit because as soon as you mentioned finding that disparity I wanted to know what the data would look like when corrected for attrition! Very cool insight, thanks for sharing.

  • user

    AUTHOR matt

    Posted on 3:37 pm June 29, 2016.
    Reply

    My two cents: Women tend to be more self deprecating, act more humble and downplay their skills, while men tend to act more confident and exaggerate their skills. Is it possible that this subtly influenced the interviewers?

    • user

      AUTHOR paul

      Posted on 9:40 pm June 29, 2016.
      Reply

      Absolutely
      If they ask how competent are you at doing task X , and you say you are excellent at task X, that sounds better than saying you are merely good at task X.

    • user

      AUTHOR karen straughan

      Posted on 1:26 am June 30, 2016.
      Reply

      I think it would be more likely to influence the interviewees.

      Women were slightly favored by interviewers. That’s entirely likely to be about diversity hiring practices. But despite being getting equal or better results on interviews, women gave up more often. It’s the giving up that is, at least in part, applicant for applicant, contributing to the dearth of women in STEM.

      So here’s my two cents: Do you think women might believe they’re being discriminated against when they aren’t? You know, because of cultural narratives? And that because they believe they’re being discriminated against based on something they can’t change (their sex), they might give up more easily than men? That they might blame some outside force they can’t change (sexism), rather than the fact that they just need to hone their interview skills?

      That is, do you think maybe so many women give up after one bad interview because there’s a Greek Chorus floating around them telling them they might as well give up? No matter how much they apply themselves, men will still be favored just because penis (when the opposite seems the case, at least in this field)?

      • user

        AUTHOR Garrett

        Posted on 4:12 am November 28, 2016.
        Reply

        “Hey, I’ve got an idea, let’s all shut up and pretend sexism doesn’t exist in the workforce, in order to protect the delicate sensibilities of the women applying for jobs.”

        _Now_ who’s the one advocating for sissification and coddling?

        Also, do not make the mistake of extrapolating this website’s experiment, with its tiny, TINY sample size, as saying anything even close to truth about the tech industry in general. From the evidence at hand it makes just as much sense to conclude that the high attrition rate of females using this site is because a recent ad campaign they ran attracted a number of casual female users that swamped the even tiner set of serious female users signed on through other means. These numbers are so small they could be derailed by practically anything. An interviewing.io blog post trolling through a Facebook knitting forum could derail numbers this small.

        Of course, if you have larger, more controlled studies that back up this attrition difference, I’m eager to see them.

  • user

    AUTHOR CV

    Posted on 3:25 pm June 29, 2016.
    Reply

    There is a nice documentary about how this gender gap is mostly because of biology
    One of the studies found that how wealthier the country is, how bigger the gender gap will be because people will study the things that they find intresting, while in poor countries (uganda, ghana) the gender gap decreases because the technical jobs will have a bigger chance at a better life

    link to the documentary

    • user

      AUTHOR Rhi

      Posted on 5:53 pm June 29, 2016.
      Reply

      while it’s true that women nurture, share, engage in non-technical careers, it’s also true that those ‘female’ careers have always been looked down on since men ‘allowed’ us to venture out of the kitchen, and are paid less partially on purpose and partially due to economics.

      while I am not trying to guilt trip men, and this isn’t even about them because I personally don’t hire men anymore because they lie too often about their skills and make me lose time and money far more often then women, I will tell you that we are all biologically even influenced by the past. Just ask a male scientist.

      and sure economists such as Sowell, Williams will say the gender gap is a myth because women pick different careers, women take care of their kids, they work less, and all other variables and whatnot, the fact remains that some of us, ‘insecure’ women, don’t actually need a PHD in liberal economics to comprehend that even JK Rowling had to hide her full name before publishing because boys don’t relate to women or girls writers and would not buy the book. And publishers are in the industry for the money, not for illusions of ‘equality’ in any way, shape, or form.

      Do you really think that after centuries of major three religions which in their religious texts, images, councils, and among leaders have centuries portrayed women as ‘less’ spiritually, and physically than men, and that just because we have the internet and toilet paper with scent in the West these days male world will suddenly drop their unconscious and conscious bigotry? Yeah, right. And the witch hunts never happened.

      • user

        AUTHOR Vindi

        Posted on 2:02 pm June 30, 2016.
        Reply

        Maybe engineering is harder than liberal arts and produce more, maybe there are reason why pediatricians are paid less than surgeons? No it must be discrimination against women. Surly jobs that produce majority of work facilitates are not dangerous and are paid premium, no men are just dumb.
        Have suggestion for you. Try fit conclusion into data not other way around.

  • user

    AUTHOR anon

    Posted on 3:01 pm June 29, 2016.
    Reply

    Is attrition related to women feeling more of an impact, or is resilience in the face of attrition events a matter of interest and need? If men have a stronger desire for the high paying position then women, such as would be the case if women have alternate means to acquire high spending potential and men do not, then women would have less reason to put up with attrition events. If men have more interest in the subject matter, they would have an easier time shrugging off attrition events.

    Despite earning only 43.5% of the money earned in the US by individuals, women spend roughly 60% of the money spent in the US by individuals. Clearly women are receiving money via additional avenues not available to men (and men most even be supplying these additional avenues). As such, they have less reason to seek a higher paying position.

    Additionally, differences in the nature of male and female interests, whilst not absolute by any means, have been shown to be manifest long before society can act on them. The key difference in these interests is between the social and the physical.

    Together these two effects will account for far more of the gender gap than discrimination. Indeed, discrimination against the direction of the gender gap is highly unlikely to overturn these effects.

  • user

    AUTHOR Jack

    Posted on 2:52 pm June 29, 2016.
    Reply

    Purely conjecture, but could this issue be something that starts even at the beginning of Computer Science education? Do women tend to give up on CS more than men when struggling, resulting in a gender gap that grows wider and wider as you progress through the education levels?

    This could possibly explain why the industry has a large gender gap, because so many are weeded out before they even sit down for an interview.

    • user

      AUTHOR Bernd Paysan

      Posted on 6:26 pm June 29, 2016.
      Reply

      I didn’t struggle when I studied computer science. It was easy. I left the “difficult” math exams after half of the allotted time to still get full points. Many girls dropped out because the required math was “too difficult” for them. Of course, about half of the boys dropped out after the first few exams, too. It is difficult, but you know when you are good enough to pass the tests. The few girls who stayed passed the final exams easily, too. All the people who stayed until the final exams where those who didn’t have big difficulties. That’s the point of these exams. It’s not like you don’t know if you are good or bad. If you are good, you stay, if you are bad, you quit. If you don’t get it, you’ll be thrown out by not passing the exams.

      Nowadays, some universities make the exams easier to get more people out, which also closes the gender gap partly. The industry reacts by doing the same exams as 20 years ago in their interviews, because they want the same level of quality.

  • user

    AUTHOR Nick Brosnahan

    Posted on 1:47 pm June 29, 2016.
    Reply

    Did you modulate the interviewers voices as well or just the interviewees? I’m curious to see if you would see a change in men dropping out when they perceive a female interviewer vs. women dropping out if they perceive a male interviewer. It’s not clear from the article if this was just one-direction or it was both directions.

    • user

      AUTHOR Aline Lerner

      Posted on 4:38 pm June 29, 2016.
      Reply

      We didn’t modulate interviewer voices this time, but it’s definitely something we’d like to try in the future.

  • user

    AUTHOR Parson James

    Posted on 1:41 pm June 29, 2016.
    Reply

    Obviously, we need to lower standards and impose hiring quotas for women in this sector. Otherwise, any other course of action would be misogynistic and biased.

    The last thing we would want to do would to be hold women accountable for their actions and ascribe to them some degree of agency in their decisions to give up early in this context.

    That would be sexist. They aren’t decisive, free-willed humans after all. They are women fighting the all-invasive Patriarchy which robs them of agency and accountability.

    • user

      AUTHOR Bharath

      Posted on 3:47 pm June 29, 2016.
      Reply

      You are exactly the kind of misogynistic tool, other men like me are ashamed off. Being a smartass doesn’t solve societal problems. We can’t ignore 50% of the world’s population because they don’t belong to the your gender. Try to help your peers deal with failure rather than criticize them.

      • user

        AUTHOR Parson James

        Posted on 5:04 pm June 29, 2016.
        Reply

        Genius, women control the majority of private wealth in the US, get sentenced more lightly for the same crimes as men do, and yet white women are the majority of beneficiaries of affirmative action in this country. To allude to some mythical discrimination against women in technology while job seeking in the US is idiotic, given the incentives, set-asides, legal diversity requirements, etc. That are in place pushing companies to hire female employees. If women don’t like being rejected, then they need to “woman up” and learn a little resiliency. Perhaps a step in that direction would be removing those hiring incentives, set-asides, etc. so womenow actually have to compete based on their skills and merits.

        On top of that, women in technology in the US are overwhelmingly from Asian countries where feminist coddling and social engineering are nonexistent, and that is a very generous description of that part of the world.

        Crappy social science as embodied in this “experiment” seems to rule the day in America. I for one welcome my fellow hard scientists from Asia, male or female, as they at least have a sense of rigor and analytics skills wheneeded it comes to distinguishing science from ideologically-driven BS.

      • user

        AUTHOR Peer

        Posted on 12:35 am June 30, 2016.
        Reply

        Kind of funny that you call the parent misogynistic, because what he describes is pretty much what those 3rd wave feminists and genderistas always claim.

      • user

        AUTHOR vortex

        Posted on 12:57 am June 30, 2016.
        Reply

        Hey Bharath, strawman much? Nobody said to ignore women. Parson just promoted the idea that merit and independence is not limited to men. The pandering of internet white knights to women often results in the infantilization of said women, but “the other” is the sexist.

        You sir, are trained like pavlov’s dogs, thowing around “misogynistic” “smartass”, instead of trying to comprehent what Parson was all about. You are the tool, and not a very sharp one at that.

        pls no bully, english is my third language

    • user

      AUTHOR Rhi

      Posted on 6:06 pm June 29, 2016.
      Reply

      Don’t worry, sweetie, nobody female wants your dead-end sheep job where your frustrations accumulate to the point you have to vent it out online.

      • user

        AUTHOR vortex

        Posted on 2:31 am June 30, 2016.
        Reply

        More like leeching of, ey? But hey, lets kick in the face of the one man that is actually trying to promote treating women as adults, instead as overgrown children.

        Welcome to upside-down land, were the constructive thinkers are treated like plebs, and the idiots set the framework for everybody else.

        Gods, i hope that all men/women in those “dead-end sheep jobs” would just sit down, and do nothing. Just for a week. Just starve these ungrateful parasites and their high horses alltogehter.

        Instead of compassion with those wageslaves, on wich your way of life depends on, you show nothing less than contempt and malice. No wonder MGTOW is a thing.

        I hope you manage to pull your head out of your a**, and overcome this bitter snarky persona of yours. Cheers

    • user

      AUTHOR JC

      Posted on 12:11 am June 30, 2016.
      Reply

      Damn straight! lets put MORE quotas on companies to force companies to choose candidates less skilled for the role. These people are just “giving up more easily” that’s all. You can still hire them, it’s not like they’ll give up easily during the job either!

    • user

      AUTHOR vortex

      Posted on 12:43 am June 30, 2016.
      Reply

      Right on the money Parson. Although other men try to shame you, based on their ideology of “everyone is equal in every way”, stay the course. Treating women as adults is not sexism, its respect.

  • user

    AUTHOR Alfred Meierberger

    Posted on 1:36 pm June 29, 2016.
    Reply

    Hey, I’d like to point out a few things here.
    1. There is no one who wants to hinder women studying tech. CS (Computer Science) students would love to have more than 3% women (CS student myself).
    2. No one thinks women are inherently unqualified for tech jobs, on the contrary (except maybe old and open sexists, but they do not make up any significant percentage)
    3. Where the f*** did you pull that last chart from? I mean you clearly stated it is hypothetical (thanks for that) but for everybody just looking over the article it suggests you found a huge gap, which you did not.

    But, why are there so little women in tech then? Well, let me ask a counterquestion: Why are there so little men in linguistics? Because men are less interested in language studies. And telling by experience, lots of women are just not that interested to study CS. Why? Maybe because there is so much bias towards tech people (Nerds, geeks, sweaty, unwashed long hair and living in mum’s basement) which deters women. But that is just something time will fix, and these stereotypes will go away as the stigma associated with tech fades.
    My Conclusion: There is no systemic bias towards women, but women are biased towards tech.

    • user

      AUTHOR Eisboer

      Posted on 1:44 am June 30, 2016.
      Reply

      Gosh. You just read a scientific study about that topic and even they didn’t jump to conclusions. Why do you need to speculate from your own very limited experience? Your whole text has no impact at all, because it’s not at all based on any scientific principle.

      If you think “women just don’t like tech” try to do a study about that. Find out if it’s really true, and then present your findings. Else you will only convince people who are already on your side.

      • user

        AUTHOR Christoph

        Posted on 11:04 am June 30, 2016.
        Reply

        Since there are no barriers of entry for women to go into STEM (there is even encouragement!), and there are still far less women in STEM, and women generally have good enough grades to learn or study anything they want, what would be your conclusion?

        A. People choose whatever career or curriculum they like best and are most interested in. Since we observe less women in STEM, that means women are less interested in STEM.

        B. There are multiple invisible forces at work, which hinder women to go into STEM. Since we observe less women in STEM, the evil forces must be discovered.

        A is based on a positivists world view on people. People have free will and want to pursue their own aims and happiness, if sufficient social and economic security allows them to do so. We observe less women in STEM. The simple, Occam’s Razor explanation is that women don’t want to go into STEM.

        B assumes that there are “hidden” forces, like biases and hurdles for women to go into tech. That’s far harder to test for, and extremely hard to prove. And since that is not my hypothesis, I am not forced to provide any “counterproof”. I have yet to see any conclusive study which shows that. Just finding low numbers of women in STEM, while there are more women in universities in general, doesn’t prove that.

        If two (or more) hypotheses can explain a phenomenon, you should pick the most rational and simple one. Assuming that people have free will and do what they want is simpled and more based on actual experience and science than invisible forces. So for now I will believe that A is the correct explanation, until someone finds more observations AND proves that A cannot explain the observations. That’s actually how science works. Just making up more and more complicated hypotheses, but failing to show how the new ones are explaining the observations better is not science.

        The “study” you asked for is to investigate what school children want to do as a profession after school. I only found a German newspaper article about that for now, but I guess you can get the idea from the graph:

        http://www.spiegel.de/schulspiegel/berufseinstieg-schueler-mit-berufswahl-ueberfordert-a-1004735.html

        They asked 500 students. A whopping 2% of girls want to go into a technical profession, with 27% of boys. 6% of boys want to go into IT, with 0% of girls. Girls predominantly want to go into social jobs, like medicine and teaching.

        Now of course one can assume that there the biases against girls/women happen earlier, in schools. Actually finding out WHY girls don’t WANT to go into tech is a study worth investigating, not some unprovable invisible bias. But maybe it’s just as simple as “they don’t want to”, and that’s it.

  • user

    AUTHOR sharon

    Posted on 12:02 pm June 29, 2016.
    Reply

    I believe gender impacts how we interview by more than just the sound of our voices. Women behind voice modulation that causes them to sound like men may still lack the swagger/confidence in their interview that men often exhibit, leading one to believe they are less qualified than their male counterparts. Because we are learning to account for that as interviewers, I wonder if the voice modulation does actively hurt women’s perceived performance.

    • user

      AUTHOR Savvas

      Posted on 4:39 pm June 29, 2016.
      Reply

      I think you are right and that the attrition of women suggests exactly that. Especially so, if you consider the fact that after a couple of interviews performance is the same.

    • user

      AUTHOR Savvas

      Posted on 4:42 pm June 29, 2016.
      Reply

      I think you hit the nail on the head here and that does explain attrition AND that after a couple of interviews performance is similar.

  • user

    AUTHOR Chris

    Posted on 12:02 pm June 29, 2016.
    Reply

    Thanks for putting this research together and thanks for making it available to everyone! The discussion of attrition impacting the pipeline is really fascinating and applies more broadly than just interviews. For someone trying to learn a new skill such as coding, even having something mean said to you counts towards the attrition event total. So discussing attrition has the potential to really progress our understanding of the tech gender gap. Kudos!

  • user

    AUTHOR jack meoff

    Posted on 11:59 am June 29, 2016.
    Reply

    I’m sure people will love talking to the witness protection program voice during interviews. Maybe for live coding exercises you can put the candidates in a dimly lit room and put a blur effect over their faces.

    Come on man, this is ridiculous and you know it.

  • user

    AUTHOR Ian

    Posted on 11:39 am June 29, 2016.
    Reply

    “Rather, it’s about women being bad at dusting themselves off after failing”

    I think this framing is problematic, because it suggests that it is due to some FLAW in women. Your findings (which are compelling and worthy of repetition elsewhere) need to be taken in the context of what we understand about the sociology and psychology of gender.

    From my (admittedly weak) understanding, it’s probably more accurate to say that women are more likely to PERSONALIZE failure than men are. In other words, when women fail at something they are more likely to believe it is because there’s something wrong with them. Whereas when men fail, they are more likely to believe it is random bad luck or that there’s something wrong with everyone else. This ties in nicely to what we know about things like stereotype threat and other psychosocial phenomena that are implicated in gendered behaviours like this.

    If that is indeed the case, it’s highly likely that the women who are NOT dropping out aren’t necessarily “good at dusting themselves off”, but rather are less likely to attribute their failures internally (and conversely for the men who ARE dropping out). I would love to see you collaborate with a social psychologist and see how this plays out in a more scientific setting. It would definitely challenge some of the prevailing theories around the barriers facing women in STEM fields.

    • user

      AUTHOR Joe M

      Posted on 12:35 pm June 29, 2016.
      Reply

      Seems to me like you’re playing word games without changing the essential point, namely, women handle failure less effectively in this context.

    • user

      AUTHOR chris

      Posted on 1:26 pm June 29, 2016.
      Reply

      > From my (admittedly weak) understanding, it’s probably more accurate to say that women are more likely to PERSONALIZE failure than men are.

      So, in other words, they are bad at dusting themselves off after falling…

      And isn’t also “problematic” to say that “when men fail, they are more likely to believe it is random bad luck or that there’s something wrong with everyone else.”

      • user

        AUTHOR Ian

        Posted on 12:24 pm June 30, 2016.
        Reply

        Somewhere in the realm of things, there is some abstract, disembodied, objective measure of how personally responsible people are for their successes and failures. Human beings don’t have access to that measure. Our evaluations are highly subjective. The evidence suggests that there is a bias among men to see failures as not their (our) fault; whereas women are biased in the other direction.

        It is not in any way accurate to say, therefore, that women are “bad at dusting themselves off”, any more than it would be accurate to say that “men are bad at understanding their own limitations”.

        • user

          AUTHOR Taka

          Posted on 11:23 pm July 1, 2016.
          Reply

          Weighing in as a man. I generally see my failures as strongly my own fault, but never-the-less deal with and push past that. Totally anecdotal, but that’s still one more hard datapoint than you’ve provided us with.

          > It is not in any way accurate to say, therefore, that women are “bad at
          > dusting themselves off”, any more than it would be accurate to say that
          > “men are bad at understanding their own limitations”.

          I mean, you don’t think that’s true of men? I’d say that it’s true of most people, but the evidence on self-evaluation above seems to imply that it’s especially true for men. Sounds like you just don’t want anything bad said about women as a group, regardless of whether it’s true or not. Kind of hypocritical, given all the shitty things women say about men as a group, true or not.

    • user

      AUTHOR Rosie

      Posted on 2:31 pm June 29, 2016.
      Reply

      You are correct. Here’s an article about a study that found a confidence gap between the genders. http://www.theatlantic.com/magazine/archive/2014/05/the-confidence-gap/359815/

      Now there are a couple of things that could be happening as well. It could be that males may have a inner locus of control and females may have an external locus of control, which impacts whether or not they see themselves as having influence over events and subsequent outcomes.

      Interestingly, in a study conducted on children in grade 4 and 5, there are differences in positive and negative feedback for male and female children. When girls failed they were encouraged to blame their personal lack of ability, while boys were encouraged to blame their own lack of effort, or they were encouraged to blame a source external to them, like a challenging situation or a person. As for positive feedback, the girls were encouraged to give credit to trying hard or being nice, or credit reasons that were out of the girl’s self-control. In short, girls were taught to internalize failure and externalize success, while boys are taught to internalize success and externalize failure.

      • user

        AUTHOR Tam

        Posted on 11:40 pm June 29, 2016.
        Reply

        Rosie, can you please share some more information about the study you refer to in the last paragraph? I’d be very interested to learn more.

  • user

    AUTHOR Herbert

    Posted on 11:37 am June 29, 2016.
    Reply

    “Contrary to what we expected (and probably contrary to what you expected as well!), masking gender had no effect on interview performance”

    No, in fact, that’s exactly what I expected.

    While it’s convenient to think of the world in black-and-white, that’s not often a true model of reality. To believe that voice tone matters is to believe that men are systematically hearing a female voice and ranking it less competent, i.e., that there are simply some significant number of people in this world who are unapologetic sexists.

    The main problem with this is that most tech interviewers are young enough that they went through college with a lot of women. These days, women are significantly more likely to attend college than men, so for anyone under about 40, their entire career experience is evidence against the belief that women are fundamentally incompetent. Why would we believe they would rank a female voice as less competent?

    That didn’t used to be true. The “blatant sexist” can be an accurate model when interviewers are much older. For example, orchestra auditions are infamously sexist in this way (hence why blind auditions have seen some success). Up until fairly recently, conductors tended to be older men, who went to college back when not many women did.

    • user

      AUTHOR Trevor

      Posted on 12:06 pm June 29, 2016.
      Reply

      You don’t need malice for that kind of effect – implicit biases have been identified in many other areas.

      • user

        AUTHOR Taka

        Posted on 11:24 pm July 1, 2016.
        Reply

        Controlled for age, as the above comment mentions?

  • user

    AUTHOR Vaniver

    Posted on 11:31 am June 29, 2016.
    Reply

    Do you think ‘veterancy’ effects are driving the difference (both men and women get better the more interviews they’ve done, and men have done more interviews)? It seems like you could check for that pretty easily by looking at graphs of interview quality by number of interviews done, for both men and women.

    Unless veterancy effects explain this, it looks to me like your point about attrition goes in the opposite direction. That is, women who perform poorly are more likely to leave the pool than men who perform poorly, so we should expect the female interviewee pool to *overrepresent* the population skill distribution by a larger degree than the male interviewee pool. That is, our gap in results becomes *harder* to explain, not easier.

    (Why do I think that? Imagine that the interview was perfect, both men and women were equally split between the four levels of skill, and the attrition numbers for poor performance were as described in this post, but there was no attrition after good performance. All the 3 and 4 women would be around, as well as all of the 3 and 4 men, but more 1 and 2 men would be around than 1 and 2 women, meaning the female mean would be higher, even though we started off with an equal skill distribution.)

  • user

    AUTHOR Ron Wencel

    Posted on 10:57 am June 29, 2016.
    Reply

    Do you keep data on the sex of the interviewers themselves? I’m curious if their sex could have played any role in the candidates decision to drop out, and if those female candidates interviewed by female interviewers were less likely to drop out. I imagine that if the attrition is caused by females feeling intimidated by the sense of not belonging or worthiness as is theorized, that having a role model of their own gender might have helped to combat that.

  • user

    AUTHOR Josh

    Posted on 10:27 am June 29, 2016.
    Reply

    Very interesting read. I’m curious as how the data slices up when looking at the gender matching between interviewer and interviewee. Is there bias when you look at the four combinations of interviewing gender pairs.

  • user

    AUTHOR AntiCuck

    Posted on 10:25 am June 29, 2016.
    Reply

    “and why that difference matters for the thorny issue of the gender gap in tech.”

    There is no gender gap, there is only qualified and unqualified. Artificially hiring women because, you know, women, under the guise of fighting this non-existent boogyman will only make “tech” worse.

    Way to brainlessly perpetuate the myth though.

    • user

      AUTHOR YayForData

      Posted on 11:39 am June 29, 2016.
      Reply

      There literally IS a gender gap, in that there are not as many women working in IT as there are men. This is not a secret, and it is not a figure subject to interpretation. It’s X != Y. Whether you think it is a problem, and whether you think it’s caused deliberately by Patriarchal machinations within the industry is more subjective, but from a very literal context there IS a statistical gap. And this article tries to use real world data to explain why it exists.

      • user

        AUTHOR David

        Posted on 12:16 pm July 1, 2016.
        Reply

        So, by your reasoning, there’s a systemic bias in the NBA against white people, a “gap” that is very real and NEEDS to be closed. It’s not that black people tend to be more suited to basketball, or that more of them grow up playing it, it’s a problem that needs interference to fix. /s

        • user

          AUTHOR Garrett

          Posted on 4:24 am November 28, 2016.
          Reply

          The NBA used to be dominated by white people. I suppose you would have used exactly the same words in exactly the same way to justify that, too.

    • user

      AUTHOR Joe M

      Posted on 12:33 pm June 29, 2016.
      Reply

      So, you, didn’t bother to read the article, did you?

  • View Comments (223) ...