We built voice modulation to mask gender in technical interviews. Here’s what happened.

interviewing.io is a platform where people can practice technical interviewing anonymously and, in the process, find jobs based on their interview performance rather than their resumes. Since we started, we’ve amassed data from thousands of technical interviews, and in this blog, we routinely share some of the surprising stuff we’ve learned. In this post, I’ll talk about what happened when we built real-time voice masking to investigate the magnitude of bias against women in technical interviews. In short, we made men sound like women and women sound like men and looked at how that affected their interview performance. We also looked at what happened when women did poorly in interviews, how drastically that differed from men’s behavior, and why that difference matters for the thorny issue of the gender gap in tech.

The setup

When an interviewer and an interviewee match on our platform, they meet in a collaborative coding environment with voice, text chat, and a whiteboard and jump right into a technical question. Interview questions on the platform tend to fall into the category of what you’d encounter at a phone screen for a back-end software engineering role, and interviewers typically come from a mix of large companies like Google, Facebook, Twitch, and Yelp, as well as engineering-focused startups like Asana, Mattermark, and others. For more context, some examples of interviews done on the platform can be found on our public recordings page.

After every interview, interviewers rate interviewees on a few different dimensions.

Feedback form for interviewers
Feedback form for interviewers

As you can see, we ask the interviewer if they would advance their interviewee to the next round. We also ask about a few different aspects of interview performance using a 1-4 scale. On our platform, a score of 3 or above is generally considered good.

Women historically haven’t performed as well as men…

One of the big motivators to think about voice masking was the increasingly uncomfortable disparity in interview performance on the platform between men and women1. At that time, we had amassed over a thousand interviews with enough data to do some comparisons and were surprised to discover that women really were doing worse. Specifically, men were getting advanced to the next round 1.4 times more often than women. Interviewee technical score wasn’t faring that well either — men on the platform had an average technical score of 3 out of 4, as compared to a 2.5 out of 4 for women.

Despite these numbers, it was really difficult for me to believe that women were just somehow worse at computers, so when some of our customers asked us to build voice masking to see if that would make a difference in the conversion rates of female candidates, we didn’t need much convincing.

… so we built voice masking

Since we started working on interviewing.io, in order to achieve true interviewee anonymity, we knew that hiding gender would be something we’d have to deal with eventually but put it off for a while because it wasn’t technically trivial to build a real-time voice modulator. Some early ideas included sending female users a Bane mask.

Early voice masking prototype
Early voice masking prototype (drawing by Marcin Kanclerz)

When the Bane mask thing didn’t work out, we decided we ought to build something within the app, and if you play the videos below, you can get an idea of what voice masking on interviewing.io sounds like. In the first one, I’m talking in my normal voice.

And in the second one, I’m modulated to sound like a man.2

Armed with the ability to hide gender during technical interviews, we were eager to see what the hell was going on and get some insight into why women were consistently underperforming.

The experiment

The setup for our experiment was simple. Every Tuesday evening at 7 PM Pacific, interviewing.io hosts what we call practice rounds. In these practice rounds, anyone with an account can show up, get matched with an interviewer, and go to town. And during a few of these rounds, we decided to see what would happen to interviewees’ performance when we started messing with their perceived genders.

In the spirit of not giving away what we were doing and potentially compromising the experiment, we told both interviewees and interviewers that we were slowly rolling out our new voice masking feature and that they could opt in or out of helping us test it out. Most people opted in, and we informed interviewees that their voice might be masked during a given round and asked them to refrain from sharing their gender with their interviewers. For interviewers, we simply told them that interviewee voices might sound a bit processed.

We ended up with 234 total interviews (roughly 2/3 male and 1/3 female interviewees), which fell into one of three categories:

  • Completely unmodulated (useful as a baseline)
  • Modulated without pitch change
  • Modulated with pitch change

You might ask why we included the second condition, i.e. modulated interviews that didn’t change the interviewee’s pitch. As you probably noticed, if you played the videos above, the modulated one sounds fairly processed. The last thing we wanted was for interviewers to assume that any processed-sounding interviewee must summarily have been the opposite gender of what they sounded like. So we threw that condition in as a further control.

The results

After running the experiment, we ended up with some rather surprising results. Contrary to what we expected (and probably contrary to what you expected as well!), masking gender had no effect on interview performance with respect to any of the scoring criteria (would advance to next round, technical ability, problem solving ability). If anything, we started to notice some trends in the opposite direction of what we expected: for technical ability, it appeared that men who were modulated to sound like women did a bit better than unmodulated men and that women who were modulated to sound like men did a bit worse than unmodulated women. Though these trends weren’t statistically significant, I am mentioning them because they were unexpected and definitely something to watch for as we collect more data.

On the subject of sample size, we have no delusions that this is the be-all and end-all of pronouncements on the subject of gender and interview performance. We’ll continue to monitor the data as we collect more of it, and it’s very possible that as we do, everything we’ve found will be overturned. I will say, though, that had there been any staggering gender bias on the platform, with a few hundred data points, we would have gotten some kind of result. So that, at least, was encouraging.

So if there’s no systemic bias, why are women performing worse?

After the experiment was over, I was left scratching my head. If the issue wasn’t interviewer bias, what could it be? I went back and looked at the seniority levels of men vs. women on the platform as well as the kind of work they were doing in their current jobs, and neither of those factors seemed to differ significantly between groups. But there was one nagging thing in the back of my mind. I spend a lot of my time poring over interview data, and I had noticed something peculiar when observing the behavior of female interviewees. Anecdotally, it seemed like women were leaving the platform a lot more often than men. So I ran the numbers.

What I learned was pretty shocking. As it happens, women leave interviewing.io roughly 7 times as often as men after they do badly in an interview. And the numbers for two bad interviews aren’t much better. You can see the breakdown of attrition by gender below (the differences between men and women are indeed statistically significant with P < 0.00001).

Also note that as much as possible, I corrected for people leaving the platform because they found a job (practicing interviewing isn’t that fun after all, so you’re probably only going to do it if you’re still looking), were just trying out the platform out of curiosity, or they didn’t like something else about their interviewing.io experience.

A totally speculative thought experiment

So, if these are the kinds of behaviors that happen in the interviewing.io microcosm, how much is applicable to the broader world of software engineering? Please bear with me as I wax hypothetical and try to extrapolate what we’ve seen here to our industry at large. And also, please know that what follows is very speculative, based on not that much data, and could be totally wrong… but you gotta start somewhere.

If you consider the attrition data points above, you might want to do what any reasonable person would do in the face of an existential or moral quandary, i.e. fit the data to a curve. An exponential decay curve seemed reasonable for attrition behavior, and you can see what I came up with below. The x-axis is the number of what I like to call “attrition events”, namely things that might happen to you over the course of your computer science studies and subsequent career that might make you want to quit. The y-axis is what portion of people are left after each attrition event. The red curve denotes women, and the blue curve denotes men.

Now, as I said, this is pretty speculative, but it really got me thinking about what these curves might mean in the broader context of women in computer science. How many “attrition events” does one encounter between primary and secondary education and entering a collegiate program in CS and then starting to embark on a career? So, I don’t know, let’s say there are 8 of these events between getting into programming and looking around for a job. If that’s true, then we need 3 times as many women studying computer science than men to get to the same number in our pipelines. Note that that’s 3 times more than men, not 3 times more than there are now. If we think about how many there are now, which, depending on your source, is between 1/3 and a 1/4 of the number of men, to get to pipeline parity, we actually have to increase the number of women studying computer science by an entire order of magnitude.

Prior art, or why maybe this isn’t so nuts after all

Since gathering these findings and starting to talk about them a bit in the community, I began to realize that there was some supremely interesting academic work being done on gender differences around self-perception, confidence, and performance. Some of the work below found slightly different trends than we did, but it’s clear that anyone attempting to answer the question of the gender gap in tech would be remiss in not considering the effects of confidence and self-perception in addition to the more salient matter of bias.

In a study investigating the effects of perceived performance to likelihood of subsequent engagement, Dunning (of Dunning-Kruger fame) and Ehrlinger administered a scientific reasoning test to male and female undergrads and then asked them how they did. Not surprisingly, though there was no difference in performance between genders, women underrated their own performance more often than men. Afterwards, participants were asked whether they’d like to enter a Science Jeopardy contest on campus in which they could win cash prizes. Again, women were significantly less likely to participate, with participation likelihood being directly correlated with self-perception rather than actual performance.3

In a different study, sociologists followed a number of male and female STEM students over the course of their college careers via diary entries authored by the students. One prevailing trend that emerged immediately was the difference between how men and women handled the “discovery of their [place in the] pecking order of talent, an initiation that is typical of socialization across the professions.” For women, realizing that they may no longer be at the top of the class and that there were others who were performing better, “the experience [triggered] a more fundamental doubt about their abilities to master the technical constructs of engineering expertise [than men].”

And of course, what survey of gender difference research would be complete without an allusion to the wretched annals of dating? When I told the interviewing.io team about the disparity in attrition between genders, the resounding response was along the lines of, “Well, yeah. Just think about dating from a man’s perspective.” Indeed, a study published in the Archives of Sexual Behavior confirms that men treat rejection in dating very differently than women, even going so far as to say that men “reported they would experience a more positive than negative affective response after… being sexually rejected.”

Maybe tying coding to sex is a bit tenuous, but, as they say, programming is like sex — one mistake and you have to support it for the rest of your life.

Why I’m not depressed by our results and why you shouldn’t be either

Prior art aside, I would like to leave off on a high note. I mentioned earlier that men are doing a lot better on the platform than women, but here’s the startling thing. Once you factor out interview data from both men and women who quit after one or two bad interviews, the disparity goes away entirely. So while the attrition numbers aren’t great, I’m massively encouraged by the fact that at least in these findings, it’s not about systemic bias against women or women being bad at computers or whatever. Rather, it’s about women being bad at dusting themselves off after failing, which, despite everything, is probably a lot easier to fix.

1Roughly 15% of our users are female. We want way more, but it’s a start.

 

2If you want to hear more examples of voice modulation or are just generously down to indulge me in some shameless bragging, we got to demo it on NPR and in Fast Company.

3In addition to asking interviewers how interviewees did, we also ask interviewees to rate themselves. After reading the Dunning and Ehrlinger study, we went back and checked to see what role self-perception played in attrition. In our case, the answer is, I’m afraid, TBD, as we’re going to need more self-ratings to say anything conclusive.

266 thoughts on “We built voice modulation to mask gender in technical interviews. Here’s what happened.”

  1. “and why that difference matters for the thorny issue of the gender gap in tech.”

    There is no gender gap, there is only qualified and unqualified. Artificially hiring women because, you know, women, under the guise of fighting this non-existent boogyman will only make “tech” worse.

    Way to brainlessly perpetuate the myth though.

    1. There literally IS a gender gap, in that there are not as many women working in IT as there are men. This is not a secret, and it is not a figure subject to interpretation. It’s X != Y. Whether you think it is a problem, and whether you think it’s caused deliberately by Patriarchal machinations within the industry is more subjective, but from a very literal context there IS a statistical gap. And this article tries to use real world data to explain why it exists.

      1. So, by your reasoning, there’s a systemic bias in the NBA against white people, a “gap” that is very real and NEEDS to be closed. It’s not that black people tend to be more suited to basketball, or that more of them grow up playing it, it’s a problem that needs interference to fix. /s

      2. The NBA used to be dominated by white people. I suppose you would have used exactly the same words in exactly the same way to justify that, too.

  2. Very interesting read. I’m curious as how the data slices up when looking at the gender matching between interviewer and interviewee. Is there bias when you look at the four combinations of interviewing gender pairs.

  3. Do you keep data on the sex of the interviewers themselves? I’m curious if their sex could have played any role in the candidates decision to drop out, and if those female candidates interviewed by female interviewers were less likely to drop out. I imagine that if the attrition is caused by females feeling intimidated by the sense of not belonging or worthiness as is theorized, that having a role model of their own gender might have helped to combat that.

  4. Do you think ‘veterancy’ effects are driving the difference (both men and women get better the more interviews they’ve done, and men have done more interviews)? It seems like you could check for that pretty easily by looking at graphs of interview quality by number of interviews done, for both men and women.

    Unless veterancy effects explain this, it looks to me like your point about attrition goes in the opposite direction. That is, women who perform poorly are more likely to leave the pool than men who perform poorly, so we should expect the female interviewee pool to *overrepresent* the population skill distribution by a larger degree than the male interviewee pool. That is, our gap in results becomes *harder* to explain, not easier.

    (Why do I think that? Imagine that the interview was perfect, both men and women were equally split between the four levels of skill, and the attrition numbers for poor performance were as described in this post, but there was no attrition after good performance. All the 3 and 4 women would be around, as well as all of the 3 and 4 men, but more 1 and 2 men would be around than 1 and 2 women, meaning the female mean would be higher, even though we started off with an equal skill distribution.)

  5. “Contrary to what we expected (and probably contrary to what you expected as well!), masking gender had no effect on interview performance”

    No, in fact, that’s exactly what I expected.

    While it’s convenient to think of the world in black-and-white, that’s not often a true model of reality. To believe that voice tone matters is to believe that men are systematically hearing a female voice and ranking it less competent, i.e., that there are simply some significant number of people in this world who are unapologetic sexists.

    The main problem with this is that most tech interviewers are young enough that they went through college with a lot of women. These days, women are significantly more likely to attend college than men, so for anyone under about 40, their entire career experience is evidence against the belief that women are fundamentally incompetent. Why would we believe they would rank a female voice as less competent?

    That didn’t used to be true. The “blatant sexist” can be an accurate model when interviewers are much older. For example, orchestra auditions are infamously sexist in this way (hence why blind auditions have seen some success). Up until fairly recently, conductors tended to be older men, who went to college back when not many women did.

    1. You don’t need malice for that kind of effect – implicit biases have been identified in many other areas.

  6. “Rather, it’s about women being bad at dusting themselves off after failing”

    I think this framing is problematic, because it suggests that it is due to some FLAW in women. Your findings (which are compelling and worthy of repetition elsewhere) need to be taken in the context of what we understand about the sociology and psychology of gender.

    From my (admittedly weak) understanding, it’s probably more accurate to say that women are more likely to PERSONALIZE failure than men are. In other words, when women fail at something they are more likely to believe it is because there’s something wrong with them. Whereas when men fail, they are more likely to believe it is random bad luck or that there’s something wrong with everyone else. This ties in nicely to what we know about things like stereotype threat and other psychosocial phenomena that are implicated in gendered behaviours like this.

    If that is indeed the case, it’s highly likely that the women who are NOT dropping out aren’t necessarily “good at dusting themselves off”, but rather are less likely to attribute their failures internally (and conversely for the men who ARE dropping out). I would love to see you collaborate with a social psychologist and see how this plays out in a more scientific setting. It would definitely challenge some of the prevailing theories around the barriers facing women in STEM fields.

    1. Seems to me like you’re playing word games without changing the essential point, namely, women handle failure less effectively in this context.

      1. > the essential point, namely, women handle failure less effectively in this context

        What does “handle failure” mean to you? Does it just mean that you have to persist in the face of difficulties?

        It’s possible that this is about some “women being bad at dusting themselves off after failing”. However, it is equally possible that it’s about some “men being bad at recognizing their relative economic efficiency”, and therefore taking a much longer time to see that they’re unsuited to the career that they’ve fixated on.

        Getting an unpleasant result from something that you’re exploring, and deciding that you’ve got better options elsewhere is what most of us call “success”. “Fail faster” is, and ought to be, a goal for most people.

        Remember: These women aren’t saying, “Oh, bad score, poor little me, I’m going to go home and cry for the rest of my life.” They’re saying, “Bad score, that’s not working for me, I’m going to do something different.” “Something different” might mean finding a different way to practice interviews or it might mean deciding to take your skills to the finance or healthcare sectors, but actively choosing to do something that you think you are better suited to is not “failure”.

        Think about the most monumentally incompetent solo computer consultant you’ve ever encountered. Ask your colleagues for the names of the one consultant that they will absolutely never hire again. They were all (or almost all) men, weren’t they? And he had absolutely no clue that he was a bumbling idiot, didn’t he? IMO the world would be better off (and one large computer environment that I worked in would have been *much* better off) if more men were as adept at recognizing their “failures” as the average woman.

    2. > From my (admittedly weak) understanding, it’s probably more accurate to say that women are more likely to PERSONALIZE failure than men are.

      So, in other words, they are bad at dusting themselves off after falling…

      And isn’t also “problematic” to say that “when men fail, they are more likely to believe it is random bad luck or that there’s something wrong with everyone else.”

      1. Somewhere in the realm of things, there is some abstract, disembodied, objective measure of how personally responsible people are for their successes and failures. Human beings don’t have access to that measure. Our evaluations are highly subjective. The evidence suggests that there is a bias among men to see failures as not their (our) fault; whereas women are biased in the other direction.

        It is not in any way accurate to say, therefore, that women are “bad at dusting themselves off”, any more than it would be accurate to say that “men are bad at understanding their own limitations”.

      2. Weighing in as a man. I generally see my failures as strongly my own fault, but never-the-less deal with and push past that. Totally anecdotal, but that’s still one more hard datapoint than you’ve provided us with.

        > It is not in any way accurate to say, therefore, that women are “bad at
        > dusting themselves off”, any more than it would be accurate to say that
        > “men are bad at understanding their own limitations”.

        I mean, you don’t think that’s true of men? I’d say that it’s true of most people, but the evidence on self-evaluation above seems to imply that it’s especially true for men. Sounds like you just don’t want anything bad said about women as a group, regardless of whether it’s true or not. Kind of hypocritical, given all the shitty things women say about men as a group, true or not.

    3. You are correct. Here’s an article about a study that found a confidence gap between the genders. http://www.theatlantic.com/magazine/archive/2014/05/the-confidence-gap/359815/

      Now there are a couple of things that could be happening as well. It could be that males may have a inner locus of control and females may have an external locus of control, which impacts whether or not they see themselves as having influence over events and subsequent outcomes.

      Interestingly, in a study conducted on children in grade 4 and 5, there are differences in positive and negative feedback for male and female children. When girls failed they were encouraged to blame their personal lack of ability, while boys were encouraged to blame their own lack of effort, or they were encouraged to blame a source external to them, like a challenging situation or a person. As for positive feedback, the girls were encouraged to give credit to trying hard or being nice, or credit reasons that were out of the girl’s self-control. In short, girls were taught to internalize failure and externalize success, while boys are taught to internalize success and externalize failure.

      1. Rosie, can you please share some more information about the study you refer to in the last paragraph? I’d be very interested to learn more.

  7. I’m sure people will love talking to the witness protection program voice during interviews. Maybe for live coding exercises you can put the candidates in a dimly lit room and put a blur effect over their faces.

    Come on man, this is ridiculous and you know it.

  8. Thanks for putting this research together and thanks for making it available to everyone! The discussion of attrition impacting the pipeline is really fascinating and applies more broadly than just interviews. For someone trying to learn a new skill such as coding, even having something mean said to you counts towards the attrition event total. So discussing attrition has the potential to really progress our understanding of the tech gender gap. Kudos!

  9. I believe gender impacts how we interview by more than just the sound of our voices. Women behind voice modulation that causes them to sound like men may still lack the swagger/confidence in their interview that men often exhibit, leading one to believe they are less qualified than their male counterparts. Because we are learning to account for that as interviewers, I wonder if the voice modulation does actively hurt women’s perceived performance.

    1. I think you are right and that the attrition of women suggests exactly that. Especially so, if you consider the fact that after a couple of interviews performance is the same.

    2. I think you hit the nail on the head here and that does explain attrition AND that after a couple of interviews performance is similar.

  10. Alfred Meierberger

    Hey, I’d like to point out a few things here.
    1. There is no one who wants to hinder women studying tech. CS (Computer Science) students would love to have more than 3% women (CS student myself).
    2. No one thinks women are inherently unqualified for tech jobs, on the contrary (except maybe old and open sexists, but they do not make up any significant percentage)
    3. Where the f*** did you pull that last chart from? I mean you clearly stated it is hypothetical (thanks for that) but for everybody just looking over the article it suggests you found a huge gap, which you did not.

    But, why are there so little women in tech then? Well, let me ask a counterquestion: Why are there so little men in linguistics? Because men are less interested in language studies. And telling by experience, lots of women are just not that interested to study CS. Why? Maybe because there is so much bias towards tech people (Nerds, geeks, sweaty, unwashed long hair and living in mum’s basement) which deters women. But that is just something time will fix, and these stereotypes will go away as the stigma associated with tech fades.
    My Conclusion: There is no systemic bias towards women, but women are biased towards tech.

    1. Gosh. You just read a scientific study about that topic and even they didn’t jump to conclusions. Why do you need to speculate from your own very limited experience? Your whole text has no impact at all, because it’s not at all based on any scientific principle.

      If you think “women just don’t like tech” try to do a study about that. Find out if it’s really true, and then present your findings. Else you will only convince people who are already on your side.

      1. Since there are no barriers of entry for women to go into STEM (there is even encouragement!), and there are still far less women in STEM, and women generally have good enough grades to learn or study anything they want, what would be your conclusion?

        A. People choose whatever career or curriculum they like best and are most interested in. Since we observe less women in STEM, that means women are less interested in STEM.

        B. There are multiple invisible forces at work, which hinder women to go into STEM. Since we observe less women in STEM, the evil forces must be discovered.

        A is based on a positivists world view on people. People have free will and want to pursue their own aims and happiness, if sufficient social and economic security allows them to do so. We observe less women in STEM. The simple, Occam’s Razor explanation is that women don’t want to go into STEM.

        B assumes that there are “hidden” forces, like biases and hurdles for women to go into tech. That’s far harder to test for, and extremely hard to prove. And since that is not my hypothesis, I am not forced to provide any “counterproof”. I have yet to see any conclusive study which shows that. Just finding low numbers of women in STEM, while there are more women in universities in general, doesn’t prove that.

        If two (or more) hypotheses can explain a phenomenon, you should pick the most rational and simple one. Assuming that people have free will and do what they want is simpled and more based on actual experience and science than invisible forces. So for now I will believe that A is the correct explanation, until someone finds more observations AND proves that A cannot explain the observations. That’s actually how science works. Just making up more and more complicated hypotheses, but failing to show how the new ones are explaining the observations better is not science.

        The “study” you asked for is to investigate what school children want to do as a profession after school. I only found a German newspaper article about that for now, but I guess you can get the idea from the graph:

        http://www.spiegel.de/schulspiegel/berufseinstieg-schueler-mit-berufswahl-ueberfordert-a-1004735.html

        They asked 500 students. A whopping 2% of girls want to go into a technical profession, with 27% of boys. 6% of boys want to go into IT, with 0% of girls. Girls predominantly want to go into social jobs, like medicine and teaching.

        Now of course one can assume that there the biases against girls/women happen earlier, in schools. Actually finding out WHY girls don’t WANT to go into tech is a study worth investigating, not some unprovable invisible bias. But maybe it’s just as simple as “they don’t want to”, and that’s it.

  11. Obviously, we need to lower standards and impose hiring quotas for women in this sector. Otherwise, any other course of action would be misogynistic and biased.

    The last thing we would want to do would to be hold women accountable for their actions and ascribe to them some degree of agency in their decisions to give up early in this context.

    That would be sexist. They aren’t decisive, free-willed humans after all. They are women fighting the all-invasive Patriarchy which robs them of agency and accountability.

    1. You are exactly the kind of misogynistic tool, other men like me are ashamed off. Being a smartass doesn’t solve societal problems. We can’t ignore 50% of the world’s population because they don’t belong to the your gender. Try to help your peers deal with failure rather than criticize them.

      1. Genius, women control the majority of private wealth in the US, get sentenced more lightly for the same crimes as men do, and yet white women are the majority of beneficiaries of affirmative action in this country. To allude to some mythical discrimination against women in technology while job seeking in the US is idiotic, given the incentives, set-asides, legal diversity requirements, etc. That are in place pushing companies to hire female employees. If women don’t like being rejected, then they need to “woman up” and learn a little resiliency. Perhaps a step in that direction would be removing those hiring incentives, set-asides, etc. so womenow actually have to compete based on their skills and merits.

        On top of that, women in technology in the US are overwhelmingly from Asian countries where feminist coddling and social engineering are nonexistent, and that is a very generous description of that part of the world.

        Crappy social science as embodied in this “experiment” seems to rule the day in America. I for one welcome my fellow hard scientists from Asia, male or female, as they at least have a sense of rigor and analytics skills wheneeded it comes to distinguishing science from ideologically-driven BS.

      2. Kind of funny that you call the parent misogynistic, because what he describes is pretty much what those 3rd wave feminists and genderistas always claim.

      3. Hey Bharath, strawman much? Nobody said to ignore women. Parson just promoted the idea that merit and independence is not limited to men. The pandering of internet white knights to women often results in the infantilization of said women, but “the other” is the sexist.

        You sir, are trained like pavlov’s dogs, thowing around “misogynistic” “smartass”, instead of trying to comprehent what Parson was all about. You are the tool, and not a very sharp one at that.

        pls no bully, english is my third language

    2. Don’t worry, sweetie, nobody female wants your dead-end sheep job where your frustrations accumulate to the point you have to vent it out online.

      1. More like leeching of, ey? But hey, lets kick in the face of the one man that is actually trying to promote treating women as adults, instead as overgrown children.

        Welcome to upside-down land, were the constructive thinkers are treated like plebs, and the idiots set the framework for everybody else.

        Gods, i hope that all men/women in those “dead-end sheep jobs” would just sit down, and do nothing. Just for a week. Just starve these ungrateful parasites and their high horses alltogehter.

        Instead of compassion with those wageslaves, on wich your way of life depends on, you show nothing less than contempt and malice. No wonder MGTOW is a thing.

        I hope you manage to pull your head out of your a**, and overcome this bitter snarky persona of yours. Cheers

    3. Damn straight! lets put MORE quotas on companies to force companies to choose candidates less skilled for the role. These people are just “giving up more easily” that’s all. You can still hire them, it’s not like they’ll give up easily during the job either!

    4. Right on the money Parson. Although other men try to shame you, based on their ideology of “everyone is equal in every way”, stay the course. Treating women as adults is not sexism, its respect.

  12. Nick Brosnahan

    Did you modulate the interviewers voices as well or just the interviewees? I’m curious to see if you would see a change in men dropping out when they perceive a female interviewer vs. women dropping out if they perceive a male interviewer. It’s not clear from the article if this was just one-direction or it was both directions.

    1. We didn’t modulate interviewer voices this time, but it’s definitely something we’d like to try in the future.

  13. Purely conjecture, but could this issue be something that starts even at the beginning of Computer Science education? Do women tend to give up on CS more than men when struggling, resulting in a gender gap that grows wider and wider as you progress through the education levels?

    This could possibly explain why the industry has a large gender gap, because so many are weeded out before they even sit down for an interview.

    1. I didn’t struggle when I studied computer science. It was easy. I left the “difficult” math exams after half of the allotted time to still get full points. Many girls dropped out because the required math was “too difficult” for them. Of course, about half of the boys dropped out after the first few exams, too. It is difficult, but you know when you are good enough to pass the tests. The few girls who stayed passed the final exams easily, too. All the people who stayed until the final exams where those who didn’t have big difficulties. That’s the point of these exams. It’s not like you don’t know if you are good or bad. If you are good, you stay, if you are bad, you quit. If you don’t get it, you’ll be thrown out by not passing the exams.

      Nowadays, some universities make the exams easier to get more people out, which also closes the gender gap partly. The industry reacts by doing the same exams as 20 years ago in their interviews, because they want the same level of quality.

  14. Is attrition related to women feeling more of an impact, or is resilience in the face of attrition events a matter of interest and need? If men have a stronger desire for the high paying position then women, such as would be the case if women have alternate means to acquire high spending potential and men do not, then women would have less reason to put up with attrition events. If men have more interest in the subject matter, they would have an easier time shrugging off attrition events.

    Despite earning only 43.5% of the money earned in the US by individuals, women spend roughly 60% of the money spent in the US by individuals. Clearly women are receiving money via additional avenues not available to men (and men most even be supplying these additional avenues). As such, they have less reason to seek a higher paying position.

    Additionally, differences in the nature of male and female interests, whilst not absolute by any means, have been shown to be manifest long before society can act on them. The key difference in these interests is between the social and the physical.

    Together these two effects will account for far more of the gender gap than discrimination. Indeed, discrimination against the direction of the gender gap is highly unlikely to overturn these effects.

  15. There is a nice documentary about how this gender gap is mostly because of biology
    One of the studies found that how wealthier the country is, how bigger the gender gap will be because people will study the things that they find intresting, while in poor countries (uganda, ghana) the gender gap decreases because the technical jobs will have a bigger chance at a better life

    link to the documentary

    1. while it’s true that women nurture, share, engage in non-technical careers, it’s also true that those ‘female’ careers have always been looked down on since men ‘allowed’ us to venture out of the kitchen, and are paid less partially on purpose and partially due to economics.

      while I am not trying to guilt trip men, and this isn’t even about them because I personally don’t hire men anymore because they lie too often about their skills and make me lose time and money far more often then women, I will tell you that we are all biologically even influenced by the past. Just ask a male scientist.

      and sure economists such as Sowell, Williams will say the gender gap is a myth because women pick different careers, women take care of their kids, they work less, and all other variables and whatnot, the fact remains that some of us, ‘insecure’ women, don’t actually need a PHD in liberal economics to comprehend that even JK Rowling had to hide her full name before publishing because boys don’t relate to women or girls writers and would not buy the book. And publishers are in the industry for the money, not for illusions of ‘equality’ in any way, shape, or form.

      Do you really think that after centuries of major three religions which in their religious texts, images, councils, and among leaders have centuries portrayed women as ‘less’ spiritually, and physically than men, and that just because we have the internet and toilet paper with scent in the West these days male world will suddenly drop their unconscious and conscious bigotry? Yeah, right. And the witch hunts never happened.

      1. Maybe engineering is harder than liberal arts and produce more, maybe there are reason why pediatricians are paid less than surgeons? No it must be discrimination against women. Surly jobs that produce majority of work facilitates are not dangerous and are paid premium, no men are just dumb.
        Have suggestion for you. Try fit conclusion into data not other way around.

  16. My two cents: Women tend to be more self deprecating, act more humble and downplay their skills, while men tend to act more confident and exaggerate their skills. Is it possible that this subtly influenced the interviewers?

    1. Absolutely
      If they ask how competent are you at doing task X , and you say you are excellent at task X, that sounds better than saying you are merely good at task X.

    2. karen straughan

      I think it would be more likely to influence the interviewees.

      Women were slightly favored by interviewers. That’s entirely likely to be about diversity hiring practices. But despite being getting equal or better results on interviews, women gave up more often. It’s the giving up that is, at least in part, applicant for applicant, contributing to the dearth of women in STEM.

      So here’s my two cents: Do you think women might believe they’re being discriminated against when they aren’t? You know, because of cultural narratives? And that because they believe they’re being discriminated against based on something they can’t change (their sex), they might give up more easily than men? That they might blame some outside force they can’t change (sexism), rather than the fact that they just need to hone their interview skills?

      That is, do you think maybe so many women give up after one bad interview because there’s a Greek Chorus floating around them telling them they might as well give up? No matter how much they apply themselves, men will still be favored just because penis (when the opposite seems the case, at least in this field)?

      1. “Hey, I’ve got an idea, let’s all shut up and pretend sexism doesn’t exist in the workforce, in order to protect the delicate sensibilities of the women applying for jobs.”

        _Now_ who’s the one advocating for sissification and coddling?

        Also, do not make the mistake of extrapolating this website’s experiment, with its tiny, TINY sample size, as saying anything even close to truth about the tech industry in general. From the evidence at hand it makes just as much sense to conclude that the high attrition rate of females using this site is because a recent ad campaign they ran attracted a number of casual female users that swamped the even tiner set of serious female users signed on through other means. These numbers are so small they could be derailed by practically anything. An interviewing.io blog post trolling through a Facebook knitting forum could derail numbers this small.

        Of course, if you have larger, more controlled studies that back up this attrition difference, I’m eager to see them.

  17. This is a harsh group of critics! Interesting work for sure and I’m glad you included the last bit because as soon as you mentioned finding that disparity I wanted to know what the data would look like when corrected for attrition! Very cool insight, thanks for sharing.

  18. Maybe women are less likely to go through a bunch of work-based rejections because their experience is that rejections are going to just be followed by more rejections and never an acceptance. Maybe women give up because it’s the rational thing to do after doing well in interviews and never getting called or always coming in second place.

    You can’t assume all other conditions being equal for these women because they’re not.

  19. As a transwoman who transitioned 20 years into her engineering career, I can fill in some anecdotal information on this (if not broad data.) I’d like to begin with saying that I love real data and analysis going into issues like this. I’m glad this experiment was performed and the data fully presented even though it conflicted with the original hypothesis. I learned a good bit reading this.

    First, real data. There are studies that have found that resumes with female or black sounding names are rejected more frequently, anonymizing the name removes the bias. There is a gap at the very beginning of the pipeline even before interviews taken place.

    Second, during the personal interactions before, during, and after interviewing I find that I am “talked down to” far more frequently. What I mean by that is simple concepts are being explained to me, often in painful detail, before the questions are fully asked. As one example, when someone takes the time to explain how sporting games work as a preamble to design discussions involving information about them, that’s pretty obviously biased, that had never occurred before.

    Third, I have noticed that my appearance and how I dress influence how I am treated. If I wear a bit more makeup, wear nail polish, dress more fashionable, and curl my hair, I am treated far more patronizingly than if I wear the “standard geek wear” of jeans, t-shirt, hair in a ponytail, and virtually no makeup. I have heard the same feedback from all of the genetic woman I have asked about this, it is a common experience. Women know they need to dress down to be treated seriously.

    Fourth, I have noticed that the results for me have dramatically changed. In the past when I interviewed, I received offers for nearly every interview I had, about 7 out of 8. Now, I receive offers in about 2 out of 8. My technical skills have not deteriorated that dramatically in just a couple of years between looking for a new position. I have now received feedback along the lines of, “While your communication skills are excellent, we feel that you aren’t very technical.” I have never received a single comment before even hinting that I’m “not very technical.” I have peer-reviewed technical publications that are referenced by others. I have been a senior principle engineer in the past and architected systems on very large scales (Amazon amongst them.) It is hard for me to reconcile that these differences have nothing to do with gender.

    Fifth, the offers I have received are often for lower titles than I have had in the past, there is a well documented bias that men are promoted based on potential when they haven’t done the job and that women are promoted only after they have done the job for a time. In coming from a principle position, I have literally heard, “well, you haven’t been a principle here before, it’s much harder here, let’s try you out in a senior engineer position.” Seriously? A senior principle position at Amazon responsible for the design of a well known and publicly used product earning billions of dollars is literally much easier than a senior engineer position elsewhere? I doubt it.

    Sixth, rejections based on “not a good culture fit” occur. I’ve never heard that before. Ever. When you hear that from a company with 60 male engineers and 2 female engineers, you know exactly what that means. There’s a wonderful article about Paypal that they had problems finding a woman with a good culture fit. Later in the article it mentioned that they frequently solved design discussions by wrestling in the aisles.

    Seventh, woman are far more prominently represented in testing or quality assurance engineer roles, in full software engineer positions there are fewer women. Women tend to be shunted to the “less technical” positions.

    Eight, I’d never met a female principle software engineer before being hired as one. Everyone around me says things like, “I knew one once. Hmm, I think the fact that I can remember the single one is the problem.”

    Having literally been on both sides of this equation and speaking with other technical transwomen who have experienced the same differential treatment, appearance does matter. With voice only, the technical results may be the same, but the interviewer is free to imagine the appearance of the person being interviewed. It’s not just about perceived gender, it’s about the appearance of the specific person, body language, and other hard to quantify social cues. The treatment in a technical question completely divorced from the social interaction surrounding an interview does not necessarily reflect the questioning and results of real interviews. It also fails to reflect the pipeline, offers, and other aspects of the process.

    It’s also amusing to hear long time male friends hear my experiences and try to mansplain to me how they haven’t happened or aren’t valid in some way.

    This article in no way proves that we can all pat ourselves on the back because there is no bias. It proves that significant in-roads have been made into a single aspect of the process for which I am grateful and it shows that continually investigating the process does have positive results.

    1. ^^^^^Why I love being my own boss and having my own business. Ole! And if I ever crash and burn, I’d prefer living in a shelter. More humans in a shelter. Heh.

    2. I assure you, as male geek/nerd, you have to dress down to be taken seriously; it’s just not makeup and lipstick. It’s T-shirts and shorts instead of suits and ties. I think you know that ;-). It’s just a new experience for you that as female nerd, you have to dress down where you don’t actually wear a dress literally.

      I don’t know if you do hormone therapy, but many trans people do, and the hormones of course do affect you. The therapy has many effects on your brain, see e.g. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2879914/ even though your history as man has left its traces – you can’t switch over to a full “female” brain by taking estrogen, your brain will stay partly male. But due to this therapy, you are not exactly the same person as before, and that’s the point of transgender: you wanted to become a different person. Pretending you are still the same, now in the “correct” body is not honest.

      Let’s phrase it differently: We have a number of challenging fields like medicine or law, which were dominated by males 50 years ago about as much as IT is today. They have shifted to a somewhat female domination, at least in the younger generation. The culture of these male-dominated fields were just like any other male-dominated field, because that’s how men work – I don’t want to say that this is good or desirable, but that it did not prevent women to enter the field gradually to eventually dominate it. Prejudice and culture can change. These fields are difficult, too, but the challenges are different.

      Therefore, we have to continue to investigate the reasons for this particular gender gap, and not use “prejudice” as single reason. There is prejudice, of course, there always is, but as a single explanation (“there’s a gender gap, because men are full of prejudice”) it is just an example of prejudice at work – looking for simple explanations, and wildly extrapolating. And while prejudice alone means that the process of women entering any male-dominated field is going to be gradually (not over night), in the long run, prejudice can be overcome. The non-prejudice parts that cause this gender gap – if they exist, can not be overcome that easily.

      I’ve seen the opposite report of what you say of a transman; he said he told people that his former female self was “his sister”, and people told him while he works in the same field as his sister, his work was considerably better. Maybe it was prejudice, or maybe it really was better, and the hormone therapy was the cause. We have to find out. IMHO it’s to easy to blame prejudice alone.

      I also know a transwoman who I consider does a good job as software engineer, but I think of her more as cross-dresser, because her male features of face and body are too strong (big nose, wide shoulders, voice too deep), and I know she was a man before; she’s open about that, and our community accepts that. I do have the impression that she is considerably less self-confident than men, and that does affect her in-person presentations. It does not affect online, written discussion.

      It could well be that this effect is measured here, because even when your voice sounds male, all the signs about your ego will still go through unfiltered. If you show that you are not very confident about what you present, you’ll be rated down. If you have a female voice (or appearance) and show less confidence, people will adjust for it somewhat, if they know that this reduced self-confidence does not match with the actual skills.

      1. I do not do hormone therapy, my brain has not changed because of that, and I pass perfectly well without it. I have confirmed with friends that the differences in my behavior is that I am more happy and if anything, my confidence has gone up. My technical performance has improved as well with the reduction of my anxiety.

        Are you transgender? It doesn’t sound like it. If you aren’t please don’t mansplain to me what the “point” of transgender is. Your understanding as expressed in your words is deeply flawed. I am in the correct body and I have never wanted to be a different person. You do not understand, please do not think that your outsider knowledge of me is somehow more comprehensive than my insider knowledge of me.

        Instead of addressing my actual points you tell me how my experiences and opinions are invalid. Your first reaction is to tell me how I am flawed based upon your incomplete knowledge rather than thinking that I might have spent a lot of time throughout my life researching, learning, and observing how the interplay between women and men works. That knowledge has been very important for my actual survival. I bet far more time than you have spent doing the same.

        You even spend time informing me that in the past there were male dominated occupations that are now becoming more female dominated because of the revelation that prejudice and culture have changed. Do you seriously think I don’t know that? Or that your conclusions are somehow profoundly insightful? Please reread that paragraph and try to see how patronizing it sounds if someone took the time to tell you those things?

        Being transgender for my entire life and a scientist with degrees in biology, medicine, mathematics, and computer science with extensive experience designing, running experiments, and analyzing the resulting data I have experience in how to properly determine what is going on and how to gather data. I have done numerous experiments by varying certain aspects of the interview process, my looks, my voice, the perceived gender of my name, etc because I am fascinated at the differences that have cropped up. I don’t just interview to find a job, I have interviewed far more than I need to as part of my own private interest in determining what is going on. It’s all anecdotal because of the difficulties of controls in sociological experiments and the small sample size, but fascinating to me in the possible patterns it exposes.

        It would have been nice if you started to discuss the sociological context of how men and women are raised, societal pressures and expectations, inherent biases in numerous systems, or any number of other possible influences and how that might have any impact in this. Instead your “helpful explanation” is proof positive that you believe you can instruct others about the validity of their own experiences. You don’t even comprehend that you might have a more incomplete view of what is going on than the person experiencing it and that may cause you to perceive things inaccurately. Please read and try to see things from a point of view other than your own, you might learn something because other people do have experience and knowledge you do not.

        As a very personal note to help a trans person in your life, please try to not think of the transwoman as a crossdesser because of your opinions about her, especially her appearance, calling a trans person who has transitioned a cross dresser is deeply insulting on numerous levels because of the connotations it has and the perceived validity of her motivations. I have numerous friends who are cross dressers and there is nothing wrong with cross dressing, but because of historical reasons calling a transwoman a crossdesser is very insulting.

      2. TM,

        I’m going to say this the nicest way possible, but.. Most employers don’t want to deal with the constant explanation that comes with having a trans employee. Especially for anything “customer facing” the fact of the matter is that, in much of the world, a trans person is going to have a very hard time “fitting in”. And no, your job shouldn’t be all about conforming, but it does matter. Don’t show up for an interview as a plumber wearing a 3 piece suit. And don’t show up for a banking interview wearing overalls. You have to look/act the part. And, being trans is not acting the part in most jobs (and nearly all IT jobs). It’s akin to me going out and getting a face tattoo and being upset that people won’t hire me for executive positions anymore.

      3. I work in backend system design and do not do customer facing work at all.

        Also, no one knows that I am transgender in the work world. I have kept that side completely segregated and do not reveal or call it out in anyway.

        And before you say that people figure it out on their own. I pass so well that other transgender people believe that I’m a genetic female – I’m not easily read by the people who are the best at detecting it.

        To further put that aspect to rest, I have eventually ran into co-workers who are straight, gay, lesbian, and transgender outside work in social settings and even after years of working with me, every single one told me they never knew or even thought I was transgender. Again, I’m very hard to read.

        Perhaps you have seen or noticed the more easily read transgender people, but you have not detected many of us who easily pass and are currently around you. Again, your experience and perceptions are incomplete, it would be nice if you didn’t assume you’ve seen it all.

        I have had multiple instances of being hit on during interviews. Does that ever happen to men? Not in my experience. Do you think that may influence their opinion of my skills or whether to hire me? Even an obvious wedding ring does not prevent it. That’s another trick women learn, always wear a wedding ring to an interview, even if you aren’t married, it reduces but does not eliminate the occurrence of being hit on. Do men need to take that into consideration? I bet you never even thought that these were things that women had to take into account because you have never experienced it. You have not seen it all or even heard of it all.

        And to forestall the statement that “things are getting better” with regards to the younger generation, I’ve been hit on almost entirely by engineers in the 25-35 year old range. The growing “brogrammer” phenomenon has increased the problem, not reduced it.

        I never wore anything but a suit to engineering interviews before, I spent most of my life and work on the East Coast often in financial positions, it’s expected. I have never worn a suit as a woman to a West Coast interview although I did as a man.

        On the West Coast wearing a demure, completely covering and slightly baggy sweater and a long skirt with your hair down is enough to influence the results – it also increases the chance of being hit on. The “uglier” you make yourself, the better the chances of being hired – wearing natural looking makeup that washes you out and emphasizes the worst features helps your chances.

        Please tell me how this is a shared experience and there is no bias and everything is hunky dory in the women in tech world.

      4. You might want to go back and reread everything she wrote, then delete your comment. For starters, “principle software engineer” isn’t exactly a customer service job.

        The rest of your comment is so awful, I dunno where to start with it.

      5. Wow. You start out with “I’m going to say this the nicest way possible,” and then produce a degrading turd of a lecture like that? Please remove yourself from the internet.

    3. I just wanted to say to you, TM, that I really appreciate your thoughtful informative comments and your incredibly tolerant responses here. They were fantastic to read. Thank you so much. You inspire.

    4. Thank you for a view from the other side – this is a good datapoint to have as a counterargument. I appreciate you sharing this with us.

    5. Finally someone in the conversation with some real ground-truth experience! Thank you for being here.
      I implore you to turn your attention to Karen Straughan’s comments here, and give them some much needed pushback on their claims that computer science is inherently “men’s work” and that the best thing we can do for women entering the field is “stop scaring them away by claiming there is bias”.

  20. I think this is a great start but shows that the problem needs to be analyzed on a more fundamental level. From what I see the problem is on the interviewers side not the interviewees! I think if companies focused more on the candidates people skills you will find the gender gap decrease.

    1. How is it the inverviewers fault, when they interview for the technical skills required for the job, instead of focusing on “people skills”? A job requires certain talents and skills, some more, some less so.

      These requirements are not made up on the spot, but based on what a company is looking for. When they are looking for tech-adapt people they will weight the tech skill more heavily, when they are looking for PR/sales people (or other positions with a lot of people interacting) they focus on people skills.

      In essence you are arguing for “lowering the bar” by weighting what’s actually important less (technical skills) while weighting not-so important skills (people skills) more heavily.

    2. >From what I see the problem is on the interviewers side not the interviewees

      But why? That was the original assumption, hence the voice modulation approach. It did not change a thing, leading to a conclusion totally opposite to yours.

    3. I have had to give about 20 technical interviews over the past 12 months. I am always looking for skills and problem-solving abilities first. Man or woman, I have no room for someone who doesn’t get it. I may be wrong, but I try to level-out my expectations because women (especially Indian women) tend to be less assertive. Sometimes much less. Encouraging someone to speak up is very important when interviewing, because code doesn’t challenge you or embarrass you (or judge you, either). IOW, I don’t care if someone is shy or lacks certain people skills — can they code?

  21. Nitpick:

    “to get to pipeline parity, we actually have to increase the number of women studying computer science by an entire order of magnitude.”

    Your formulation makes it sound like you’re advocating for this as the solution to the problem. However, as you later mention, fixing the attrition issue is probably a more effective, more realistic solution.

  22. Methodology is a little weak – all the participants knew they were involved in a study, and you did provide a sort of gender indicator. Since the gender imbalance issue is widely known, you need to control for the possibility of interviewers consciously attempting to be non-discriminatory. The unexpectedly higher ratings for the higher pitched mask and lower for the lower pitched mask potentially indicated that kind of behavior.

  23. You mean to tell me that women doubt themselves quicker than men… No, how could it be? Women have always been taught to not put themselves down… no, wait, just kidding. Well, look at it this way, at least they were not asked if they were planning to have a baby anytime soon. Always a win-win!

  24. Did you do any checks to confirm the modulation worked well? It would be interesting to ask the interviewers afterwards what gender they thought the interviewee was. There may still be some bias based on what gender the interviewer thinks they are talking to.

  25. This got me wondering as to how much the propensity for males to engage in risk taking and reward seeking behaviours as compared to females, may be a factor in the results you are seeing. As well as differences in competitive behaviour. Given that your platform has essentially gamified the process of technical interviewing, it may be that this “game” stimulates male dominated behavioural tendencies more so than female tendencies. It may be that the perceived rewards for continued participation in the process after initial failure are not a great enough motivating factor for females, while the failure and success (risk and reward) mechanics of the “game” provide greater motivation for males to persist in seeking reward in-spite of their initial failures.

    Your platform may be a good analogue for some of the differences seen in the gambling behaviours of males and females. Specifically the relationship between failure and re-engagement.
    Males dominate females in terms of the number of regular gamblers. Males having a tendency to begin gambling at a younger age and being more likely to become problem gamblers. Behaviours that can be beneficial in some circumstances can be detrimental in others.

    What I am suggesting is that this may go deeper than an inability for one to dust oneself off after an initial failure (i.e. personal shortcomings) and may have some ties to instinctual behavioural characteristics. Food for thought.

  26. I remember hearing about interviewing.io on NPR. I think the idea is great but the execution is very poor. The pitch of a “male” or a “female” voice isn’t the only distinguishing characteristic. Men and women simply speak differently: inflection, intensity, duration, diction, etc. etc. etc. make it pretty easy to distinguish a male speaker from a female speaker, even when the voice is “masked.” In all three interviewing.io voice-modulated examples I’ve heard, it’s been pretty easy to tell if the speaker is male or female (yes, without knowing the speaker’s gender beforehand). I would be surprised if most, if not all, interviewers didn’t figure out the gender of the interviewee pretty early on in the conversation. I’m wondering if interviewing.io is more a good PR move for employers rather than an effective tool for increasing diversity in the workplace. Has interviewing.io considered this? Have you done any research on it? (Play a few voice-modulated examples for a focus group and ask them to guess the speaker’s gender?)

    1. Sorry but I do not agree on your thesis that men and women can be “easily” distinguished based on their speaking habits.

      Linguistics is a complex subject and peoples way of speaking gets affected by many more things than just “gender”. Being a native speaker or not, what native language somebody speaks (habits from that translate to English), where somebody grew up and so much more.

      Do you have any studies going into that topic? The only thing I found, being from some “Europe-University Viadrina”, called “Language and Gender”. But that’s neither a study nor does it look in any way useful or scientific, it’s just a random collection of quotes while the author makes a lot of very weird claims, without actually sourcing them in any way.

  27. Emelio Lizardo

    Not surprising. For men the career is everything. No career, no life.

    For women, work and career are choices. Mom is the career of choice or last resort. Because men will support her.

  28. First off, to make sure that there are no misconceptions about the points I will try to make below: When I talk about “men”, “women”, “boys”, “girls”, I talk about statistical means. I assume that the distributions I refer to are normally distributed, unless specified otherwise. On the other hand, that means that the individual person can be anywhere on that distribution.

    “Specifically, men were getting advanced to the next round 1.4 times more often than women. Interviewee technical score wasn’t faring that well either — men on the platform had an average technical score of 3 out of 4, as compared to a 2.5 out of 4 for women.

    Despite these numbers, it was really difficult for me to believe that women were just somehow worse at computers, so when some of our customers asked us to build voice masking to see if that would make a difference in the conversion rates of female candidates, we didn’t need much convincing.”

    People are different. People have different abilities and interests. Obviously, men and women are also different. Seeing that men and women are already different superficial (i.e. they look different), wouldn’t it be fair to assume they “function” differently, i.e. have different interests and abilities?

    Then why is it hard to accept that men might simply be more apt in computer related topics? Why should that conjecture be specifically excluded? It explains everything, AND it matches observations. Men might simply be more interested in technical stuff than women. Why is that bad? I still fail to see the point of the social engineering idea that instead of having people do stuff they are apt in AND enjoy, to push them into something they might not. Studies have shown that women in rich countries tend to be LESS likely to go for technical jobs, BECAUSE the economic and social security enables them to do what THEY want. So also data proves that women apparently simply do not want to deal with technical issues, if they can freely choose not to. That is sufficient to explain the employment gap in STEM (in western societies). I sincerely ask why we should ignore that huge elephant in the room and try to make up experiments to find evidence for a conjecture which is created specifically from the mindset “but, that cannot be!”. Maybe it can be and is so, and that’s fine.

    Why do we “need” more women in STEM? In my job, I want to have colleagues who know what they’re doing. What gender they have is not important for that. And if it so happens that women are not interested and maybe not equally able do work in tech, what’s the big deal? I think the real issue in western societies is that jobs which are actually supporting the welfare and stability of the society are done to a large extent by women, and are massively undervalued (by pay, which also explains the mythical gender pay gap, which doesn’t account for job fields etc.). That’s the issue that needs fixing. Fighting to gain a foothold in STEM, because apparently wages are higher there misses the point. What women and feminists should fight for, is that jobs women are actually interested in and doing to be valued much higher than they are right now. That would be a fight which goes hand in hand with the interest of women (i.e. the bulk of the distribution), and not the small fraction which is interested and able to work in STEM.

    Fight for better pay and conditions in hospitals, kindergartens, schools, etc! The rich investor can only do his job because there is a huge workforce of underpaid people who keep the real life of his back, so that he can play casino at the stock markets. Show them who’s really running daily life!

  29. First of all: thanks for the data and the article. I wonder if the gender can also be perceived by other aspects of the interaction then the voice, e.g choice of words, rhythm, and just generally the content of the conversation. Thinks “mansplaining” – it is a good clues that the person doing is male. Lack thereof might be a clue as well. Maybe, if interviewers are aware of the voice modulation, they tend to start listening even more for those “interaction clues”.

    Maybe it would be interesting to have the interviewer record what their perceived gender of the candidate is, and see if the rating correlates with that.

    1. Michael Knight

      “Mansplaining” can ONLY be done by men. It’s in the definition. The same dismissive behaviour, when performed by women, is “explaining”

  30. karen straughan

    Having been heavily interested in gender differences for many years myself, I think this study might be interesting to you: http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0029265

    It took data from an earlier study that had found sex differences in personality to be very small. By evaluating the data differently (at the univariate level, rather than compiling the data into “the big five” multivariate categories), they found that men and women have about a 10% overlap in personality traits. When they removed the single biggest difference (sensitivity, higher in women), they were still left with only a 24% overlap.

    If sensitivity is the largest single personality trait that differs between men and women, that might explain your attrition levels.

    What the appropriate action to take is, to my mind, NOT to put X times as many women as men through the pipeline. This is actually counterproductive. You already have a talented pool of women. Plugging 10 times more women into the pipeline, only to have them keep on giving up at the first difficulty… well, if role models are valuable, that’s only going to create 10 times as many role models for failure, that other women are going to see.

    What MIGHT just work (and no, I don’t think parity is a reasonable goal, but what might work for the industry and for talented women (and men)), is to just stop. Just stop with all the talk of possible bias in hiring. Stop with all the talk of the sexism in the industry. Just. Stop.

    If women are more easily discouraged than men, then what message are they getting when so much time and attention is being given to convincing everyone that they’re being discriminated against? That not only do they have to deal with all the same difficulties of computer science (it’s not a cakewalk, ffs), but you can add systemic sexism and old boys’ clubs and bias in hiring on top of that.

    You ever wonder if men are more confident (in part) because they’re constantly told they have male privilege? If you’ve been told all your life that you’re probably going to do fine because you’re a man, why would you quit after a first bad interview? For all you know, you could suck at the interview and still be hired into that privileged old boys’ club, just because you have an outie rather than an innie. Or maybe you stick it out because you’ve been told there’s no excuse for not succeeding–all that male privilege, yo. No one to blame there, and nothing to do but suck it up and get better.

    You ever wonder if a woman might have absorbed all of these messages about systemic sexism and anti-woman bias and blah blah blah from the culture, and then the first bad thing happens, that she might think, “OMG, all this sexism. I know they didn’t hire me because I’m a woman, because EVERYONE KNOWS the industry is biased against women. Therefore, because I can’t change the fact that I’m a woman [you know, the way a man might work to change his interview skills], I might as well give up.”

    Don’t you guys think that maybe, just maybe, the pervasive rhetoric around this issue might actually be contributing to it? I’m very pleased that you guys have published the results of this research. When my daughter graduated high school with a partial IB and A’s in almost everything, but with a particular gift for math, chemistry and physics, I TOLD her to get into those fields. The hiring bias is INSANE–in her favor. If she went into mechanical or chemical engineering, she could write her own paychecks. If she went into industrial radiography, she’d be pulling in obscene amounts of money a week out of school. Companies would be throwing offers at her 3 months before graduation.

    Right now, the tech industry can’t get enough competent women. They’re falling over themselves and each other to hire competent women. They’re doing it for all the wrong reasons, in my opinion, but that doesn’t change the fact that they’re doing it. It’s never been easier for a woman to get into a tech field.

    And then a massive percentage of them just throw up their hands after one or two bad interviews? Well, maybe they’ve internalized the more pervasive message, which is the exact opposite of what I’ve told my daughter?

    1. I do believe this is what my high-school debate team – if I’d gone to a school classy enough to have one – would have called “special pleading”.

      Look, I agree that sometimes taking about a trauma can magnify the impact of that trauma. War veterans sometimes have to deal with this counterintuitive fact when deciding whether and how to get counseling, for example. But you might want to leaven your above criticism with some understanding of context: The people coming here are already a self-selected group. They’re STEM careerists looking for insights to improve their interviewing skills. If the take-away is that the field is totally unbiased, and that women are merely whiny quitters, isn’t that message _even_worse_ than one that (supposedly wrongly) teaches them to be vigilant of bias, in themselves and others?

      You’ve noted yourself that tech companies are falling over each other to hire women. Programming as a mainstream career choice has been around for about 30 years, give-or-take. That’s barely enough time to grow beyond the generation and the specific culture that spawned it. Perhaps tech CEOs and hiring managers are not being reverse-sexist idiots as you imply, but instead they are aware of how the field is _unnecessarily_ biased towards too-narrow of a culture that happens to favor a given gender, and are trying to leverage an untapped supply of coders by pulling them in and forcing them to advocate for changes that make them more comfortable, and by making those changes, paving the way for an even broader group in their wake.

      How many men found their way into the field by playing video games? How many corporate environments originally formed to cater to them, and now need to reorient themselves? Do you think the era of “booth babes” at trade shows is over? The company party I went to _last_month_ took place in a bar with go-go dancers in bikinis suspended in cages from the ceiling. It made _me_ uncomfortable, and I’m a man! It made my female co-worker furious and she filed a complaint. Good for her.

      How much more reliable could software be, if companies could raise their retention rate, not by simply throwing more money at the problem, but by establishing practices that are convenient for adult women and inspire loyalty, like on-site childcare, in-house charter schooling, flexible leave and vacation schedules, or really good telecommuting infrastructure? One new program that particularly interests me, is a partnership my company has established with a local college. Developers can teach, or assist in teaching, the programming classes at the college as part of their job duties. The program is in pilot and a much greater ratio of women signed up for it than the ratio of women to men in the company. I signed up for it too. Respect for a greater diversity of appearance and behavior improves the situation for unconventional men too.

      How many dev teams would function more smoothly if the macho fist-pounding gorilla types had to learn to hold their behavior in check because making decisions the Man Way is not even theoretically the norm now, and patiently building consensus the Woman Way results in fewer mistakes?

      Pointless and damaging gender and racial bias in the tech industry is NOT over. The transitions and adaptations the industry must make to maximize recruitment and retention are ongoing. We’re not at the point where “just stop talking about it” is the better tactic. If you believe we are, well, you’ve been reading too many scholarly articles and not getting enough ground truth.

  31. Maybe, just maybe, there is a gap between the sexes in tech because of different interests between the sexes? Just like in chess, mma fighting and vidya. People apparently think that evolution, primarily the evolution of sexual dimorphism in humans, stops at the neck. This assumption leads to unnecessary, or perceived problems that can’t be fixed.

    1. Even if that’s true, that gender dimorphism doesn’t stop at the neck, the crowd that’s trying to inflate the importance of traditional gender roles certainly aren’t a credible source for any truth on the matter whatsoever.

      1. Unlike the “crowd” that is desperately reaching for a narrative to continue seeing women as a victimized class, despite every privilege and opportunity uniquely afforded to women and girls, and their dominance in education, both as teachers and as students?

        The people who insist loudly they aim to disrupt the status quo have the educational system and politics on their side, with feminist organizations like N.O.W. overturning attempts at fairer legislation, feminist politicians like Obama or Trudeau repeating debunked statistics, and even the U.N. producing reports about violence and harassment that quietly need to be retracted, for being sourced from fairy tales. Who lacks credibility here?

  32. I wonder whether it matters whether the interviewer is male or female. Would that make a difference? Are names revealed? I’m not familiar with the platform and from the article I got that they want to keep things anonymous so I’m supposing they’re not. But maybe just first names? I so, that might jeopardise the gender masking.
    And last, is there a possibility that we subconsciously pick up on gender in the way people talk? I’m totally going out on a limb here but if we want to get to the bottom of this we need to take all possible factors into account, right?

  33. Christopher Allen

    > As it happens, women leave interviewing.io roughly 7
    > times as often as men after they do badly in an interview.

    If men and women were coming in with equal skill levels—and therefore equally likely to do badly on their first interview—shouldn’t this mean that on average women will do *better* than men, because the male scores will be weighed down by stubbornly persistent incompetent men?

    It would be interesting to see what the average male and female scores are on their first interview vs. average male and female scores on their 2nd, 5th, etc.

    (I suppose it’s possible that there will be no difference if the most competent women are also being removed from the interviewing pool because they’re snapped up by employers keen to hire them.)

    1. Those are some really good points. How do we know that the women who seemed to “quit” prematurely, didn’t actually end up getting work through another platform for example? That’s pure speculation of course, but the mere possibility undercuts the author’s entire thesis; that “oh I guess women are rarely successful in tech because they just don’t try hard enough.”

      This “study” simply isn’t rigorous enough or verified enough to draw clear, meaningful conclusions from. It seems it is unfortunately *more* then enough to be used to further entrench prejudices already there, though.

  34. My experience toward attrition and how men and women cope with it.

    Men usually just tell each other straight away they’ve done bullshit, even in harsh word, cause it IS bullshit. No biggie, no hard feelings. My work has been critizied, not me. I can just do it better next time.
    (e.g. Linus’ mailinglist)

    Telling the same thing to a women, and she doubts herself, her abilities. She’s not just brushing it off. (Not true for all women, obviously but that’s what I’ve noticed.)

    In school I have noticed this aswell. Boys’ work was criticized more harshly. Maybe teachers thought (or noticed by experience) they could take it? We had some girls who began to cry when their work was criticized the way boy’s were (Whether this was genuine or they just played the teachers, I dont know). And there were always more boys in class and none of them cried or was really openly upset.
    As a result, teacher’s were generally more lax about the girl’s work.
    (Sure, this is all subjective, but may hold some clues, so sharing it anyway)

    1. My career was electrical construction engineering on the power side. At first female project participation was limited to interior design and a few architects. The number of female architects exploded and they also began working their way into the engineering side. As a rule they believed their degree made them qualified. As we all know, a degree means you are highly trainable and that is all.

      My experience over time with many female designers is they have a much higher error rate (as evidenced by error and omissions change orders), they are very risk averse (leaning upon men for the tough decisions), and react emotionally to criticism or a demand they make the tough decisions themselves.

      I have never seen a man cry in a design meeting or construction progress meeting when confronted with design shortcomings or construction failures. I have seen many women shed tears over this. And invariably, the macho construction men ride to the rescue of the damsel in distress. The women know this works and the men can’t help themselves.

      In essence, much higher failure rates (and the associated costs) are tolerated to accommodate female ineptitude and emotionalism in an industry where such traits were once quickly excised for the good of the project, the team, and the companies involved.

      Hate my post if you want. It is based upon 35 years of experience beginning in 1979 (male only) and ending in 2014 (gynocentric tolerant). I’m sorry, but facts are facts.

      1. So the newcomers to your field weren’t as good at – or as secure in – their work as the seasoned professionals judging them, and all the people with vaginas were newcomers?

        Mighty fine horse you’ve got there, but I don’t suppose you’ve taken into account… It does look suspiciously like a cart… And that thing you’re calling a cart just took a big crap on the road…

  35. Thank you for posting this. It was an interesting read, and I look forward to seeing any future results as you continue to look at this.

    I wanted to suggest that if we divide people into the (false but useful for these purposes) binary of women and men and look at average starting points on your theoretical attrition curve, we shouldn’t expect them to be in the same place. That is, by the time  women are at the stage of being interviewed for jobs, they have, on average, encountered quite a number more potential attrition events. I am a PhD candidate in a STEM field and I have always had a great love of mathematics. I started getting discouraging feedback even as a child, but there was plenty more to come in university. There are more formal results that suggest this is not an atypical experience for women, and that women encounter this type of negative feedback more often than men do in STEM. So one possibility is that women and men have the same attrition curve, but women are generally much further along it by the time they are at the interview stage. But even if it is not the case that the curve is the same, I would still expect women to be further along their curve. (Assuming, of course, that it even works to have such a curve. I recognise that in that part of the article you were more just musing on the topic and not presenting it as solid data, and I am similarly participating in this type of thinking out loud. It’s quite possible that a single curve like this lacks the right parameters to really model it.) I mean, hell, even just reading these comments there are far more people insisting that women just aren’t as good at tech than there are suggesting that men aren’t as good at tech. The worst position men seem to be put in is of being merely the same as women. What is the cumulative effect of seeing this in the comments of every piece on this topic? Does that push any women further along their attrition curve?

    Another alternative is this:
    If the disparity disappears once you weed out people who quit after one or two interviews, then perhaps it isn’t really about which gender dusts themselves off better but some other parameter that happens to have been correlated with gender for non-direct reasons. It’s possible you happen to have more women than men who are near that “final straw” point when entering the interviews, and that people already considering giving up don’t perform as well in interviews. Especially since for those remaining people the performance seems even across gender.

    Something that isn’t clear to me:
    When you say most people opted in for the the study, does it apply to both interviewers and interviewees? That is, did all of the interviewers opt in or did any choose not to? If there are any interviewers who didn’t participate, I would be concerned about the bias that would introduce. It could, for instance, select for interviewers who don’t see the merit in the experiment because they are fine with the gender gap or accept stereotype-based explanations for it, and maybe these people are more likely to have a negative impact in women’s rankings and performance. Or could introduce bias of some other origin.

    One last bias thought:
    Interviewers who know they are being evaluated might behave differently. I don’t mean they do it deliberately or with some intention to mislead, but that they suddenly have a heightened awareness of the issue and that might slightly shift how they behave because it is present in their thoughts. How did the subset of unmodulated interviewees compare to the modulated groups and to the pre-study numbers you produced? Any difference there?

    To anyone in the comments who hasn’t read it, I highly recommend Cordelia Fine’s book “Delusions of Gender”. It presents, critiques, and synthesizes a fair bit of research around aptitude and gender taken from neuroscience and psychology. Perhaps counterintuitively, I especially wish to recommend the book to those who already support women in STEM, because it will provide you with published references that back up your explanations when talking to those who don’t want us around (who are much less likely to be willing to simply read the book). And there is enough material that even someone well read on the topic has a high likelihood of encountering some new studies or new takes on them. It’s quite accessibly written and also available as an audiobook and I swear I have no connection whatsoever to the author. 🙂

    1. “Women and men” is not a false binary and asserting otherwise just demonstrates that you’re out of step with reality.

      1. I’m not sure of the nature of your objection here. Is it about labels or about being unaware of the full set of categories?

        Perhaps we do not live in the same country, so to clarify: where I live, the words “women” and “men” generally refer to gender and “male” and “female” refer to biological sex. Admittedly, I don’t know if that is the norm elsewhere.

        Having said that, neither gender nor biological sex is actually binary. Gender is cultural and refers to socially constructed behavioural expectations, so it is by its very nature something that varies depending when and where you are. I am going to assume I don’t need to further elaborate on how gender in particular is not binary, but if you’re still unclear just google for “gender versus sex” or something similar. There are plenty of articles about it, as well as formal definitions at bodies that provide resources and standards like the APA.

        As for biological sex, it simply is not the case that human’s either have XX or XY for sex chromosomes. To be sure, these are the most common two configurations, but there are people with a single X or XXY or XYY or XXYY as well as a number of considerably less common configurations including XXX, XXXX, and others which you can find by googling if you really care.

        In fact, there are even XX males because there are (rare) circumstances during meiosis that can leave the SRY gene (which is normally male) on an X chromosome.

        There is also the issue of genotype and phenotype and people with XY chromosomes who are insensitive to androgens.

        For all that, gender is even more expansive if we start trying to nail down formal boundaries to categories. The complexity of all of this is why I said it wasn’t an unreasonable simplification at this time to artificially binarise things, but when we create a simplified model it is useful to at least acknowledge that we’ve done so.

  36. The logic of coming to the conclusions from the study are not totally clear to the reader in this article (which is great anyways!). If I got this right, the conclusion for performance in IT job interviews and (more speculatively) in tech jobs in general is this:

    Due to it taking less discouragements for women to drop out than for men, the average woman-in-tech is more of a novice than the average man-in-tech, and because of that (and just that) women perform less well on average.

  37. Honestly, I am quite surprised by the flaws of your method. In your hypothesis, you assume the gender gap stems from interviewers’ bias that might be negative towards women, i.e. at the same level of qualification, women are perceived to be less competent than men. You assume that you can mask gender by altering the interviewee’s voice. However, the voice is only one part of communication, the other being, obviously, your choice of words etc., how much you want to show off, how self-confident you are. In other words: As a woman, you are trained to communicate in a different way than men, this starts as early as pre-K.

    Your hypothesis assumes a sort of crude sexism that might have prevailed a couple of decades ago. Apart from some very old and some very angry men, nobody believes this today. Things are actually more subtle.

    To all the commentors who think that women are just not as into tech as men: Just shut up. Take a look across times and cultures, and you will find that the question of what is a typically female and a typically male occupation changes widely.

    1. Sorry to disappoint you…. but the participation rate and results show that women aren’t really cut for this. Sure, there are lots of women in the industry… but not as many as men and not as good either. So Facts. If you are good, kudos. Congrats.

    2. Jason Voorhees

      >To all the commentors who think that women are just not as into tech as men: Just shut up.

      It is a demonstrable fact that this is the case. Yes, it may very well be subject to change, as were a lot of things in the past, but the rock solid fact remains that women, right now, are much less into tech than men are.

    3. Yeah, devil interviewers recognizing a woman’s behaviour behind a man voice and punishing her just for being a woman… *sigh* Accept it, men are generally better in a tech world. In fact, the loudly women that complains a lot about this tend to be the ones with less public work covering their statements

    4. karen straughan

      “To all the commentors who think that women are just not as into tech as men: Just shut up. Take a look across times and cultures, and you will find that the question of what is a typically female and a typically male occupation changes widely.”

      Yes, and oddly enough, in many cultures that are more patriarchal, you are more likely to find women in computer science, engineering, tech, etc, than in cultures that feminism has had its way with for a hundred years. Isn’t that interesting?

  38. “Rather, it’s about women being bad at dusting themselves off after failing.” Ooor it’s about a society which teaches girls and women they are never good enough?

    1. Or maybe men and women are inherently different and men can be better at some things than women are, and vice versa. Maybe technical jobs are not the strong suit of the majority of women. Maybe biology created gender roles and not society. Maybe it’s okay for women to be different from men but still allowed the same opportunity. Maybe we don’t have to pretend men and women are exactly the same. Maybe we can cherish our differences instead of highlighting them and acting like it’s some massive conspiracy.

    2. Peter, you ever hear of Ada Lovelace? She is considered to be the world’s first programmer. While Charles Babbage worked on building the first calculators, he only ever saw them as calculators; but Ada realized that data itself, *any* data could be represented by states within a machine, and more importantly that instructions could be given to the machine to change it’s operation, in a word, programming. She predicted the very foundation of our technology before it even existed!

      But she was born in 1815, and very few thinkers were willing to take a woman’s ideas seriously. Her theories on computation would have to wait more than a century before they were validated by the work of Alan Turing and others. (fun fact: Joan Clarke was a brilliant cryptanalyst who worked with Turing, but she also has been somewhat neglected by history)

      Think about that for just a moment. You use the internet every day, less than 100 years after Turing’s first machine that we can think of as a modern “computer” What wonders, what technologies could we have reaped if nearly 200 years had passed since Ada Lovelace’s work had started the digital age in her lifetime?

      This is the very real, and deeply tragic cost of sexism in tech. By shutting out an entire section of people; you’re shutting out the ideas and opportunities they bring to the table. True, maybe eventually a few get to contribute; or maybe eventually more homogenous groups will innovate; but in the meantime we’re all losing out by not having access to the ideas and opportunities diversity brings. In the long run, men lose out too when women are pressured to stay away from tech.

      (P.S. One more honorable mention is of Navy Rear Admiral Grace Hopper, who invented the first modern compiler. Ada did invent the theory of computation and describe, though she lacked tools to execute it; but Grace Hopper is the reason your computer has more sophisticated programming then a glorified calculator. Countless other women have contributed to the growth of the digital age, though far too many of them have gone unnoticed and underappreciated.)

      1. There’s a big difference between “Group A isn’t as good as Group B in Thing” than “Group A is completely incapable of doing Thing”.

      2. karen straughan

        @Kent, regarding the story of Ada Lovelace.

        You seem to be laboring under the misconception that revolutionary or innovative ideas were immediately accepted by the establishment when men presented them, back then.

        Ignaz Semmelweis was the privileged male who proposed that hand-washing could reduce infection rates in hospitals. At the time, germs had not yet been discovered as a source of disease, and the “nervous excitation” theory was still prevalent. He initiated a program in the hospital where he worked of having staff wash their hands in chlorinated lime between patients. Childbed mortality rates were reduced by 80% in just one year.

        He went to his superiors with his findings. They wouldn’t listen. He showed them the results. They didn’t care. He was fired. Immediately, childbed mortality rates rocketed right back ot to previous levels.

        More than this, he was shunned by the entire medical community, was unable to find steady employment afterward, and ended up dying penniless and friendless in a mental institution.

        Too bad the rampant sexism against men at the time (early 1800s) prevented the establishment from being accepting of his radical ideas. If only the matriarchy had listened, how many women and babies could have been saved? /s

      3. This strikes me as a particularly odious false equivalency, given that Ada Lovelace was of the gender that wasn’t even allowed to vote until 70 years after she was dead and buried. (1850, vs 1930 in England, roughly.)

  39. Pingback: Scientists built voice modulation to mask gender in technical interviews. Here’s what happened. | Curtis Ryals Reports

  40. A thought on your initial results of “no apparent change”. Maybe your assumption that voice modulation is a gender mask is incorrect? There might be subtle cues in language choice or communication style that are not affected by the voice modulation.

    You might be able to determine effectiveness by asking the interviewer to guess interviewee gender at the end of the interview.

    1. karen straughan

      Okay. Maybe there are give-aways. But since there was a slightly higher score for women without the disguise than women disguised as men, and for men who were disguised as women over undisguised men (as in, whether man or woman, the test subject did slightly better if they *sounded* like a woman than if they *sounded* like a man)… well, I don’t know whether the interviewers were detecting the gender of the applicant from other subtle cues. Why wouldn’t they have detected those subtle cues from men modulated to sound like women, and given them the same slightly lower scores as unmodulated males? Why would natural sounding women get slightly higher scores than women modulated to sound like men?

      The slight increase was consistent: whoever sounded like a woman did better than they did when they sounded like a man.

      1. The slight increase was also not statistically significant, given that we’re looking at a mere 238 people. So let’s not start writing any theses just yet.

  41. Pingback: We built voice modulation to mask gender in technical interviews. Here’s what happened. – interviewing.io blog – Pingie.com

  42. “Why I’m not depressed by our results and why you shouldn’t be either”

    And IF results would prove that women are generally worse at computer science, why in heaven’s name would that make you depressed?

    I am a female programmer and I don’t care… I am also a runner and I don’t care that man in general are faster. Should we make some very elaborate study, that would prove that mabe they are faster, but that’s only because women are more shy? To hell with that!

    1. Steady, there! You can’t just come in here throwing the narrative for a loop!

      That said… I’ve been working in tech (mostly software and hardware development) for several years, mostly in hiring. I have hired both men and women, but never had men quit or elect to work lower responsibility (which tends to correlate to fewer hours or personal commitments to deadlines) just because they had kids or something. I have had numerous female employees ask to reduce work hours or simply not return from maternity leave — only to pop up as contributors on the open-source side later on.

      Technical ability? I don’t know. I’ve met remarkably talented people of both genders (and in-between genders and genders I never knew) and never really gave it a second thought… but I have certainly observed that women tend to opt-out of the high responsibility roles more than men. When men opt-out it is usually because they are actually burned-out (which sucks a lot worse, imo — some people never recover from that).

  43. Pingback: We built voice modulation to mask gender in technical interviews. Here’s what happened. - pinyourstory.in is India’s biggest and definitive platform for startups and entrepreneurs related stories, news, resources, research reports and analysis of the

  44. I think it has more to do with culture. People in the nerdy world tend to fail at social cohesion with people who have interests outside of the nerdy world. This is true of any culture, but I think it’s worse in any profession historically male dominated. People who don’t feel they stand together with their peers tend to feel rejected. I think in combination with women being more sensitive to rejection and women tending to be the social butterfly it’s natural to want to find a career where it feels right. Which usually isn’t one where you need to be good at fending off doubts. My self identity is rebellious to expectations outside of my own and proud of my geeky-ness. I don’t struggle with that. I totally see it with other women. That feeling of rejection is less a bias and more a cultural gap that further feeds into a feeling of rejection.

  45. Great blog! I really enjoy it and this post was no exception. I recounted it to my wife, who is very non-tech and also quite uncompetitive, and she was not very surprised by it. She is supremely uninterested in all things tech, and I think we all tend to forget that much (most?) of humanity shares her views.

    Slightly apropos, perhaps, and definitely only anecdotal, but here is a story from a doctor I happened to have a longer conversation with. He said he had twins – boy and girl – and when they were young he coached both their soccer teams. On the differences between genders, he described the boys as follows:

    If you noticed in a game that they weren’t passing enough, at half time if you said ‘team, you need to pass the ball! I want you to pass the ball more!’ each one of them would be convinced that he was talking to someone else. To them it wasn’t criticism, and it had nothing to do with them.

    The girls, he said, were exactly the opposite: each was convinced that he was talking to them and them alone. And, importantly, they would perceive it as criticism and reacted quite strongly.

    I suspect this is a complicated area, and I really enjoy the exploration.

  46. >> Why I’m not depressed by our results and why you shouldn’t be either

    Here’s a crazy hypothesis. Maybe, just maybe, men and women are different? Maybe they have different strengths and aptitudes? Crazy right? I know!!!

      1. The curves extrapolated from two data points each……the stupid – it burrrnssss!

      2. karen straughan

        @Mad beans.

        It’s a strength to be willing or able to continue even when discouraged by a negative result. (It can also be a weakness, but still…)

        There are plenty of aggregate differences in men’s and women’s personalities that can explain wildly different attrition rates despite similar levels of ability.

        Those differences in tech seem to persist even in the face of the “tech is for girls! Grrl power!” messaging that’s been going on for a decade or more, and despite the fact that girls have been getting more positive feedback in school (in terms of teacher/student interactions and in terms of grades) than boys since the mid 1990s (this is a well-guarded finding on the part of the UUAW in 1994, no less, that has never been publicized by them–both boys and girls agreed that boys got more attention from teachers in class, but both boys and girls ALSO agreed that the entire surplus and then some was negative attention, and that girls got much more praise and encouragement than boys).

        On top of this, several studies now in the UK and Canada, as well as the US, have discovered that female teachers in primary and middle school grade boys’ work down compared to the assessment of gender-blinded evaluators. By the time you get to high school and university, the trend includes male teachers, and extends to giving girls bonus points compared to evaluators who are gender blinded, rather than just docking males. More than this, by halfway through primary school, boys can detect the bias from female teachers–when asked to bet on what grade they’d get on a paper, they estimated much lower if they knew that the evaluator would be female and would know they were a boy than if they believed the evaluator was a man or would not be told they were a boy, and in primary school, some 85% or more of teachers are women.

        This has been going on for some time, with little to no fanfare, all while male drop-out rates have been increasing relative to girls, and their likelihood of entering college goes down.

        So what we seem to have is a conundrum. It seems that there is a significant amount of systemic bias against boys in school–everything from how likely you are to be scolded by a teacher to being docked marks just for being a boy. (I can direct you to the work of Dr. James Brown, who’s worked in education from kindergarten to university for over 50 years, and he could tell you things that would make your hair turn white).

        And yet somehow the environment that does all this creates men who are much less likely to give up after a single bad interview than their female peers.

        How can this be, if there is no difference in aptitude. Perhaps the difference in aptitude is seated within a personality trait, rather than a technical ability?

      3. Absolutely not disagreeing with you here, but would you mind dropping some links – even if just to another link aggregation? This sounds like a goldmine of evidence that I’d like to have on hand for the future.

      4. karen straughan

        You can google: venus mars measuring global sex differences in personality

        It will almost certainly take you to a really cool study.

        As for the stuff on bias in education, I can ask James Brown for his links (mine were all lost when my old computer got spilled on).

      5. Ah yes, I’ve seen that study. It is very interesting.

        As far as the education links go, if it’s not too much trouble to get ahold of them, I’d very much appreciate them, thanks!

      6. In my last two interviews (both of which landed me jobs) I used the following line: “I believe three personality traits make me a good programmer. I am lazy, stubborn, and suspicious.”

        Then I waited a beat, and said (more or less) “I’m lazy so I’m constantly looking for a way to automate things. I’m stubborn, so when I hit a bizarre bug, I throw everything at it. And I’m suspicious, so I insert logging hooks, and write tests, and check configurations.”

        It gets a laugh and it’s a good humble-brag line at the same time. But the reason I bring it up is, you’ve touched on the greater level of external encouragement that women receive, and the greater reaction they have to its absence, and that stands in contrast to my admission of stubbornness as a positive trait. I was self-taught, and pushed ahead against an uncooperative and vexing machine to learn my trade, in a complete absence of positive feedback from any living soul. Sheer bloody-mindedness, as Sir Pratchett would call it, was at the very center of this pursuit and career from the beginning and to the present day.

        This is really not a fulfilling career for anyone who is easily discouraged.

        Nevertheless, I have met many good female developers, and they all showed that same bloody-mindedness. I’m glad they’re around. I would never say that workplace sexism is neutral for them, let alone works in their favor, though. It’s a lot harder for them to get their jobs done when some hard-to-isolate group of co-workers is keeping a minimum distance out of fear, and another hard-to-isolate group is constantly distracting themselves by asking “is she flirting with me?” over and over in their heads every time they stand over the same screen.

  47. So this experiment shows there is a ‘women bonus’ in tech. Men should demand equal treatment under the law. Sexism has no place in the industry.

Leave a Comment

Your email address will not be published.

Scroll to Top