We ran the numbers, and there really is a pipeline problem in eng hiring.

If you say the words “there’s a pipeline problem” to explain why we’ve failed to make meaningful progress toward gender parity in software engineering, you probably won’t make many friends (or many hires). The pipeline problem argument goes something like this: “There aren’t enough qualified women out there, so it’s not our fault if we don’t hire them.”

Many people don’t like this reductive line of thinking because it ignores the growing body of research that points to unwelcoming environments that drive underrepresented talent out of tech: STEM in early education being unfriendly to children from underrepresented backgrounds, lack of a level playing field and unequal access to quality STEM education (see this study on how few of California’s schools offer AP Computer Science for instance), hostile university culture in male-dominated CS programs, biased hiring practices, and ultimately non-inclusive work environments that force women and people of color to leave tech at disproportionately high rates.¹

However, because systemic issues can be hard to fix (they can take years, concerted efforts across many big organizations, and even huge socioeconomic shifts), the argument against the pipeline problem tends to get reduced to “No, the candidates are there. We just need to fix the bias in our process.”

This kind of reductive thinking is also not great. For years, companies have been pumping money and resources into things like unconscious bias training (which has been shown not to work), anonymizing resumes, and all sorts of other initiatives, and the numbers have barely moved. It’s no wonder tech eventually succumbs to a “diversity fatigue” that comes from trying to make changes and not seeing results.

We ran the numbers and learned that there really IS a pipeline problem in hiring — there really aren’t enough women to meet demand… if we keep hiring the way we’re hiring. Namely, if we keep over-indexing on CS degrees from top schools, and even if we remove unconscious bias from the process entirely, we will not get to gender parity. And yes, there is a way to surface strong candidates without relying on proxies like a college degree. We’ll talk about that toward the end.

Our findings ARE NOT meant to diminish the systemic issues that make engineering feel unwelcome to underrepresented talent, nor to diminish our ability to work together as an industry to effect change — to enact policy changes like access to CS in public schools, for instance. Our findings ARE meant to empower those individuals already working very hard to make hiring better who find themselves frustrated because, despite their efforts, the numbers aren’t moving. To those people, we say, please don’t lose sight of the systemic problems, but in the short term, there are things you can do that will yield results. We hope that, over time, by addressing both systemic pipeline issues and biases, we will get to critical mass of women from all backgrounds in engineering positions, and that these women, in turn, will do a lot to change the pipeline problem by providing role models and advocates and by changing the culture within companies.

Lastly, an important disclaimer before we proceed. In this post, we chose to focus on gender (and not on race). This decision was mostly due to the dearth of publicly available data around race and intersectionality in CS programs/bootcamps/MOOCs.² While this analysis does not examine race and intersectionality, it is important to note that we recognize: 1) Not all women have the same experience in their engineering journey, and 2) tech’s disparities by gender is no more important than the lack of representation of people of color in engineering. We will revisit these subjects in a future post.

The percentage of women engineers is low and likely worse than reported

It’s very hard to have a cogent public conversation about diversity when there is no standardization of what statistic means what. As this post is about engineering specifically, we needed to find a way to get at how many women engineers are actually in the market and work around two big limitations in how FAAMNG (Facebook, Amazon, Apple, Microsoft, Google, and Netflix) report their diversity numbers.

The first limitation is that FAAMNG’s numbers are global. Why does this matter? It turns out that other countries, especially those where big companies have their offshore development offices, tend to have a higher percentage of female developers.³ In India, for instance, about 35% of developers are women; in the U.S., it’s 16%. Why are these numbers reported globally? The cynic in me says that it’s likely because the U.S. numbers, on their own, are pretty dismal, and these companies know it.⁴ To account for this limitation and get at the U.S. estimate, we did some napkin math and conservatively cut each company’s female headcount by 20%.

The second limitation is that reported numbers are based on “technical roles,” which Facebook at least defines very broadly: “A position that requires specialization and knowledge needed to accomplish mathematical, engineering, or scientific related duties.” I expect the other giants use something similar. What are the implications of this fairly broad definition? Given that product management and UX design count as technical roles, we did some more napkin math and removed ~20% to correct for PMs and designers.⁵

With these limitations in mind, below is a graph comparing the makeup of the U.S. population to its representation in tech at FAAMNG companies where said data was available, as well as an estimate of women in engineering specifically.

If we want to reach gender parity in engineering, especially when we correct for women in the U.S. (and whether they’re actually software engineers), you can see that we have a long way to go.

Is it a pipeline problem?

So, are there just not enough qualified women in the hiring pool? It turns out that we’re actually hiring women at pretty much the same rate that women are graduating with CS degrees from four-year universities — out of the 71,420 students who graduated with a CS degree in 2017, 13,654, or ~20%, were women.⁶ So maybe we just need more women to get CS degrees?

Top tech companies and their non-profit arms have been using their diversity and inclusion budgets to bolster education initiatives, in the hopes that this will help them improve gender diversity in hiring. Diversity initiatives started taking off in earnest in 2014, and in 4 years, enrollment in CS programs grew by about 60%. It’s not anywhere near enough to get to gender parity.

And even if we could meaningfully increase the number of women enrolling in CS programs overall, top companies have historically tended to favor candidates from elite universities (based on some targeted LinkedIn Recruiter searches, 60% of software engineers at FAANG hold a degree from a top 20 school). You can see enrollment rates of women in 3 representative top computer science programs below. Note that while the numbers are mostly going up, growth is linear and not very fast.

To see if it’s possible to reach gender parity if we remove unconscious bias but keep hiring primarily from top schools, let’s build a model. For the purposes of this model let’s focus solely on new jobs — if companies want to meet their diversity goals, at a minimum they need to achieve parity on any new jobs they’ve created. Based on the US BLS’s projections, the number of software engineering jobs is estimated to increase by 20% by 2028 (or about 1.8% annually). Today, the BLS estimates there are about 4 million computer-related jobs. This projects to about 70,000 new jobs created this year, increasing to 85,000 new jobs created in 2028.

If the goal is to hit gender parity in the workforce, our goal should be to have 50% of these new seats filled by women.

To see if this is possible, let’s project the growth of the incoming job pool over the same timeframe. Based on NCES’ 2017 survey, computer science graduates have grown annually anywhere between 7% and 11% this decade. Let’s optimistically assume this annual growth rate persists at 10%. Let’s also assume that the
percentage of graduates who are women remains at 20%, which has been true for the last 15 years. But, there are some gotchas.

First, there’s no guarantee that the seats earmarked for women actually get filled by women, particularly in a world where male CS graduates will continue to outnumber females 4-to-1. Not all of these jobs will be entry-level, so some portion of these jobs will be pulling from an already female-constrained pool of senior candidates. Finally, there’s no guarantee that traditional 4-year colleges will be able to support the projected influx of computer science candidates, particularly from the top-tier universities that companies usually prefer. Below, we graph the net new seats we’d need to fill if women held half of software engineering jobs (blue line) vs. how many women are actually available to hire if we keep focusing largely on educational pedigree in our recruiting efforts (red line). As you can see, it’s not possible to hit our goals, whether or not we’re biased against women at any point in the hiring process.⁷

So if the pipeline is at least partially to blame, what can we do?

You saw above that enrollment in undergraduate computer science programs among women is growing linearly. Rising tuition costs coupled with 4-year universities’ inability to keep up with demand for computer science education have forced growing numbers of people to go outside the system to learn to code.

Below is a graph of the portion of developers who have a bachelor’s degree in computer science and the portion of developers who are at least partially self-taught, according to the Stack Overflow Developer Surveys from 2015 to 2019. As you can see, in 2015, the numbers were pretty close, and then, with the emergence of MOOCs, there was a serious spurt, with more likely to come.

The rate of change in alternative, more affordable education is rapidly outpacing university enrollment. Unlike enrollment in traditional four-year schools, enrollment in MOOCs and bootcamps is growing exponentially.

In 2015 alone, over 35 million people have signed up for at least one MOOC course, and in 2018 MOOCs collectively had over 100M students. Of course, many people treat MOOCs as a supplement to their existing educational efforts or career rather than relying on MOOCs entirely to learn to code. This is something we factored into our model.

Despite their price tag (most charge on the order of $10-20K), bootcamps seem like a rational choice when compared to the price of top colleges. Since 2013, bootcamp enrollment has grown 9X, with a total of 20,316 grads in 2018. Though these numbers represent enrollment across all genders⁸ and the raw number of grads lags behind CS programs (for now), below you can see that the portion of women graduating from bootcamps is also on the rise and that graduation from online programs has actually reached gender parity (as compared to 20% in traditional CS programs).

Of course, one may rightfully question the quality of grads from alternative education programs. We factored in bootcamp placement rates in building our updated model below.

Outside of alternative education programs, the most obvious thing we can do to increase the supply of qualified women engineers is to expand our pipeline to include strong engineers who don’t hail from top schools or top companies.

In previous posts, we looked at the relationship between interview performance and traditional credentialing and found that participation in MOOCs mattered almost twice as much for interview performance than whether the candidate had worked at a top company. And top school was least predictive of performance and sometimes not at all. And some of my earlier research indicates that the most predictive attribute of a resume is the number of typos and grammatical errors (more is bad), rather than top school or top company. In this particular study, experience at a top company mattered a little, and a degree from a top school didn’t matter at all.

But, even if lower-tier schools and alternative programs have their fair share of talent, how do we surface the most qualified candidates? After all, employers have historically leaned so hard on 4-year degrees from top schools because they’re a decent-seeming proxy. Is there a better way?

But culling non-traditional talent is hard… that’s why we rely on pedigree and can’t change how we hire!

In this brave new world, where we have the technology to write code together remotely, and where we can collect data and reason about it, technology has the power to free us from relying on proxies, so that we can look at each individual as an indicative, unique bundle of performance-based data points. At interviewing.io, we make it possible to move away from proxies by looking at each interviewee as a collection of data points that tell a story, rather than a largely signal-less document a recruiter looks at for 10 seconds and then makes a largely arbitrary decision before moving on to the next candidate.

Of course, this post lives on our blog, so I’ll take a moment to plug what we do. In a world where there’s a growing credentialing gap and where it’s really hard to figure out how to separate a mediocre non-traditional candidate from a stellar one, we can help. interviewing.io helps companies find and hire engineers based on ability, not pedigree. We give out free mock interviews to engineers, and we use the data from these interviews to identify top performers, independently of how they look on paper. Those top performers then get to interview anonymously with employers on our platform (we’ve hired for Lyft, Uber, Dropbox, Quora, and many other great, high-bar companies). And this system works. Not only are our candidates’ conversion rates 3X the industry standard (about 70% of our candidates ace their phone screens, as compared to 20-25% in a typical, high-performing funnel), about 40% of the hires made by top companies on our platform have come from non-traditional backgrounds. Because of our completely anonymous, skills-first approach, we’ve seen an interesting phenomenon happen time and time again: when an engineer unmasks at the end of a successful interview, the company in question realizes that the student who just aced their phone screen was one whose resume was sitting at the bottom of the pile all along (we recently had someone get hired after having been rejected by that same company 3 times based on his resume!).

Frankly, think of how much time and money you’re wasting competing for only a small pool of superficially qualified candidates when you could be hiring overlooked talent that’s actually qualified. Your CFO will be happier, and so will your engineers. Look, whether you use us or something else, there’s a slew of tech-enabled solutions that are redefining credentialing in engineering, from asynchronous coding assessments like CodeSignal or HackerRank to solutions that vet candidates before sending them to you, like Triplebyte, to solutions that help you vet your inbound candidate pool, like Karat.

And using these new tools isn’t just paying lip service to a long-suffering D&I initiative. It gets you the candidates that everyone in the world isn’t chasing without compromising on quality, helps you make more hires faster, and just makes hiring fairer across the board. And, yes, it will also help you meet your diversity goals. Here’s another model.

How does changing your hiring practices improve the pipeline?

Above, you saw our take on status quo supply and demand of women engineers — basically how many engineers are available to hire using today’s practices versus how many we’d need to actually reach gender parity. Now, let’s see what it looks like when we include candidates without a traditional pedigree (yellow line).

As you can see, broadening your pipeline isn’t a magic pill, and as long as demand for software engineers continues to grow, it’s still going to be really hard, systemic changes to our society notwithstanding. If we do make these changes, however, the tech industry as a whole can accelerate its path toward gender parity and potentially get there within a decade.

What about specific companies? An interactive visualization.

So far we’ve talked about trends in the industry as a whole. But, how do these insights affect individual employers? Below is an interactive model where you visualize when Google, Facebook, or your company (where you can plug in your hiring numbers) will be able to hit their goals based on current hiring practices versus the more inclusive ones we advocate in this post. Unlike the industry as a whole, built into this visualization is the idea of candidate reach, as well as hire rates — one company can’t source and hire ALL the women (as much as they might want to). Of course, the stronger your brand, the higher your response rates will be.

We made some assumptions about response rates to sourcing outreach for both Google and Facebook. Specifically, we guessed a 60%-70% response rate for these giants based on the strength of their brand and their army of sourcers — when those companies reach out and tenaciously follow up, you’ll probably respond eventually.⁹ We also made some assumptions about their hire rates (5-10% of interviewed candidates). You can see both sets of assumptions below. And you can see that even with all the changes we propose, in our model, Google and Facebook will still not get to gender parity!

We also included a tab called “Your company” where you can play around with the sliders and see how long it would take your company to get to gender parity/whether it’s possible. There, we made much more conservative assumptions about response rates!

As you can see, for the giants, getting to gender parity is a tall order even with broadening your pipeline to include non-traditional candidates. And while it may be easier for smaller companies to get there without making drastic changes, when you’re small is exactly the right time to get fairer hiring into your DNA. It’s much harder to turn the ship around later on.

Conclusion

Regardless of whether you’re a giant or a small company, as long as hiring practices largely limits itself to top schools, the status quo will continue to be fundamentally inefficient, unmeritocratic, and elitist, and any hope of reaching gender parity will be impossible. Look, there are no easy fixes or band-aids when it comes to diversifying your workforce. Rather than continuing to look for hidden sources of women engineers (I promise, we’re not all hiding somewhere, just slightly out of reach) or trying to hire all the women from top schools, the data clearly shows that the only path forward is to improve hiring for everyone by going beyond top schools and hiring the best people for the job based on their ability, not how they look on paper.

I was recently in a pitch meeting where I got asked what interviewing.io’s mission is. I said that it’s to make hiring more efficient. The investors in the room were a bit surprised by this and asked, given that I care about hiring being fair, why that’s not the mission. First off, “fair” is hard to define and open to all manners of interpretation, whereas in an efficient hiring market, a qualified candidate, by definition, gets the job, with the least amount of pain and missteps. In other words, meritocracy is a logical consequence of efficiency. Secondly, and even more importantly, while I firmly believe that most people at companies want to do “the right thing”, it’s much easier to actually do the right thing in a big organization when it’s also cheaper, better, and faster.

All that’s to say that there are no shortcuts, and the most honorable (and most viable) path forward is to make hiring better for everyone and then hit your diversity goals in the process (or at least get closer to them). Software engineering is supposed to be this microcosm of the American dream — anyone can put in the work, learn to code, and level up, right? Until we own our very conscious biases about pedigree and change how we hire, that dream is a hollow pipe.

Thank you to the interviewing.io data & eng team for all the data modeling, projections, and visualizations, as well as everyone who proofread the myriad long drafts.

Footnotes