Gender-bias in Academia: The Case Strumia

Dr Alessandro Strumia is a physicist working at CERN, where he is Principal Investigator of an ERC Advanced research grant on the topic “Understanding the mass scales in nature.” Ie, he spends your tax money on the kind of research that I criticize in my book. He also recently published an analysis of publication and citation-rates in his field. At a workshop on gender diversity at CERN last weekend, he used this analysis to argue that women are underrepresented in physics because they are underperforming.

I did not attend the workshop and have not seen a recording of the talk, but I have seen the slides (a PDF version of which is here). The slides contain statements that are both inaccurate and exceedingly unprofessional.

For example, he begins his talk by stating that “smarter people are less affected by implicit bias,” but this is wrong. Studies have shown repeatedly that intelligence does not protect from thinking biases. Yes, intelligence is useful to overcome certain types of biases (mostly those that can be exposed with mathematical reasoning), but only once people are aware they are biased to begin with.

Strumia’s mistaken belief that intelligent people are less affected by cognitive biases does not remotely surprise me. I have encountered this very same attitude (“We are too smart to be biased!”) among almost all high-energy theorists and phenomenologists I have spoken with about the issue. That in itself is a bias, known as the “bias blind spot.”

But that Strumia is ill-informed about the very topic he speaks about at a scientific workshop is not the biggest problem with his presentation. Far worse is that he names and attacks two women, apparently because he is annoyed he did not get a job that he was shortlisted for. Nonsense like this just does not belong in a research presentation.

After complaints ballooned on social media on Monday, CERN pulled the slides from the net quickly and has since suspended Strumia. What will happen to his ERC grant is unclear. A large number of members of the particle physics community have meanwhile signed a statement declaring that they distance themselves from the content of Strumia’s talk.

Now, as you know, I have also recently taken up bibliometric analysis. And I admit I found some of the data Strumia showed interesting. We did, in our paper, also look at gender differences, but not for citation counts. We looked at an entirely different quantity, that of research broadness, and for this we did not find any gender differences.

The gender difference that Alessandro Strumia and his co-author Ricardo Torre find is huge. It’s a more than 100% difference in the total number of citations that researchers accumulate throughout their career.

I don’t think that the number of citations is a good measure for scientific performance, but if the difference between the genders was so large, it might mean that women and men chose their research projects in distinctly different ways. That would be interesting. I thus decided to look into this for a bit.

The key figure that Strumia presents on his slides is the total number of citations that researchers accumulate since the publication of their first paper:

Figure from slide 16 of Alessandro Strumia’s talk.

That the horizontal axis is labeled “scientific age” is unfortunate because this term has been coined to emphasize that the scientific age might differ from the chronological time passed since PhD or first paper. If a researcher takes a career-break, for example because of health reasons or for parental leave, their scientific age goes on hold. However, there is presently no standardized way to determine the scientific age, and in any case, you couldn’t do it from publication data alone, you’d also need biographic information.

Since women are more likely to take leave for child-raising, their citations should on the average increase somewhat slower, simply because they have more breaks in which they don’t publish. However, it seems unlikely that this would make such a huge difference. So, while the label on the axis is inaccurate, I don’t think it’s all that relevant.

When I saw this graph, however, another worry came to my mind immediately. When we did our previous analysis, we found that the vast majority of people who use the arXiv publish only one or two papers and are never heard of again. This is in agreement with the well-known fact that the majority of physicists drop out of academic careers.

I am not sure why this surprised me when it showed up in the data. Maybe because, if you work in the field, the drop-outs are pretty much invisible. They leave and you forget about them. But they are there, in the stats, big and fat.

Now, the total number of citations for such drop-outs will accumulate very slowly because they don’t publish new papers any more. And we know that women are more likely to drop out – that’s the “leaky pipeline” and reason why I find myself increasingly often, if not the only woman in the room, then at least the oldest woman in the room. And I’m only 42.

If you leave the drop-outs in the citation analysis, the leaky pipe will pull down the average of female authors more than of male authors.

I hence asked one of our PhD students, Tobias Mistele, to plot the same quantity as Strumia did for our data sample, but to only keep authors who have more than 5 papers in total, and who have published a paper in the last 3 years. This is sloppy way to shrink down the pool to “active researchers only.” It’s maybe not the most sophisticated way to do it, but it should give us an idea how large the contribution from the drop-outs is.

If we plot the number of citations for active researchers only, we see no noticeable difference between men and women:

arXiv data, active researchers only

When normalized to the number of authors per paper (as Strumia did), there is also no noticeable difference between men and women.

I must add a warning here. We do not use the same data set as Strumia and Torre. They use data from inspire, we use data from the arXiv. This means our data set does not reach back in time as far, and it includes disciplines besides high energy physics. So the absolute numbers are not directly comparable.

Another caveat I must add is that we are using a different method to identify male and female authors. We use the author-id algorithm that is explained in our earlier paper, and then try to match first names with a database for common anglo-saxon first names. Naturally, this means that the authors who remain in our sample are most likely to be of Western origin. By this method we assign a gender to 19% of authors. This is in contrast to Strumia and Torre who use a more elaborate gender-id procedure that allows them to match 60%. The remaining authors in our sample break down to 70,295 male und 53,165 female researchers. After applying the above mentioned cuts, we are left with 12,654 male and 8,177 female. That’s not a huge number, but decent.

Let me also mention that probably a similar effect is behind another finding in Strumia’s talk. He points out that women, on the average, are hired into faculty positions earlier. A paper that appeared on the arXiv yesterday argued that this is not a signal that women have an unfair advantage, but simply a consequence of women leaving at a higher rate. If they aren’t hired early, they’ll not be hired at all, which means the average age of hiring is smaller.

Finally, to state the obvious, this is a blogpost, not a paper. The above is a quick and dirty way to check whether removing dropouts significantly affects the large difference between men and women, and the answer seems to be yes. However, we will have to do a more careful analysis to arrive at definite conclusions. I haven’t checked my biases.

---

I want to thank Tobias Mistele for doing the graphs so quickly and Alessandro Strumia and Ricardo Torre for helpful communication.