The present phase of stagnation in the foundations of physics is not normal

Nothing is moving in the foundations of physics. One experiment after the other is returning null results: No new particles, no new dimensions, no new symmetries. Sure, there are some anomalies in the data here and there, and maybe one of them will turn out to be real news. But experimentalists are just poking in the dark. They have no clue where new physics may be to find. And their colleagues in theory development are of no help.

Some have called it a crisis. But I don’t think “crisis” describes the current situation well: Crisis is so optimistic. It raises the impression that theorists realized the error of their ways, that change is on the way, that they are waking up now and will abandon their flawed methodology. But I see no awakening. The self-reflection in the community is zero, zilch, nada, nichts, null. They just keep doing what they’ve been doing for 40 years, blathering about naturalness and multiverses and shifting their “predictions,” once again, to the next larger particle collider.

I think stagnation describes it better. And let me be clear that the problem with this stagnation is not with the experiments. The problem is loads of wrong predictions from theoretical physicists.

The problem is also not that we lack data. We have data in abundance. But all the data are well explained by the existing theories – the standard model of particle physics and the cosmological concordance model. Still, we know that’s not it. The current theories are incomplete.

We know this both because dark matter is merely a placeholder for something we don’t understand, and because the mathematical formulation of particle physics is incompatible with the math we use for gravity. Physicists knew about these two problems already in 1930s. And until the 1970s, they made great progress. But since then, theory development in the foundations of physics has stalled. If experiments find anything new now, that will be despite, not because of, some ten-thousands of wrong predictions.

Ten-thousands of wrong predictions sounds dramatic, but it’s actually an underestimate. I am merely summing up predictions that have been made for physics beyond the standard model which the Large Hadron Collider (LHC) was supposed to find: All the extra dimensions in their multiple shapes and configurations, all the pretty symmetry groups, all the new particles with the fancy names. You can estimate the total number of such predictions by counting the papers, or, alternatively, the people working in the fields and their average productivity.

They were all wrong. Even if the LHC finds something new in the data that is yet to come, we already know that the theorists’ guesses did not work out. Not. A. Single. One. How much more evidence do they need that their methods are not working?

This long phase of lacking progress is unprecedented. Yes, it has taken something like two-thousand years from the first conjecture of atoms by Democritus to their actual detection. But that’s because for most of these two-thousand years people had other things to do than contemplating the structure of elementary matter. Like, for example, how to build houses that don’t collapse on you. For this reason, quoting chronological time is meaningless. We should better look at the actual working time of physicists.

I have some numbers for you on that too. Oh, yes, I love numbers. They’re so factual.

According to membership data from the American Physical Society and the German Physical Society the total number of physicists has increased by a factor of roughly 100 between the years 1900 and 2000.* Most of these physicists do not work in the foundations of physics. But for what publication activity is concerned the various subfields of physics grow at roughly comparable rates. And (leaving aside some bumps and dents around the second world war) the increase in the number of publications as well as in the number of authors is roughly exponential.

Now let us assume for the sake of simplicity that physicists today work as many hours per week as they did 100 years ago – the details don’t matter all that much given that the growth is exponential. Then we can ask: How much working time starting today corresponds to, say, 40 years working time starting 100 years ago. Have a guess!

Answer: About 14 months. Going by working hours only, physicists today should be able to do in 14 months what a century earlier took 40 years.

Of course you can object that progress doesn’t scale that easily, for despite all the talk about collective intelligence, research is still done by individuals. This means processing time can’t be decreased arbitrarily by simply hiring more people. Individuals still need time to exchange and comprehend each other’s insights. On the other hand, we have also greatly increased the speed and ease of information transfer, and we now use computers to aid human thought. In any case, if you want to argue that hiring more people will not aid progress, then why hire them?

So, no, I am not serious with this estimate, but I it explains why the argument that the current stagnation is not unprecedented is ill-informed. We are today making more investments into the foundations of physics than ever before. And yet nothing is coming out of it. That’s a problem and it’s a problem we should talk about.

I’ve recently been told that the use of machine learning to analyze LHC data signals a rethinking in the community. But that isn’t so. To begin with, particle physicists have used machine learning tools to analyze data for at least three decades. They use it more now because it’s become easier, and because everyone does it, and because Nature News writes about it. And they would have done it either way, even if the LHC would have found new particles. So, no, machine learning in particle physics is not a sign of rethinking.

Another comment-not-a-question I constantly have to endure is that I supposedly only complain but don’t have any better advice for what physicists should do.

First, it’s a stupid criticism that tells you more about the person criticizing than the person being criticized. Consider I was criticizing not a group of physicists, but a group of architects. If I inform the public that those architects spent 40 years building houses that all fell to pieces, why is it my task to come up with a better way to build houses?

Second, it’s not true. I have spelled out many times very clearly what theoretical physicists should do differently. It’s just that they don’t like my answer. They should stop trying to solve problems that don’t exist. That a theory isn’t pretty is not a problem. Focus on mathematically well-defined problems, that’s what I am saying. And, for heaven’s sake, stop rewarding scientists for working on what is popular with their colleagues.

I don’t take this advice out of nowhere. If you look at the history of physics, it was working on the hard mathematical problems that led to breakthroughs. If you look at the sociology of science, bad incentives create substantial inefficiencies. If you look at the psychology of science, no one likes change.

Developing new methodologies is harder than inventing new particles in the dozens, which is why they don’t like to hear my conclusions. Any change will reduce the paper output, and they don’t want this. It’s not institutional pressure that creates this resistance, it’s that scientists themselves don’t want to move their butts.

How long can they go on with this, you ask? How long can they keep on spinning theory-tales?

I am afraid there is nothing that can stop them. They review each other’s papers. They review each other’s grant proposals. And they constantly tell each other that what they are doing is good science. Why should they stop? For them, all is going well. They hold conferences, they publish papers, they discuss their great new ideas. From the inside, it looks like business as usual, just that nothing comes out of it.

This is not a problem that will go away by itself.

---

If you want to know more about what is going wrong with the foundations of physics, read my book “Lost in Math: How Beauty Leads Physics Astray.”

---

* That’s faster than the overall population growth, meaning the fraction of physicists, indeed of scientists of general, has increased.

New paper claims that LIGO’s gravitational wave detection from a neutron star merger can’t be right

Two weeks ago, New Scientist warmed up the story about a Danish groups’ claim that the LIGO collaboration’s signal identification is flawed. This story goes back to a paper published in Summer 2017.

After the publication of this paper, however, the VIRGO gravitational wave interferometer came online, and in August 2017 the both collaborations jointly detected another event. Not only was this event seen by the two LIGO detectors and the VIRGO detector, several telescopes also measured optical signals that arrived almost simultaneously and fit with the hypothesis of the event being a neutron-star merger. For most physicists, including me, this detection removed any remaining doubts about LIGO’s event-detection.

Now a few people have pointed out to me that the Journal of Cosmology and Astroparticle Physics (JCAP) recently published a paper by an Italian group which claims that the gravitational wave signal of the neutron-star merger event must be fishy:

GRB 170817A-GW170817-AT 2017gfo and the observations of NS-NS, NS-WD and WD-WD mergers
J.A. Rueda et al
JCAP 1810, 10 (2018), arXiv:1802.10027 [astro-ph.HE]

The executive summary of the paper is this. They claim that the electromagnetic signal does not fit with the hypothesis that the event is a neutron-star merger. Instead, they argue, it looks like a specific type of white-dwarf merger. A white-dwarf merger, however, would not result in a gravitational wave signal that is measurable by LIGO. So, they conclude, there must be something wrong with the LIGO event. (The VIRGO measurement of that event has a signal-to-noise ratio of merely two, so it doesn’t increase the significance all that much.)

I am not much of an astrophysicist, but I know a few things about neutron stars, most notably that it’s more difficult to theoretically model them than you may think. Neutron stars are not just massive balls that sit in space. They are rotating hot balls of plasma with pressure gradients that induce various phases of matter. And the equation of state of nuclear matter in the relevant ranges is not well-understood. There’s tons of complex and even chaotic dynamics going on. In short, it’s a mess.

In contrast to this, the production of gravitational waves is a fairly well-understood process that does not depend much on exactly what the matter does. Therefore, the conclusion that I would draw from the Italian paper is that we are misunderstanding something about neutron stars. (Or at least they are.)

But, well, as I said, it’s not my research area. JCAP is a serious journal, and the people who wrote the paper are respected astrophysicists. It’s not folks you can easily dismiss. So I decided to look into this a bit.

First, I contacted the spokesperson of the LIGO collaboration, David Shoemaker. This is still the same person who last year answered my question what the collaboration’s response to the Danish criticism is by merely stating he has full confidence in LIGO’s results. Since the Danish group raised the concern that the collaboration suffers from confirmation bias, this did little to ease my worries.

This time I asked Shoemaker for a comment on the Italian groups’ new claim that the LIGO measurement conflicts with the optical measurements. Turns out that his replies landed in my junk folder until I publicly complained about the lack of response, which prompted him to try a different email account. Please see update below.

Secondly, I noticed that the first version of the Italian group’s paper that is available on the arXiv heavily referenced the Danish group.

Curiously enough, these references seem to have entirely disappeared from the published version. I therefore contacted Andrew Jackson from the Danish group to hear if he has something to say about the Italian group’s claims and whether he’d heard of them. He didn’t respond.

Third, I contacted the corresponding author of the Italian paper, Jorge Rueda, but he did not correspond with me. I then moved on to the paper’s second author Remo Ruffini, which was more fruitful. According to Wikipedia, Ruffini is director of the International Centre for Relativistic Astrophysics Network and co-author of 21 textbooks about astrophysics and gravity.

I asked Ruffini whether he had been in contact with the LIGO collaboration about their findings on the neutron star merger. Ruffini did not respond to this question, though I asked repeatedly. When I asked whether they have any reason to doubt the LIGO detection, Ruffini referred me to (you’ll love this) the New Scientist article.

I subsequently got Ruffini’s permission to quote his emails, so let me just tell you what he wrote in his own words:

“Dear Sabine not only us but many people are questioning the Ligo People as you see in this link: the drama is of public domain. Remo Ruffini”

Michael Brooks, btw, who wrote the New Scientist article knew about the story because I had written about it earlier, so it has now gone around a full circle. After I informed Ruffini that I write a blog he told me that:

“we are facing the greatest dramatic disaster in all scientific world since Galileo. Do propagate this dramatic message to as many people as possible.”

Yo.

Update: Here is the response from Shoemaker that Google pushed in the junk folder (not sure why). I am sorry I complained about the lack of response without checking the junk folder - my bad.

He points out that there is a consensus in the community that the gravitational wave event in question can be explained as a neutron-star merger. (Well, I guess it’s a consensus if you disregard the people who do not consent.) He also asks me to mention (as I did earlier) that the data of the whole first observing run is available online. Alas, this data does not include the 2017 event that is under discussion here. For this event only a time-window is available. But for all I can tell, the Italians did not even look at that data.

Basically, I feel reassured in my conclusion that you can safely ignore the Italian paper.

2nd Update: The non-corresponding corresponding author of the Italian paper has now popped up after being alerted about this blogpost. He refuses to comment on his co-author’s claims that LIGO is wrong and the world needs to be told. Having said this, I wish all these people would sort out their issues without me.

Modified gravity, demystified [video]

Here is the promised follow-up on my earlier video about dark matter. This time I explain how Modified Newtonian Dynamics gives rise to flat rotation curves and what’s the deal with the Tully-Fisher relation. Fixed my make-up issues but now I put the microphone in the wrong place, hence the noise from my shirt. Sorry about that. Click on “CC” in the bottom bar to get English captions.


The problem with the language of the automatic transcription disappeared as spontaneously as it had appeared. As you can see, if it works, it works remarkably well, except that it’s missing all punctuation.

Update: Now available with German and Italian captions. Click on gear icon/subtitles in bottom bar to change language.

Guest Post: Phillip Helbig reviews “Lost in Math”

[Phillip Helbig worked in cosmology and gravitational lensing at Hamburg and Jodrell Bank Observatories and the Kapteyn Astronomical Institute. Although no longer employed in academia, he regularly attends conferences and writes book reviews for The Observatory, as well as the occasional journal paper. Phillip is a regular commenter on this blog.]

I've read a huge number of popular-science books, and my first impression is that Sabine's book is very well written. One could also think that Sabine is a native speaker (or, rather, a native writer) of English. The style is breezy without rambling, and direct quotations make it clear what the illustrious interviewees actually said, without any filter of interpretation (but see below for a caveat). Sabine's own position is very clear; this is almost an op-ed. Whether or not one agrees with her, this approach is preferable to introducing one's own biases into what might appear to the uninitiated as an objective description.

Enough praise; now for the critique. Let me emphasize, though, that I agree with everything which I don't discuss here, which is most of the book. In the interest of stimulating discussion, I'll concentrate on those few areas where I see things differently.

It is not always clear what needs to be explained. In discussions of fine-tuning and so on, one often reads about numerical coincidences, which imply that two numbers are roughly the same, but also about small (or large) numbers, which allegedly also need an explanation. (Since the inverse of a large ratio is a small ratio, I will speak only of small numbers in what follows.) It needs to be clear what is even potentially puzzling: it is always ratios near 1. In other words, if the smallness of some quantity is the result of a near cancellation, then that implies a ratio near 1 of the quantities which almost cancel; if the number is just small in relation to some other quantity because it has nothing to do with that other quantity, then it certainly needs no explanation.

Another aspect of the presentation I disagree with is the claim that the standard model has been "souped up" with dark matter and dark energy, as if these were some sort of epicycles, fudge factors brought in so that theory and observations match. On her blog, Sabine has often pointed out that general relativity says nothing about the sources of gravitation, so while dark matter might be interesting or even mysterious because we don't know what it is, it is not some sort of addition to general relativity. The same goes for the cosmological constant. Yes, Einstein initially introduced it as a fudge factor, and later abandoned it, but the universe is independent of the contingent history via which we have learned about it. From a mathematical point of view, one could just have easily included the cosmological constant from the beginning. Indeed, in other areas of physics, what is not forbidden actually happens, and if someone claims that something doesn't happen, that some quantity is 0, etc, then the burden of proof is on the person making the claim. Actually, what is interesting is that no fudge factors have had to be introduced. Despite a huge amount of cosmological data, a model with just a few parameters---all of which were known even back when there was almost no data---which was derived when there were some data but considerably less than now still fits the observations.

I have tremendous respect for George Ellis. However, I don't always agree with him, even on matters of science. I think that Sabine lets him too easily off the hook because they seem to agree on many issues. Ellis dismisses the idea that we could be living in a simulation, but is careful to point out that science cannot disprove the existence of God. One could just as well say that we cannot disprove that we are living in a simulation and dismiss the idea of God. Strictly speaking, one can disprove neither, but can use various arguments to discuss the probabilities of both. Also, after criticizing certain ideas as being non-scientific, Ellis says of one of his own ideas, that nothing is physically infinite: "There's no way I can prove it.... But we should use it as a principle." This isn't the place to argue with Ellis; my point is that if Sabine could be obstinate enough to stay in Weinberg's office even after he had essentially asked her to leave, she should have called out these two obvious contradictions on the part of Ellis. I think that this is a good example of confirmation bias. (Interestingly, Tegmark is also critical of the idea of physical infinity but, in contrast to Ellis, is a strong proponent of the multiverse.)

My main disagreement with Sabine concerns fine-tuning. I think that this is due to an unnecessary attachment to probability. Many normally think that fine-tuning and low probability go hand in hand. As Sabine points out, though, without knowledge of the underlying probability distribution, one cannot say whether an anthropic explanation involving the multiverse leads to likely values. But is that even necessary? One can discuss fine-tuning for life, in the sense that slight changes of various parameters (within the otherwise allowed range) would lead to a universe incompatible with life. There can be absolutely no debate that the universe is fine-tuned in this sense. Whether the values we observe for the physical constants are likely in some sense is unknown, but also irrelevant. One must be careful not to confuse fine-tuning in the particle-physics sense of lack of naturalness (discussed above) with the case of values being within a small region of possible parameter space (regardless of how likely that small region is by some definition.) As an aside, it is not true, as Sabine claims on p. 114, that fine-tuning goes away if one considers many changes simultaneously. A good discussion of fine-tuning, which also rebuts many common objections, including that one, can be found in the book by Lewis and Barnes.

It is also beside the point whether, also mentioned on p. 114, somewhere in parameter space there is another region compatible with life; the point is that most of it is not. A good comparison is with the "coincidence" that the Earth is just at the right distance of the Sun for the existence of life. The explanation is simple: there are many solar systems with planets at various distances from their stars. By chance, some will be at the right distance for life. It is also completely irrelevant how likely these are, as long as the probability is non-zero. The same goes for the multiverse. Given the multiverse (perhaps a daunting proposition), then fine-tuning is not puzzling at all. A good case for the multiverse is made by Max Tegmark. (Lewis and Barnes mention the multiverse in a book about fine-tuning; Tegmark does the opposite.) I think that most examples of fine-tuning are real; again, the book by Lewis and Barnes is a good summary. In one famous case of alleged fine-tuning I disagree, and that is the flatness problem. I wrote an entire paper about that, so I won't say much about it here. I'm also sure that most of the people who have thought much about the multiverse don't make this simple mistake.

Suppose I flip a coin a hundred times and it comes up heads every time. It seems that Sabine would say that this outcome is just as probable as any other outcome (which is true) and therefore that there is no reason to assume that the coin is not fair (which is false). I think that most people would disagree with Sabine, and I agree with those people. If one must discuss probabilities in conjunction with fine-tuning, or vice versa, what is relevant is not the probability per se, but rather the probability relative to some situation which is important to us.

A common theme in Sabine's book is that fundamental physics, having become "lost in math", has not made much progress in recent decades. This is a correlation, but is there a causation? Perhaps the problems are just really hard theoretically, and experimentally nothing is accessible at the moment. In neither case would this be the first time that something like this has happened. Thus, while I sympathize with the main theme of the book, I don't think that there is a watertight case for the claim.

Perhaps other approaches will be more successful, but the burden of proof is on those who make such claims. Yes, maybe progress is difficult due to lack of funding for those thinking outside the box, and without funding, it is difficult to prove whether an alternative approach would pay off.

Does beauty distract us from truth? Perhaps, in some cases, but in these I claim (probably agreeing with Sabine here) that it is not beauty per se, but rather a false sense of beauty. Aesthetics in some sense, perhaps something similar to Pirsig's "quality", has been a useful guide in some cases. At the end of the day, though, the route to truth doesn't matter; a successful theory is a successful theory regardless of the path trough which it was arrived at.

---

Some comments from me:

First, as you can see, Phillip unfortunately used his review to propagate his own notion of fine-tuning. I therefore want to warn you that this is not the way most physicists use the word and therefore not the way I use the word in my book. Please don't let yourself get confused.

Second, Ellis correctly points out that the simulation hypothesis is not science because you cannot disprove it. This is totally in line with him saying that science cannot disprove god. And, yes, Ellis puts forward metaphysical principles, but in contrast to the other physicists I spoke to, he is aware that these are unprovable.

Third, I discuss the issue of fairness in the interview with Weinberg using the example of poker. It's a useless objection because we have equally little idea what counts as "fair" in the multiverse as we know the probability distribution. Neither of those are notions that make sense scientifically.

Fourth, I address the often-raised claim that progress has slowed down because "the problems are just hard" right in the beginning of the book. To sum it up once again: no one can tell how much of the slow-down is due to the problems being harder, but certainly using flawed methodologies will not help.

Fifth, I don't think there is anything like a "false sense of beauty". You decide what is beauty for yourself. Just don't mistake your sense of beauty for a scientific criterion.

Self-driving car rewarded for speed learns to spin in circles. Or, how science works like a neural net.

When I write about problems with the current organization of scientific research, I like to explain that science is a self-organizing, adaptive system. Unfortunately, that’s when most people stop reading because they have no idea what the heck I am talking about.

I now realized there is a better way to explain it, one which has the added benefit of raising the impression that it’s both a new idea and easy to understand: Science works like a neural network. Or an artificial intelligence, just to make sure we have all the buzzwords in place. Of course that’s because neural networks are really adaptive systems, neither of which is really a new idea, but then even Coca Cola sometimes redesigns their bottles.

In science, we have a system with individual actors that we feed with data. This system tries to optimize a certain reward-function and gets feedback about how well it’s doing. Iterate, and the system will learn ways to achieve its goals by extrapolating patterns in the data.

Neural nets can be a powerful method to arrive at new solutions for data-intensive problems. However, whether the feedback loop gives the desired result strongly depends on how carefully you configure the reward function. To translate this back to my going on about the malaises of scientific research, if you give researchers the wrong incentives, they will learn unintended lessons.

Just the other day I came across a list of such unintended lessons learned by neural nets. Example: Reward a simulated car for continuously going at high speed, and it will learn to rapidly spin in a circle:

limited state info from sim though... was wondering if maintaining high speed was a reasonable reward for RL? nope... #kenBlock pic.twitter.com/rr3H1jL3gQ

— mat kelcey (@mat_kelcey) July 15, 2017

Likewise, researchers rewarded to produce papers at a high frequency will learn to rapidly spin around their own axis by inventing and debating problems that don’t lead anywhere. Some recent examples from my own field are the black hole firewall, the non-naturalness of the Higgs-mass, or the string theory swampland.

Here is another gem: “Agent pauses the game indefinitely to avoid losing.” I see close parallels to the current proliferation of theories that are impossible to rule out, such as supersymmetries and multiverses.

But it could be worse, at least we are not moving backwards:

I hooked a neural network up to my Roomba. I wanted it to learn to navigate without bumping into things, so I set up a reward scheme to encourage speed and discourage hitting the bumper sensors.

It learnt to drive backwards, because there are no bumpers on the back. https://t.co/8PdR3p6ePZ

— Custard Smingleigh (@Smingleigh) November 8, 2018

At least we are not moving backward yet. Because now that I think about it, rediscovering long-known explanations would also be a good way to feign productivity.

Of course I know of the persistent myth that scientific research is evaluated by its ability to describe observations, so I must add some words on this: I know that’s what you were told, but it’s not how it works in practice. In practice, scientists and funding agencies likewise must evaluate hypotheses prior to test to decide what is worth the time and money of testing to begin with. And the only ones able to evaluate the promise of research directions are researchers themselves.

It follows that there is no external reward function which you can impose on scientists that will optimize the return on investment. The best – indeed the only – method at your disposal is to let scientists make the evaluation internally, and then use their evaluation to distribute funding. In doing this, you may want to impose constraints on how the funding is used, eg by encouraging researchers to study specific topics. Such external constraints will reduce the overall efficiency, but this may be justifiable for societal reasons.

In case you missed it, this solution – which I have written and spoken about for more than a decade now – could come right out of the neo-libertarian’s handbook. The current system is over-regulated and therefore highly inefficient. More regulations will not fix it. This is why I am personally opposed to top-down solutions, like requirements coming from funding agencies.

However, the longer the current situation goes on, the more people we will have in the system who are convinced that what they are doing is the right thing, and the longer it will take for the problem to resolve even if you remove the flawed incentives. Indeed, in my impression the vast majority of scientists today already falls into this category: They sincerely believe that publications and citations are reliable indicators for good research.

Why do these problems persist even though they have been known for decades? I think the major reason is that most people – and that includes scientists themselves – do not understand the operation of the systems that they themselves are part of. It is not something that evolution allowed us to develop any intuitive grasp for.

Scientists in particular by and large think of themselves as islands. They do not take into account the manifold ways in which the information they obtain is affected by the networks they are part of, and neither do they consider that their assessment of this information is influenced by the opinions of others. This is a serious shortcoming in the present education of scientists.

Will drawing an analogy between scientific research and neural nets help them see the light? I don’t know. But maybe then in the not-so-far future we will all be replaced by AIs anyway. At least those sometimes get debugged.

I'm hiring: Postdoc In Quantum Foundations in Frankfurt, Germany

I am looking for a postdoctoral researcher to join me and my small group at the Frankfurt Institute for Advanced Studies for a project in quantum foundations.

This postdoc position is a two year scholarship supported by the Franklin Fetzer Fund. The research project-bound, ie the candidate will work on a particular topic under my supervision. The position comes with a modest travel budget.

Applicants should have a background in quantum foundations or quantum information, especially path integral formalism and decoherence theory. Applications should contain a CV, a list of publications, and at least 2 letters of recommendations. Documents should be sent by email to hossi@fias.uni-frankfurt.de with the subject “Postdoc 2018”.

The application deadline is December 7th, 2018.

The Frankfurt Institute for Advanced Studies is a non-profit research organization located on the North campus of the JW-Goethe University in Frankfurt, Germany. It is an international think-tank that collects researchers pursuing a large variety of topics ranging from physics to neuroscience to economics. The building is new, the people are friendly, and I am not remotely as terrible as they told you.

Further questions should be directed to hossi@fias-uni-frankfurt.de.

Book Review: “Rigor Mortis” by Richard Harris

Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions
Richard Harris
Basic Books (April 4, 2017)

15 years in the foundations of physics taught me little about the universe and much about human behavior. I eventually poured my frustration into a book, which was published a few months ago under the title “Lost in Math” and which documents that bad methodologies survive in scientific communities simply because they enable the continued production of papers.

While the foundations of physics are the research area that I am personally most interested in, its lack of progress arguably has limited societal relevance. Who really cares if we will eventually manage to quantize gravity. If our fruitless attempts at least entertain the masses, maybe that’s justification enough to finance string theorists.

But the same problems exist in other research areas, and in some cases lives are at stake. In his book “Rigor Mortis,” the US-American science journalist Richard Harris has a close look at what is going on in biomedicine and drug development. You may think that my experience with physicists should have warned me, but really I had no idea.

While I follow the popular science literature on drug development to some extent, it is certainly not a topic that I know a lot about. And those popular science accounts tend to be celebrations of the supposedly great breakthroughs, most of which we never hear of again. I was under the impression that since in the life sciences you can at least experimentally check hypotheses, it can’t possibly be as bad as in the foundation of physics. Well, I was wrong.

In “Rigor Mortis,” Harris goes through the various kinds of flawed scientific methodologies that have spread in those research communities. Poor experimental design, hypothesis-fishing, sloppy statistics, mislabeled cell-lines, contaminated but still used antibodies, the abundance of irreproducible results, outright fraud and misconduct, and long-retracted zombie-papers that continue to be cited nevertheless. No longer do I wonder why the development of new drugs has basically stalled and why the alleged breakthrough discoveries never pan out.

Harris makes some efforts to convince the reader that the problem has been recognized and some people try to do something about it. While I appreciate the attempted optimism, that’s lipstick on a pig. Yes, there have been initiatives for this and that, and some of those have indeed partly addressed a specific problem. For example, requiring researchers to pre-register trials prevents them from later changing the hypothesis they were testing. But the overarching problem that the organization of scientific research is inefficient to the point of choking progress still exists and no one is doing anything about it.

Harris book is thankfully short on the actual research studies, where I say “thankfully” because I get lost easily in elaborations about molecules with unpronounceable names and 20 different enzymes that may or may not be doing this or that. For me any article about drug development come down to “this thing fits into that thing and we hope it will have this effect.” Harris does nothing of that sort and instead focuses on the way research is pursued.

I was also relieved to find that Harris largely spares the reader dreadful stories about patients who succumbed to their illness after long suffering. It’s not that I think those stories shouldn’t be told, they have their place, but personally I would prefer if popular science articles stayed clear of them. For me it’s a reason not to read an article if I have to fear someone may die in the next paragraph.

Harris interviewed a few people whose voices appear in some places. His writing is clean and clear and easy to follow, which is to say he writes better than I, damn. It’s not a long book, but it’s full with information, and it’s scary. You should read it.

Story about LIGO noise resurfaces in New Scientist

Cover of New Scientist
Nov 3rd 2018.

The current New Scientist issue has an “exclusive feature” under the headline: “DID WE REALLY FIND GRAVITATONAL WAVES? Breakthrough physics result questioned.”

The article is by Michael Brooks and it’s a summary of a claim I wrote about last year, that the original 2015 gravitational wave detection by the LIGO collaboration was not a real signal.

This claim was made by a Danish group around the physicist Andrew Jackson. This group tried to reproduce the data analysis of the LIGO collaboration with the publicly available data and could not.

The New Scientist article quotes Duncan Brown at Syracuse, who until recently was a member of LIGO, with reassuring the reader that the Danes are “credible scientists,” and Slava Mukhanov who likewise emphasizes that the Danes are people “with a high reputation.” Slava is also on record stating that “There is no mistake” in the analysis of the Danish group. Peter Coles chimes in to say that “I think their paper is a good one and it’s a shame that some of the LIGO team have been so churlish in response.”

The New Scientist article then draws a comparison between the LIGO case and the BICEP case. BICEP looked for the so-called primordial gravitational waves, which are in a different wavelength regime than LIGOs. Their supposed signal turned out to be merely noise.

The two measurements, however, work entirely differently because BICEP did not (attempt to) directly measure gravitational waves. Instead, it looked for a secondary signal that is the imprint of the primordial gravitational waves in the cosmic microwave background. The BICEP signal was contaminated by foreground from the Milky Way. The same problem does not exist for LIGO.

Michael Brooks in the New Scientist article then points out that this is the first time we are analyzing gravitational wave signals and it’s still early days, so if an independent analysis cannot reproduce the result that’s a problem.

Interestingly, Brooks seems to have found out that the key figure in the LIGO paper about the first discovery does not actually show the quantity that they used in the data analysis. I had been told about this previously, though I cannot now recall the details. (I believe it was something about the plotted quantity not actually showing the relevant significance. Anyone knows better, pls leave a comment.)

The way that I heard about it was that some members of the collaboration wanted a pretty plot that “could be printed on a T-shirt,” ie they opted for beauty over scientific relevance. I don’t know if that’s what really happened, but it sounds plausible enough. I recall thinking at the time that if that’s true it was a dumb decision; clearly this move pissed off some people in the collaboration and those had no reason to keep their mouth shut forever.

For me the issue with the Danish group’s criticism was not whether the signal is real. LIGO people pointed out problems with the Danes’ analysis to me that even I could understand. No, the issue for me was that the collaboration didn’t make an effort helping others to reproduce their analysis. They also did not put out an official response, indeed have not done so until today. I thought then – and still think – this is entirely inappropriate of a scientific collaboration. It has not improved my opinion that whenever I raised the issue LIGO folks would tell me they have better things to do.

We have here a group of researchers not associated with the collaboration which tries to follow the analysis methods that the collaboration reported and they cannot confirm the collaboration’s results. This should not happen. If the collaboration is not able to explain their procedures so that other scientists can find out what they’re doing, that is a problem that must be fixed. This is the first time anyone analyses data for gravitational wave signals and the methodology needs to be clearly documented. Evidently, this is not presently the case.

The Danes btw haven’t been the only ones who tried to redo the LIGO analysis and didn’t manage to. I know this not because I’m obsessed with LIGO, but because people send me references about this. I also get plenty of emails and comments from cranks who think that LIGO is a fraud and just wasting tax-money and so on. All this is reason why I think the LIGO collaboration is doing a disservice to science by ignoring the matter.

I was thus happy to read in the New Scientist article that some people from the LIGO collaboration are at least working on a response. But, well, it’s certainly taking some time.

What happened after the Danish group made their claim in June last year is that  the VIRGO collaboration joined LIGO’s search for gravitational waves. So now the analysis draws on data from three detection sites. They have since seen a gravitational wave event with an optical counterpart recorded in several telescopes. Brooks reports that the Danish groups still doubts the detection because this event, which happened in August 2017, was originally labelled a “glitch”. The story about the glitches is indeed peculiar. The glitches are occasional false alarms in the detectors. They tend to not have the frequency spectrum of the real events however. So it seems to me like a stretch that the Danes are holding on to their claim, and I am not sure why New Scientist dug this up now.

If you cannot (or do not want to) access the New Scientist piece, Jennifer Ouellette has an excellent summary on Ars Technica.

---

Update: The LIGO collab has published a brief response to the New Scientist piece on their website:

“1 Nov 2018 -- Claims in a paper by Creswell et al. of puzzling correlations in LIGO data have broadened interest in understanding the publicly available LIGO data around the times of the detected gravitational-wave events. The features presented in Creswell et al. arose from misunderstandings of public data products and the ways that the LIGO data need to be treated. The LIGO Scientific Collaboration and Virgo Collaboration (LVC) have full confidence in our published results. We are preparing a paper that will provide more details about LIGO detector noise properties and the data analysis techniques used by the LVC to detect gravitational-wave signals and infer their source properties.”

Dark matter. Or what? [Video]

Fixed some problems, created some new ones. Sorry for the glassy stare at some places; I had to redo the recording but had a gigantic headache. I promise not to do that again.


Click on “CC” to get English subtitles.

I encountered the same problem with YouTube as with the previous video, that the automatic transcription falsely identifies the language as German. (This time I left it there for your amusement.) I have no idea why this is. I have all my language settings in English and really I don't think my pronunciation is so bad you’d mistake my English for German. The only thing I learned by Googling the issue is that others have it too. In case you know a fix, please let me know.

Oh, and Happy Halloween :)

Update Nov 2nd: Now with Italian and German subtitles. Click “CC” to turn on captions, click on settings/gear icon to change language.

Physics may be up against the wall, anno 1986

I thought you’d enjoy this gloriously prescient NYT article from 1986 which my husband has dug out. In it, Malcom Browne reports that Alan Chodos (then at Yale) worries about the future of particle physics:

“Unable to mount experiments that would require energies comparable to that of the Big Bang genesis event, Dr. Chodos believes, growing numbers of physicists will be tempted to embrace grandiose but untestable theories, a practice that has more than once led science into blind alleys, dogma and mysticism.

In particular, Dr. Chodos worries that “faddish” particle physicists have begun to flock all too uncritically to a notion called “superstring theory.” […] Deprived of the lifeblood of tangible experiment, physicists will “wander off into uncharted regions of philosophy and pure mathematics,'' says Dr. Chodos, leaving true physics to wither.””

Chodos later became the Associate Excecutive Officer of the American Physical Society, a position from which he retired in 2014.

Will it end? [In which I have breakfast with John Horgan]

Taking a selfie with a book on
your face is more difficult than
you may think.

I had breakfast with John The-End-Of-Science-Horgan two weeks ago, and I’m beginning to think it was a mistake.

I had backed out from an after-lecture-dinner two days earlier for which I felt guilty already, so I may have forgotten to mention I actually don’t eat breakfast. To make matters worse, I arrived late that morning because once I stepped into the shower, I noticed there were no towels in the hotel room. And when I had finally managed to dry my hair and find the place, I had to prevent an excited New Jersey taxi driver from having John pay my bill. Then we watched the taxi-man write down my credit card information in sloth-motion.

To celebrate this shitty start into the day, I ordered a coffee, just to learn that John doesn’t drink coffee. Which I should have known because he wrote about his coffee-fast on his blog. Evidently, I didn’t read this. Or maybe I did but immediately forgot about it. Either way, I’m a bad person. Even more so because John promptly also ordered a coffee. Caffeine-free, but still, now I had become somebody’s bad influence. And caffeine-free coffee, I hope y’all know, isn’t actually caffeine-free.

Luckily, the morning improved thereafter. John turned out to be a really nice guy who will cheerfully explain why science is over, which reminds me of the time I accidentally sprinkled herbal salt on a strawberry-jam sandwich. Indeed, he turned out to be so nice that now I was feeling guilty for spending Saturday morning with a nice guy somewhere in New Jersey while my husband watched the kids 4000 miles to the East.

If that makes you think my brain is a pretty fucked-up place, it gets worse from here on. That’s because to work off all that guilt, I did what you do to make authors happy: you go buy their books. So, once back in Germany, I went and bought “The End of Science,” 2015 edition. It was not a good idea.

Horgan’s book “The End of Science” was originally published 1996. I never read it because after attempting to read Stuart Kauffman’s 1995 book “At Home In the Universe” I didn’t touch a popular science book for a decade. This had very little to do with Kauffman (who I’d meet many years later) and very much to do with a basic malfunction of my central processing unit. Asked to cope with large amounts of complex, new information, part of my brain will wave bye-bye and go fishing. The result is a memory blackout.

I started having this in my early 20s, as I was working on my bachelor’s degree. At the time I was living in Frankfurt where I shared an apartment with another student. As most students, I spent my days reading. Then one day I found myself in a street somewhere in the city center without any clue how I had gotten there. This happened again a few weeks later. Interestingly enough, in both cases I was looking at my own reflection in a window when my memory came back.

It’s known as dissociative fugue, and not entirely uncommon. According to estimates, it affects about one or two in a thousand people at least once in their life. The actual number may be higher because it can be hard to tell if you even had a fugue. If you stay in one place, the only thing you may notice is that the day seems rather short.

These incidents piled up for a while. Aside from sudden wake-ups in places I had no recollection of visiting, I was generally confused about what I had or had not done. Sometimes I’d go to take a shower only to find my towel wet and conclude I probably had already taken one earlier. Sometimes I’d stand in the stair case with my running shoes, not knowing whether I was just about to go running or had just come back. I made sure to eat at fixed times to not entirely screw up my calorie intake.

Every once in a while I would meet someone I know or answer the phone while my stupid brain wasn’t taking records. For what I’ve been told, I’m not any weirder off-the-record than on-the-record. So not like I have multiple personalities. I just sometimes don’t recall what I do.

The biggest problem with dissociative fugue isn’t the amnesia. The biggest problem is that you begin to doubt your own ability to reconstruct reality. I suspect the major reason I’m not a realist and have the occasional lapse into solipsism is that I know reality is fragile. A few wacky neurons are all it takes to screw it up.

What has any of this to do with Horgan? Nothing, really, but it’s why I didn’t read his book when it came out. And then, when I met John, he unwittingly reminded me of times I’d rather have entirely forgotten.

Back then I took records of my episodes. It looked like it was primarily popular science books that would bring them on, so I stopped reading those. This indeed mostly solved the problem. That and some pills and a few years of psychotherapy. I can only guess why I never had issues with textbooks, maybe because those tend to be rather narrowly focused.

In any case, for ten years the only thing I read besides textbooks was cheap novels, notably Dean Koontz, whose writing is so repetitive and shallow that I have blissfully forgotten what those books were about. Then, in late 2005 Lee Smolin handed me a draft of his book “The Trouble With Physics” which would appear the following year. And what was I supposed to tell him? So I read Lee’s book. My memory lapses came back with a few months delay, but they were nowhere near as disruptive as earlier. And so, thanks to Lee, I slowly returned to reading popular science books.

With the self-insight that age brings, I’ve noticed my mental health issues are strongly stress-related. I’ve also learned to tell first signs of trouble. The past months I’ve worked too much, traveled too much, and said “yes” too often. It took me two weeks to make my way through the first 50 pages of Horgan’s book. It’s not going well. And so I think for now I better go back to reading Chad Orzel’s new book “Breakfast With Einstein.” Because that’s an easy read about things I know already. I’m sorry, John. Don’t take it personally.

Having said this, I thought it would be good to write down some thoughts about the supposed end of science before reviewing Horgan’s book (should I ever manage to finish it). But first let me show you an advertisement:


I don’t particular like American comedy (neither the intended nor the unintended kind) because I tend to find it unfunny. But this guy with his blender makes me laugh every time. Not sure why, maybe it’s his glassy stare.

In case you’ve never encountered these videos before, it seems to be an advert series featuring an old white guy shredding electronics with his awesome blender. “Will it blend?” he asks and infallibly ends up with a pile of grey dust.

I now picture Horgan stuffing science in his blender, pushing the button asking “Will it end?” This thought-experiment teaches us that science will end as infallibly as the Amazon Echo will blend. Because everything will end. You, and I, and John Horgan, and, yes, even Donald Trump’s complaints about the evil media. Entropy increase will get us all, eventually.

So, yeah, science will end.

But that’s not the interesting question. The interesting question is whether it’s ending right now. On the death bed, flatlining as we speak.

As most scientists, I am willing to argue the opposite, though not because I see all that much progress. On the opposite, it’s because I see so little progress. Scientific research today works extremely inefficiently because scientists waste time and money chasing after well-cited publications in high-impact journals. This inefficiency is problematic, frustrating, infuriating even. But it implies that science has untapped potential.

Whether making science more efficient is possible and whether it would actually make a difference I don’t know. I’ll see what John has to say about that. Which I should have done before I wrote my book.

I’m a bad person. And I promise I’ll read his book, eventually.

String theory pros and cons [video - no singing!]

As I mentioned, I used the prize money I got from the recent FQXi essay contest to buy a new video camera. The main reason for doing this was not better music videos (though there’s that), but that my old camcorder doesn’t have a mic port and the audio quality of the built-in mic is miserable to say the least. Turned out, however, that none of my microphones actually worked with the new camera, so you haven’t heard any more of me in the past months than in the prior 10 years.

After too much time in various forums, I finally found a microphone that works with the camera. Miserable failure that I am, I evidently managed to live to the ripe age of 42 without knowing the difference between a TRS and a TRRS plug. But, hey, now I am enlightened! Next problem was that I used the default mic settings of the camera and the result was so incredibly terrible I threw the recordings away entirely and with it went a whole weekend’s work.

Now I believe I’ve found a mic setting that works okay-ish, which is to say that listening to myself does no longer make my toenails roll up. There is still too much echo but that’s an issue which is unfixable for the time being as I have no other place to do the recording. I could have put some filter and compressor on the audio, which I will try next time.

Then of course I noticed once again too late that my white-balance is jumpy. I suspect this isn’t actually something I do, but something I don’t do, probably also some automatic setting that I’d better turn off. Also, I have screwed up my make-up, but you’re probably used to that. All of this is to say it’s more complicated than it seems and really I’d rather stick to writing.


And, erm, “Yadis and Nau” should have been “Nadis and Yau.” Sorry about that. Click on CC in the tool bar to get English captions.

First stars spell trouble for dark matter

The HERA telescope array
in South Africa [img src].

In the beginning, reheating created a hot plasma of elementary particles. The plasma expanded, cooled, and emitted the cosmic background radiation. Then gravity made the plasma clump, and darkness was upon the face of the deep, whatever that means.

Cosmologists call it the “dark ages,” the period in the early universe where matter is already too cool to emit radiation, but not yet clumpy enough to ignite nuclear fusion. At this time the universe was filled almost exclusively with rather dilute hydrogen gas. It’s not until a billion years after the Big Bang that the first stars light up, an epoch poetically called “cosmic dawn.”

We cannot directly measure light emitted from those first stars, but we can indirectly infer the stars’ presence by studying the cosmic microwave background. That’s because the early stars emit UV radiation which couples to the hydrogen gas, and for some while this coupling enables the gas to absorb light of a specific wavelength – at about 21cm. This leaves a mark in the cosmic microwave background.

The wavelengths of light stretch with the expansion of the universe, so what was 21cm back then is now deep in the radio regime. That makes it difficult to find cosmological signals because other sources – both on earth and in our galaxy – can contaminate the data.

In February, the Experiment to Detect the Global Epoch of Reionization Signature (EDGES) announced they had measured the absorption that stems from the first stars. They found it at the expected wavelength – a few meters – but stronger than the predictions said it should be.

Astrophysicists can make predictions for this absorption by using the concordance model for cosmology. This model has 8 free parameters – one of which is the amount of dark matter –  and the physics of the first stars follows from this straight-forwardly. Besides the cosmological dynamics, it’s only well-known thermodynamics and atomic physics. Compared to the large variety of today’s stars, the first stars were fairly simple. Or at least that’s what astrophysicists thought so far.

It took me a while to get around and read the EDGES paper, and I’ve since tried to find out what, if anything, astrophysicists think about the mismatch with the predictions. The answer is: not much. Most of them think it’ll go away. Maybe a measurement error. I have even been told to not pay attention to the EDGES result because the paper has not been cited all that often. Seriously.

Well, as you can tell, I looked at it anyway. I’m not an astrophysicist and I can’t judge the experimental design of the EDGES collaboration. I can only say that I don’t see obvious flaws with their data analysis. The paper seems fine to me.

Besides the possibility of a measurement error, the theoretical explanations for the signal have so far focused on what type of dark matter could possibly make it work, as the commonly considered ones don’t do the trick.

To explain the EDGES result, dark matter would need a stronger interaction with normal matter than presently assumed. If so, that could shift the temperature between gas and radiation and give rise to more absorption. However, a more strongly interacting type of dark matter is difficult to make compatible with other observations that rule out such interactions.

Stacy McGaugh – the modified gravity dude – had the brilliant idea to see what the absorption signal from the first stars would look like if there was just no dark matter. Turns out this would fit remarkably well with the EDGES data. I say “remarkably well” because the parameters that enter his calculation are known from other measurements already, so no freedom to adjust them.

Fig 1 from McGaugh, PRL 121 (2018) 081305.

The reason why the absorption is stronger without dark matter isn’t hard to understand. The more matter there is in the universe, the faster the expansion decelerates. This means without dark matter, the period in which the gas can interact with the radiation is longer, allowing more absorption.

Now, I have recently developed a soft spot for modified gravity, but I am not terribly convinced of Stacy’s argument. It’s one thing to say that galaxies probe different physics than cosmology and thus a new type of force may kick in on galactic scales. It’s another thing to just throw out dark matter from the concordance model because that screws up the whole fit from which the other parameters stem to begin with. You have to self-consistently extract the whole set of parameters from the data – you need a different model entirely.

Indeed, to recover the benefits of dark matter, Stacy employs rather heavy neutrinos. The masses are on the upper end of what is still compatible with constraints. (That’s not counting the cosmological constraints, which are tighter, because these constraints assume the concordance model, and hence don’t apply for modified gravity.) The neutrinos don’t make a difference for the EDGES signal. Still, the dark-matter-less model does not account for the third acoustic peak of the cosmic microwave background. So you have to choose, get the EDGES absorption right or get the third acoustic peak right. Frankly I’d rather have both.

As we speak, other experimental groups are trying to reproduce the EDGES result. One of them is the Hydrogen Epoch of Reionization Array (HERA) in South Africa that is supposed to be completed next year, another one the Square Kilometer Array (SKA) in Australia, planned to be in full operation by 2024. If they confirm the EDGES result, this may be decisive to distinguish dark matter from modified gravity.

Dear Dr B: What do you actually live from?

Some weeks ago a friend emailed me to say he was shocked – shocked! – to hear I had lost my job. This sudden unemployment was news to me, but not as big a surprise as you may think. I was indeed unemployed for two months last year, not because I said rude things about other people’s theories, but simply because someone forgot to renew my contract. Or maybe I forgot to ask that it be renewed. Or both.

In any case, this happened a few times before, and while my younger self wouldn’t normally let such a brilliant opportunity for outrage go to waste, I now like to pretend that I am old and wise and breathe out bullshit.

After some breathing, I learned that this time my sudden unemployment originated not in a forgotten signature, but on Wikipedia. I missed the ensuing kerfuffle about my occupation, but later someone sent me a glorious photoshopped screenshot (see above) which shows me with a painted-on mustache and informs us that Sabine Hossenfelder is known for “a horrible blog on which she makes fun of other people’s theories.”

The truly horrible thing about this blog, however, is that I’m not making fun. String theorists are happily studying universes that don’t exist, particle physicists are busy inventing particles that no one ever measures, and theorists mass-produce “solutions” to the black hole information loss problem that no one will ever be able to test. All these people get paid well for their remarkable contributions to human knowledge. If that makes you laugh, it’s the absurdity of the situation, not my blog, that’s funny.

Be that as it may, I have given a lot of interviews in the past months and noticed people are somewhat confused about what I actually work on. I didn’t write about my current research in my book because inevitably the physicists I criticize would have complained I wrote the book merely to advertise my own work. So now they just complain that I wrote the book, period. Or they complain I’m a horrible person. Which is probably correct because, you see, all that bullshit I’ve been breathing out now sticks to them.

Horrible person that I am, I don’t even work in the foundations of physics any more. I now work on quantum simulations or, more specifically, on using weakly coupled condensed-matter-systems to obtain information about a different, strongly coupled condensed matter system.

The relation between the two systems stems from a combination of analogue gravity with the gauge-gravity duality. The neat thing about this is that – in contrast to either the gauge-gravity duality or analogue gravity alone – we are dealing with two systems that can (at least in principle) be prepared in the laboratory. It’s about the real world!

This opens the possibility to experimentally test the validity of the gauge-gravity duality, or its applicability to certain systems, respectively. Current experiments (like Jeff Steinhauer’s) aren’t precise enough to actually do this, but the technology in this area is rapidly improving, so I’m hopeful that maybe in a decade or so it’ll be doable.

If that was too much terminology, I’m developing new methods to describe how large numbers of atoms interact at very low temperature.

Today, Tobias Zingg and I have a new paper on the arXiv that sums up our recent results. And that’s what I’ll be working on until my contract runs out for real, in November next year. And then what? I don’t know, but stay tuned and we’ll find out.

Yes, women in science still have a disadvantage.

Women today still face obstacles men don’t encounter and often don’t notice. I see this every day at my front door, in physics, where women are still underrepresented. Among the sciences, it’s physics where the gender-balance is most skewed.

While women are catching up on PhDs, with the ratio now at roughly 20% (US data), women are more likely to leave higher education for good. Among faculty, the percentage of women is down to about 10%. The better the job, the more likely it’s occupied by a man.

Source: APS.

The reasons for this “leaky pipeline” are manifold and no one presently knows for sure which factors are most relevant. Therefore, the question what, if anything, to do about it is hotly debated.

Source: AIP statistics.

On one end of the spectrum are those who think that the current gender-balance correctly reflects qualification and nothing needs to be done. People in this camp explain differences between genders by women’s lack of performance and not discrimination. On the other end are those who think the world will only be a good place if half of physicists are women. In this camp, any under-performance by women is caused by discrimination.

I think both of these extreme positions are unreasonable.

The current gender balance almost certainly does not correctly reflect qualification because a) women have to push back harder on gender stereotypes and b) are more likely to have difficulties combining academia with family life.

Consider this: Last time I got my hair cut, I was informed that women can’t do physics because they can’t think logically. This insight was delivered to me matter-of-factly by a middle-aged, female hair-dresser after I answered her question what I’ve been up to lately. (Wrote a book – About what? – Physics).

I didn’t pursue the matter because I don’t like to argue with people who use sharp instruments near my eyes. But I was glad I didn’t have my daughters with me. They’re still at an age where by default they believe what adults say.

Gender stereotypes like this are everywhere and they almost certainly influence career choices. We all want to belong, and if women feel that science is for men, they’re less likely to pursue this avenue. They lose motivation more easily. They call it quits sooner. This means we are missing out on qualified women and instead fill up the pool with lesser qualified men.

Men tend to underestimate how pervasive these stereotypes are, and how tiresome it is to always be the weird one. It wears you down; doubly and triply so for women who are also members of minorities.

Example: I am constantly accused of being rude, aggressive, and snarky, and have been advised multiple times (by men) to not express myself with certainty. Because that’s offensive, you see. If men behave this way, they’re brilliant geniuses, and one cannot blame them for what is merely an expression of their enthusiasm. I’m left to constantly apologize for being who I am. Or figure this: Last time I won an award, the speaker who was supposed to summarize my achievements felt the need to point out that I do great work despite being short.

Yes, I have a lot of anecdotes, and they can be summarized as a pronounced lack of respect. I’m as much a professional physicist as the men around me, yet I’m not treated the same way. I’m LOOK-A-WOMAN!

However, I try not to draw conclusions from my own experiences because the very fact that I’m still working in physics is evidence I’m not suffering all that greatly. I’d go so far to say most of my colleagues are nice guys, and even the assholes don’t normally mean to be assholes. But I hear what my female colleagues have to say, and most of them are really frustrated about the nonsense they have to deal with. And, yeah, I too have been mistaken for the secretary.

The other major disadvantage that women face is that they are hit harder by the family-unfriendliness of academia. Turn it how you wish, women are still responsible for procreation, and female fertility rapidly declines past the age of 40.

Unfortunately, the years between 35 and 40 are also critical to establishing yourself as researcher. Most physicists presently don’t secure permanent positions until their early 40s. Up to then, job-hopping and frequent international moves are the norm. Taking time off to raise kids is difficult, and the inevitable decrease in academic output and flexibility is a competitive disadvantage. A priori this disadvantage exists both for men and women, but women are on the average the younger part of the couple, and, needless to say, pregnancy and nursing is an extra burden.

Arguments that women just perform worse than men are fundamentally flawed because they’re based on shaky interpretation of data that don’t quantify what they’re supposed to show. Data say, for example, that women in science publish less and their papers are cited less often (references here). Does this mean women are less capable of doing science? No, it means that they publish less and their papers are cited less often. To put it differently, it means that women’s papers are less popular with their – predominantly male – colleagues. How about not judging women by how much their work appeals to men?

The recent case of Alessandro Strumia is an example for such shaky interpretations of data. There’s no simple way to measure whether women are having a harder time with their research. There could be all sorts of reasons, from the lack of role models to difficulties getting funding to being more frequently asked to sit in committees because, well, we need at least one woman, you see? None of these difficulties will reflect in publication records in any obvious way.

Besides, there’s no agreed-upon measure for academic success. Indeed, it is well known that rewarding a high number of publications and citations creates perverse incentives, and therefore many scientists now compete to excel on meaningless performance indicators. So why are we even talking about this?

This is not to say that the differences between publications of men and women are not interesting or should not be studied. Just that one shouldn’t jump to conclusions from them.

But I am also not an advocate of a gender-balance of one-to-one. Biological factors, such as muscle strength, arguably play a role for some professions. I find it highly questionable that a profession like physics, which mostly deals with abstract ideas, is much influenced by genetic factors. But it’s a controversial subject and, as they say, more research is needed. Regardless of whether the reason is nature or nurture, however, women demonstrate preferences different from men, and it doesn’t make sense to push them into disciplines that they might not currently feel well in.

For this reason I cannot support policies that aim to increase the ratio of women based on the premise that 50:50 is the “correct” target. It’s not only that this risks we’ll hire women who are less motivated or qualified than some men who don’t get jobs, it also pisses off the men who feel like they are now the ones at a disadvantage. And this creates yet another bias, namely the belief that women now have an easier time rather than merely having less of a disadvantage. Again the recent Strumia-case is a good example.

Having said this, some people tell me it’s justified to risk hiring lesser qualified women, at least temporarily, to restore what they consider fairness in the long run. But at this point we are down to a value-decisions. What is more important to you: That science works most efficiently, or that women catch up with men as quickly as possible? I think this is the key question which no one wants to discuss.

Be that as it may, the easiest way to disqualify yourself from any discussion on the matter is to simply disregard the existing hurdles that women face. The problems are real, and they’re far from vanishing.

---

If you want to make a difference, check out the Practical Guide to Improving Gender Equality in Research Organisations by Science Europe, and raise awareness for the therein proposed changes among your colleagues.

Gender-bias in Academia: The Case Strumia

Dr Alessandro Strumia is a physicist working at CERN, where he is Principal Investigator of an ERC Advanced research grant on the topic “Understanding the mass scales in nature.” Ie, he spends your tax money on the kind of research that I criticize in my book. He also recently published an analysis of publication and citation-rates in his field. At a workshop on gender diversity at CERN last weekend, he used this analysis to argue that women are underrepresented in physics because they are underperforming.

I did not attend the workshop and have not seen a recording of the talk, but I have seen the slides (a PDF version of which is here). The slides contain statements that are both inaccurate and exceedingly unprofessional.

For example, he begins his talk by stating that “smarter people are less affected by implicit bias,” but this is wrong. Studies have shown repeatedly that intelligence does not protect from thinking biases. Yes, intelligence is useful to overcome certain types of biases (mostly those that can be exposed with mathematical reasoning), but only once people are aware they are biased to begin with.

Strumia’s mistaken belief that intelligent people are less affected by cognitive biases does not remotely surprise me. I have encountered this very same attitude (“We are too smart to be biased!”) among almost all high-energy theorists and phenomenologists I have spoken with about the issue. That in itself is a bias, known as the “bias blind spot.”

But that Strumia is ill-informed about the very topic he speaks about at a scientific workshop is not the biggest problem with his presentation. Far worse is that he names and attacks two women, apparently because he is annoyed he did not get a job that he was shortlisted for. Nonsense like this just does not belong in a research presentation.

After complaints ballooned on social media on Monday, CERN pulled the slides from the net quickly and has since suspended Strumia. What will happen to his ERC grant is unclear. A large number of members of the particle physics community have meanwhile signed a statement declaring that they distance themselves from the content of Strumia’s talk.

Now, as you know, I have also recently taken up bibliometric analysis. And I admit I found some of the data Strumia showed interesting. We did, in our paper, also look at gender differences, but not for citation counts. We looked at an entirely different quantity, that of research broadness, and for this we did not find any gender differences.

The gender difference that Alessandro Strumia and his co-author Ricardo Torre find is huge. It’s a more than 100% difference in the total number of citations that researchers accumulate throughout their career.

I don’t think that the number of citations is a good measure for scientific performance, but if the difference between the genders was so large, it might mean that women and men chose their research projects in distinctly different ways. That would be interesting. I thus decided to look into this for a bit.

The key figure that Strumia presents on his slides is the total number of citations that researchers accumulate since the publication of their first paper:

Figure from slide 16 of Alessandro Strumia’s talk.

That the horizontal axis is labeled “scientific age” is unfortunate because this term has been coined to emphasize that the scientific age might differ from the chronological time passed since PhD or first paper. If a researcher takes a career-break, for example because of health reasons or for parental leave, their scientific age goes on hold. However, there is presently no standardized way to determine the scientific age, and in any case, you couldn’t do it from publication data alone, you’d also need biographic information.

Since women are more likely to take leave for child-raising, their citations should on the average increase somewhat slower, simply because they have more breaks in which they don’t publish. However, it seems unlikely that this would make such a huge difference. So, while the label on the axis is inaccurate, I don’t think it’s all that relevant.

When I saw this graph, however, another worry came to my mind immediately. When we did our previous analysis, we found that the vast majority of people who use the arXiv publish only one or two papers and are never heard of again. This is in agreement with the well-known fact that the majority of physicists drop out of academic careers.

I am not sure why this surprised me when it showed up in the data. Maybe because, if you work in the field, the drop-outs are pretty much invisible. They leave and you forget about them. But they are there, in the stats, big and fat.

Now, the total number of citations for such drop-outs will accumulate very slowly because they don’t publish new papers any more. And we know that women are more likely to drop out – that’s the “leaky pipeline” and reason why I find myself increasingly often, if not the only woman in the room, then at least the oldest woman in the room. And I’m only 42.

If you leave the drop-outs in the citation analysis, the leaky pipe will pull down the average of female authors more than of male authors.

I hence asked one of our PhD students, Tobias Mistele, to plot the same quantity as Strumia did for our data sample, but to only keep authors who have more than 5 papers in total, and who have published a paper in the last 3 years. This is sloppy way to shrink down the pool to “active researchers only.” It’s maybe not the most sophisticated way to do it, but it should give us an idea how large the contribution from the drop-outs is.

If we plot the number of citations for active researchers only, we see no noticeable difference between men and women:

arXiv data, active researchers only

When normalized to the number of authors per paper (as Strumia did), there is also no noticeable difference between men and women.

I must add a warning here. We do not use the same data set as Strumia and Torre. They use data from inspire, we use data from the arXiv. This means our data set does not reach back in time as far, and it includes disciplines besides high energy physics. So the absolute numbers are not directly comparable.

Another caveat I must add is that we are using a different method to identify male and female authors. We use the author-id algorithm that is explained in our earlier paper, and then try to match first names with a database for common anglo-saxon first names. Naturally, this means that the authors who remain in our sample are most likely to be of Western origin. By this method we assign a gender to 19% of authors. This is in contrast to Strumia and Torre who use a more elaborate gender-id procedure that allows them to match 60%. The remaining authors in our sample break down to 70,295 male und 53,165 female researchers. After applying the above mentioned cuts, we are left with 12,654 male and 8,177 female. That’s not a huge number, but decent.

Let me also mention that probably a similar effect is behind another finding in Strumia’s talk. He points out that women, on the average, are hired into faculty positions earlier. A paper that appeared on the arXiv yesterday argued that this is not a signal that women have an unfair advantage, but simply a consequence of women leaving at a higher rate. If they aren’t hired early, they’ll not be hired at all, which means the average age of hiring is smaller.

Finally, to state the obvious, this is a blogpost, not a paper. The above is a quick and dirty way to check whether removing dropouts significantly affects the large difference between men and women, and the answer seems to be yes. However, we will have to do a more careful analysis to arrive at definite conclusions. I haven’t checked my biases.

---

I want to thank Tobias Mistele for doing the graphs so quickly and Alessandro Strumia and Ricardo Torre for helpful communication.