Check your Biases

[slide 8 of this presentation]

Physics World recently interviewed the current director of CERN, Fabiola Gianotti. When asked how particle physicists address group-think, Gianotti explains instead why some research avenues require large communities.

You would think that sufficiently much has been written about cognitive biases and logical fallacies that even particle physicists took note, but at least the ones I deal with have no clue. If I ask them what measures they take to avoid cognitive biases when evaluating the promise of a research direction, they will either mention techniques to prevent biased data-analysis (different thing entirely), or they will deny that they even have biases (thereby documenting the very problem whose existence they deny).

Here is a response I got from a particle physicist when I pointed out that Gianotti did not answer the question about group think:

(This person then launched an ad-hominem attack at me and eventually deleted their comment. In the hope that this deletion documents some sliver of self-insight, I decided to remove identifying information.)

Here is another particle physicist commenting on the same topic, demonstrating just how much these scientists overrate their rationality:

It is beyond me why scientists are still not required to have basic training in the sociology of science, cognitive biases, and decision making in groups. Such knowledge is necessary to properly evaluate information. Scientists cannot correctly judge the promise of research directions unless they are aware how their opinions are influenced by the groups they are part of.

It would be easy enough to set up online courses for this. If I had the funding, I would do it. Alas, I don’t. The only thing I can do, therefore, is to ask everyone – and especially those in leadership positions – to please take the problem seriously. Scientists are human. Leaving cognitive biases unchecked results in inefficient allocations of research funding, not to mention that it wastes time.

In all brevity, here are the basics.

What is a social bias, what is a cognitive bias?

A cognitive bias is thinking shortcut that has developed through evolution. It can be beneficial in some situations, but in other situations it can result in incorrect judgement. A cognitive bias is similar to an optical illusion. Look at this example:

Example of optical illusion. A and B have the same color.
Click here if you don’t believe it. [Image source: Wikipedia]

The pixels in the squares A and B have the exact same color. However, to most people, square B looks lighter than A. That’s because there is a shadow over square B, so your brain factors in that the original color should have been lighter.

The conclusion that B is lighter, therefore, makes perfect sense in a naturally occurring situation. When asked to judge the color on your screen, however, you are likely to give a wrong answer if you are not aware of how your brain works.

Likewise, a cognitive bias happens if your brain factors in information that may be relevant in some situations but can lead to wrong results in others. A social bias, more specifically, is a type of cognitive bias that comes from the interaction with other people.

It is important to keep in mind that cognitive biases are not a sign of lacking intelligence. Everyone has cognitive biases and that’s nothing to be ashamed of. But if your job is to objectively evaluate information, you should be aware that the results of your evaluation are skewed by the way your brain functions.

Scientists, therefore, need to take measures to prevent cognitive biases the same way that they take measures to prevent biases in data analysis. The brain is yet another apparatus. Understanding how it operates is necessary to arrive at correct conclusions.

There are dozens of cognitive biases. I here merely list the ones that I think are most important for science:

• Communal Reinforcement
  More commonly known as “group think,” communal reinforcement happens if members of a community constantly reassure each other that what they are doing is the right thing. It is typically accompanied by devaluing or ignoring outside opinions. You will often see it come along with arguments from popularity. Communal reinforcement is the major reason bad methodologies can become accepted practice in research communities.
  
  
• Availability Cascades
  What we hear of repeatedly sounds more interesting, and we talk more about what is more interesting, which makes it sound even more interesting. This does make a lot of sense if you want to find out what important things are happening in your village. It does not make sense, however, if your job is, say, to decide what’s the most promising experiment to make progress in the foundations of physics. Availability cascades are a driving force in scientific fashion trends and can lead to over-inflated research bubbles with little promise.
  
  
• Post-purchase Rationalization
  This is the tendency to tell ourselves and others that we have not made stupid decision in the past, like, say, pouring billions of dollars in to entirely fruitless research avenues. It is a big obstacle to learning from failure. This bias is amplified by our desire to avoid cognitive dissonance, that is any threat to our self-image as a rationally thinking individual. Post-purchase rationalization is why no experiment in the history of science has ever been a bad investment.
  
  
• Irrational Escalation
  Also known as the “sunk cost fallacy” or “throwing good money after bad.” Irrational Escalation is the argument that you cannot give up now because you have invested so much already. This is one of the main reasons why research agendas survive well beyond the point at which they stopped making sense, see supersymmetry, string theory, or searches for dark matter particles that become heavier and more weakly interacting every time they are not found.
  
  
• Motivated Reasoning
  More collectively known as “wishful thinking,” motivated reasoning is the human tendency to give pep talks and then actually believe the rosy picture we painted ourselves. While usually well-intended, motivated reasoning can result in overly optimistic expectations and an insistence to hold onto irrational dreams. Surely particle physicists are just about to discover some new particle, the next round of experiments will find that dark matter candidate, etc.

There are practical measures you can implement to alleviate these biases, both in your institution and in your personal work-life. In the appendix of my book I list a few. But the most important step is that you acknowledge the existence of these biases whenever you evaluate information and at least try to correct your assessment.

The more people have told you that a crappy scientific method is okay, the more likely you are to believe it is okay. Keep that in mind next time a BSM phenomenologist tells you it is totally normal when a scientific discipline makes wrong predictions for 40 years.

The easiest way to see that particle physics has a big problem with cognitive biases is that members of this community deny they even have biases and refuse to do anything about it.

The topic of cognitive biases has been extensively covered elsewhere, and I see no use in repeating what others have said better. Google will give you all the information you need. Some good starting points are:

• The Cognitive Bias Cheat Sheet, Buster Benson 
• 20 cognitive biases that screw up your decisions, Business Insider  
• How to Reduce Bias In Decision-Making, USC Marshall Critical Thinking Initiative