How to tell science from pseudoscience

Is the earth flat? Is 5G is a mind-control experiment by the Russian government? What about the idea that COVID was engineered by the vaccine industry? How can we tell apart science from pseudoscience? This is what we will talk about today.

Now, how to tell science from pseudoscience is a topic with a long history that lots of intelligent people have written lots of intelligent things about. But this is YouTube. So instead of telling you what everybody else has said, I’ll just tell you what I think.

I think the task of science is to explain observations. So if you want to know whether something is science you need (a) observations and (b) you need to know what it means to explain something in scientific terms. What scientists mean by “explanation” is that they have a model, which is a simplified description of the real world, and this model allows them to make statements about observations that agree with measurements and – here is the important bit – the model is simpler than just a collection of all available data. Usually that is because the model captures certain patterns in the data, and any kind of pattern is a simplification. If we have such a model, we say it “explains” the data. Or at least part of it.

One of the best historical examples for this is astronomy. Astronomy has been all about finding patterns in the motions of celestial objects. And once you know the patterns, they will, quite literally, connect the dots. Visually speaking, a scientific model gives you a curve that connects data points.

This is arguably over-simplified, but it is an instructive visualization because it tells you when a model stops being scientific. This happens if the model has so much freedom that it can fit any data, because then the model does not explain anything. You would be better off just collecting the data. This is also known as “overfitting ”. If you have a model that has more free parameters as input than data to explain, you may as well not bother with that model. It’s not scientific.

There is something else one can learn from this simple image, which is that making a model more complicated will generally allow a better fit to the data. So if one asks what is the best explanation of a set of data, one has to ask when does adding another parameter not justify the slightly better fit to the data you’d get from it. For our purposes it does not matter just exactly how to calculate this, so let me say that there are statistical methods to evaluate exactly this. This means, we can quantify how well a model explains data.

Now, all of what I just said was very quantitative and not in all disciplines of science are models quantitative, but the general point holds. If you have a model that requires many assumptions to explain few observations, and if you hold on to that model even though there is a simpler explanation, then that is unscientific. And, needless to say, if you have a model that does not explain any observation, then that is also not scientific.

A typical case of pseudoscience are conspiracy theories. Whether that is the idea that the earth is flat but NASA has been covering up the evidence since the days of Ptolemais at least, or that 5G is a plan by the government to mind-control you using secret radiation, or that COVID was engineered by the vaccine industry for profit. All these ideas have in common that they are contrived.

You have to make a lot of assumptions for these ideas to agree with reality, assumptions like somehow it’s been possible to consistently fake all the data and images of a round earth and brainwash every single airline pilot, or it is possible to control other’s people’s mind and yet somehow that hasn’t prevented you from figuring out that minds are being controlled. These contrived assumptions are the equivalent of overfitting. That’s what makes these conspiracy theories unscientific. The scientific explanations are the simple ones, the ones that explain lots of observations with few assumptions. The earth is round. 5G is a wireless network. Bats carry many coronaviruses, these have jumped over to humans before, and that’s most likely where COVID also came from.

Let us look at some other popular example, Darwinian evolution. Darwinian evolution is a good scientific theory because it “connects the dots” basically by telling you how certain organisms evolved from each other. I think that in principle it should be possible to quantify this fit to data, but arguably no one has done that. Creationism, on the other hand, simply posits that Earth was created with everything in place. That means Creationism puts in as much information as you get out of it. It therefore does not explain anything. This does not mean it’s wrong. But it means it is unscientific.

Another way to tell pseudoscience from science is that a lot of pseudoscientists like to brag with making predictions. But just because you have a model that makes predictions does not mean it’s scientific. And the opposite is also true, just because a model does not make predictions does not mean it is not scientific.

This is because it does not take much to make a prediction. I can predict, for example, that one of your relatives will fall ill in the coming week. And just coincidentally, this will be correct for some of you. Are you impressed? Probably not. Why? Because to demonstrate that this prediction was scientific, I’d have to show was better than a random guess. For this I’d have to tell you what model I used and what the assumptions were. But of course I didn’t have a model, I just made a guess. And that doesn’t explain anything, so it’s not scientific.

And a model that does not make predictions can still be scientific if it explains a lot of already existing data. Pandemic models are actually a good example for scientific models which do not make predictions. It is basically impossible to make predictions for the spread of infectious diseases because that spread depends on policy decisions which themselves can’t be predicted.

So with pandemic models we really make “projections” or we can look at certain “scenarios” that are if-then cases. If we do not cancel large events, then the spread will likely look like this. If we do cancel them, the spread will more likely look like that. It’s not a prediction because we cannot predict whether large events will be canceled. But that does not make these models unscientific. They are scientific because they accurately describe the spread of epidemics on record. These are simple explanations that fit a lot of data. And that’s why we use them in the first place.

The same is the case for climate models. The simplest explanation for our observation, the one that fits the data with the least amount of assumptions, is that climate change is due to increasing carbondioxide levels and caused by humans. That’s what the science says.

So if you want to know whether a model is scientific, ask how much data it can correctly reproduce and how many assumptions were required for this.

Having said that, it can be difficult to tell science from pseudoscience if an idea has not yet been fully developed and you are constantly told it’s promising, it’s promising, but no one can ever actually show the model fits to data because, they say, they’re not done with the research. We see this in the foundations of physics most prominently with string theory. String theory, if it would work as advertised, could be good science. But string theorists never seem to get to the point where the idea would actually be useful.

In this case, then, the question is really a different one, namely, how much time and money should you throw at a certain research direction to even find out whether it’s science or pseudoscience. And that, ultimately, is a decision that falls to those who fund that research.