Does Superdeterminism save Quantum Mechanics? Or Does It Kill Free Will and Destroy Science?

[This is a transcript of the video embedded below. Some of the explanations may not make sense without the animations in the video.]


Superdeterminism is a way to make sense of quantum mechanics. But some physicists and philosophers have argued that if one were to allow it, it would destroy science. Seriously. How does superdeterminism work, what is it good for, and why does it allegedly destroy science? That’s what we’ll talk about today.

First things first, what is superdeterminism? Above all, it’s a terrible nomenclature because it suggests something more deterministic than deterministic and how is that supposed to work? Well, that’s just not how it works. Superdeterminism is exactly as deterministic as plain old vanilla determinism. Think Newton’s laws. If you know the initial position and velocity of an arrow, you can calculate where it will land, at least in principle. That’s determinism: Everything that happens follows from what happened earlier. But in quantum mechanics we can only predict probabilities for measurement outcomes, rather than the measurement outcomes themselves. The outcomes are not determined, so quantum mechanics is indeterministic.

Superdeterminism returns us to determinism. According to superdeterminism, the reason we can’t predict the outcome of a quantum measurement is that we are missing information. This missing information is usually referred to as the “hidden variables”. I’ll tell you more about those later. But didn’t this guy what’s his name Bell prove that hidden variables are wrong?

No, he didn’t, though this is a very common misunderstanding, depressingly, even among physicists. Bell proved that a hidden variables theory which is (a) local and (b) fulfills an obscure assumption called “statistical independence” must obey an inequality, now called Bell’s inequality. We know experimentally that this inequality is violated. It follows that any local hidden variable theory which fits to our observations, has to violate statistical independence.

If statistical independence is violated, this means that what a quantum particle does depends on what you measure. And that’s how superdeterminism works: what a quantum particle does depends on what you measure. I’ll give you an example in a moment. But first let me tell you where the name superdeterminism comes from and why physicists get so upset if you mention it.

Bell didn’t like the conclusion which followed from his own mathematics. Like so many before and after him, Bell wanted to prove Einstein wrong. If you remember, Einstein had said that quantum mechanics can’t be complete because it has a spooky action at a distance. That’s why Einstein thought quantum mechanics is just an average description for a hidden variables theory. Bell in contrast wanted physicists to accept this spooky action. So he had to somehow convince them that this weird extra assumption, statistical independence, makes sense. In a 1983 BBC interview he said the following:

“There is a way to escape the inference of superluminal speeds and spooky action at a distance. But it involves absolute determinism in the universe, the complete absence of free will. Suppose the world is super-deterministic, with not just inanimate nature running on behind-the-scenes clockwork, but with our behavior, including our belief that we are free to choose to do one experiment rather than another, absolutely predetermined, including the “decision” by the experimenter to carry out one set of measurements rather than another, the difficulty disappears.”

This is where the word “superdeterminism” comes from. Bell called a violation of statistical independence “superdeterminism” and claimed that it would require giving up free will. He argued that there are only two options: either accept spooky action and keep free will which would mean that Bell was right, or reject spooky action but give up free will which would mean that Einstein was right. Bell won. Einstein lost.

Now you all know that I think free will is logically incoherent nonsense. But even if you don’t share my opinion, Bell’s argument just doesn’t work. Spooky action at a distance doesn’t make any difference for free will because the indeterministic processes in quantum mechanics are not influenced by anything, so they are not influenced by your “free will,” whatever that may be. And in any case, throwing out determinism just because you don’t like its consequences is really bad science.

Nevertheless, the mathematical assumption of “statistical independence” has since widely been called the “free will” assumption, or the “free choice” assumption. And physicists stopped questioning it to the point that today most of them don’t know that Bell’s theorem even requires this additional assumption.

This is not a joke. All the alleged strangeness of quantum mechanics has its origin in nomenclature. It was forced on us by physicists who called a mathematical statement the “free will assumption”, never mind that it’s got nothing to do with free will, and then argued that one must believe in it because one must believe in free will.

If you find this hard to believe, I can’t blame you, but let me read you a quote from a book by Nicolas Gisin, who is Professor for Physics in Geneva and works on quantum information theory.

“This hypothesis of superdeterminism hardly deserves mention and appears here only to illustrate the extent to which many physicists, even among specialists in quantum physics, are driven almost to despair by the true randomness and nonlocality of quantum physics. But for me, the situation is very clear: not only does free will exist, but it is a prerequisite for science, philosophy, and our very ability to think rationally in a meaningful way. Without free will, there could be no rational thought. As a consequence, it is quite simply impossible for science and philosophy to deny free will.”

Keep in mind that superdeterminism just means statistical independence is violated which has nothing to do with free will. However, even leaving that aside, fact is, the majority of philosophers either believe that free will is compatible with determinism, about 60% of them, or they agree with me that free will doesn’t exist anyway, about 10% of them.

But in case you’re still not convinced that physicists actually bought Bell’s free will argument, here is another quote from a book by Anton Zeilinger, one of the probably most famous physicists alive. Zeilinger doesn’t use the word superdeterminism in his book, but it is clear from the context that he is justifying the assumption of statistical independence. He writes:

“[W]e always implicitly assume the freedom of the experimentalist. This is the assumption of free will… This fundamental assumption is essential to doing science.”

So he too bought Bell’s claim that you have to pick between spooky action and free will. At this point you must be wondering just what this scary mathematical expression is that supposedly eradicates free will. I am about to reveal it, brace yourself. Here we go.

I assume you are shivering in fear of being robbed of your free will if one ever were to allow this. And not only would it rob you of free will, it would destroy science. Indeed, already in 1976, Shimony, Horne, and Clauser argued that doubting statistical independence must be verboten. They wrote: “skepticism of this sort will essentially dismiss all results of scientific experimentation”. And here is one final quote about superdeterminism from the philosopher Tim Maudlin: “besides being insane, [it] would undercut scientific method.”

As you can see, we have no shortage of men who have strong opinions about things they know very little about, but not like this is news. So now let me tell you how superdeterminism actually works, using the double slit experiment as an example.

In the double slit experiment, you send a coherent beam of light at a plate with two thin openings, that’s the double slit. On the screen behind the slit you then see an interference pattern. The interference isn’t in and by itself a quantum effect, you can do this with any type of wave, water waves or sound waves for example.

The quantum effects only become apparent when you let a single quantum of light go through the slits at a time. Each of those particles makes a dot on the screen. But the dots build up… to an interference pattern. What this tells you is that even single particles act like waves. This is why we describe them with wave-functions usually denoted psi. From the wave-function we can calculate the probability of measuring the particle in a particular place, but we can’t calculate the actual place.

Here’s the weird bit. If you measure which slit the particles go through, the interference pattern vanishes. Why? Well, remember that the wave-function – even that of a single particle – describes probabilities for measurement outcomes. In this case the wave-function would first tell you the particle goes through the left and right slit with 50% probability each. But once you measure the particle you know 100% where it is.

So when you measure at which slit the particle is you have to “update” the wave-function. And after that, there is nothing coming from the other slit to interfere with. You’ve destroyed the interference pattern by finding out what the wave did. This update of the wave-function is sometimes also called the collapse or the reduction of the wave-function. Different words, same thing.

The collapse of the wave-function doesn’t make sense as a physical process because it happens instantaneously, and that violates the speed of light limit. Somehow the part of the wave-function at the one slit needs to know that a measurement happened at the other slit. That’s Einstein’s “spooky action at a distance.”

Physicists commonly deal with this spooky action by denying that wave-function collapse is a physical process. Instead, they argue it’s just an update of information. But information about… what? In quantum mechanics there isn’t any further information beyond the wave-function. Interpreting the collapse as an information update really only makes sense in a hidden variables theory. In that case, a measurement tells you more about the possible values of the hidden variables.

Think about the hidden variables as labels for the possible paths that the particle could take. Say the labels 1 2 3 go to the left slit and the labels 4 5 6 go to the right slit and the labels 7 to 12 go through both. The particle really has only one of those hidden variables, but we don’t know which. Then, if we measure the particle at the left slit, that simply tells us that the hidden variable was in the 1 2 3 batch, if we measure it right, it was in the 4 5 6 batch, if we measure it on the screen, it was in the 7 - 12 batch. No mystery, no instantaneous collapse, no non-locality. But it means that the particle’s path depends on what measurement will take place. Because the particles must have known already when they got on the way whether to pick one of the two slits, or go through both. This is just what observations tell us.

And that’s what superdeterminism is. It takes our observations seriously. What the quantum particle does depends on what measurement will take place. Now you may say uhm drawing lines on YouTube isn’t proper science and I would agree. If you’d rather see equations, you’re most welcome to look at my papers instead.

Let us then connect this with what Bell and Zeilinger were talking about. Here is again the condition that statistical independence is violated. The lambda here stands for the hidden variables, and rho is the probability distribution of the hidden variables. This distribution tells you how likely it is that the quantum particle will do any one particular thing. In Bell’s theorem, a and b are the measurement settings of two different detectors at the time of measurement. And this bar here means you’re looking at a conditional probability, so that’s the probability for lambda given a particular combination of settings. When statistical independence is violated, this means that the probability for a quantum particle to do a particular thing depends on the detector settings at the time of measurement.

Since this is a point that people often get confused about, let me stress that it doesn’t matter what the setting is at any earlier or later time. This never appears in Bell’s theorem. You only need to know what’s the measurement that actually happens. It also doesn’t matter how one chooses the detector settings, that never makes any appearance either. And contrary to what Bell and Zeilinger argued, this relation does not restrict the freedom of the experimentalist. Why would it? The experimentalist can measure whatever they like, it’s just that what the particle does depend on what they measure.

And of course this won’t affect the scientific method. What these people were worrying about is that random control trials would be impossible if choosing a control group could depend on what you later measure.

Suppose you randomly assign people into two groups to test whether a vaccine is efficient. People in one group get the vaccine, people in the other group a placebo. The group assignment is the “hidden variable.” If someone falls ill, you do a series of tests to find out what they have, so that’s the measurement. If you think that what happens to people depends on what measurement you will do on them, then you can’t draw conclusions about the efficiency of the vaccine. Alrighty. But you know what, people aren’t quantum particles. And believing that superdeterminism plays a role for vaccine trials is like believing Schrödinger’s cat is really dead and alive.

The correlation between the detector settings and the behavior of a quantum particle which is the hallmark of superdeterminism only occurs when quantum mechanics would predict a non-local collapse of the wave-function. Remember that’s what we need superdeterminism for: that there is no spooky action at a distance. But once you have measured the quantum state, that’s the end of those violations of statistical independence.

I should probably add that a “measurement” in quantum mechanics doesn’t actually require a measurement device. What we call a measurement in quantum mechanics is really any sufficiently strong or frequent interaction with an environment. That’s why we don’t see dead and alive cats. Because there’s always some environment, like air, or the cosmic microwave background. And that’s also why we don’t see superdeterministic correlations in people.

Okay, so I hope I’ve convinced you that superdeterminism doesn’t limit anyone’s free will and doesn’t kill science, now let’s see what it’s good for.

Once you understand what’s going on with the double slit, all the other quantum effects that are allegedly mysterious or strange also make sense. Take for example a delayed choice experiment. In such an experiment, it’s only after the particle started its path that you decide whether to measure which slit it went through. And that gives the same result as the usual double slit experiment.

Well, that’s entirely unsurprising. If you considered measuring something but eventually didn’t, that’s just irrelevant. The only relevant thing is what you actually measure. The path of the particle has to be consistent with that. Or take the bomb experiment that I talked about earlier. Totally unsurprising, the photon’s path just depends on what you measure. Or the quantum eraser. Of course the path of the particle depends on what you measure. That’s exactly what superdeterminism tells you!

So, in my eyes, all those experiments have been screaming us into the face for half a century that what a quantum particle does depends on the measurement setting, and that’s superdeterminism. The good thing about superdeterminism is that since it’s local it can easily be combined with general relativity, so it can help us find a theory of quantum gravity.

Let me finally talk about something less abstract, namely how one can test it. You can’t test superdeterminism by measuring violations of Bell’s inequality because it doesn’t fulfil the assumptions of Bell’s theorem, so doesn’t have to obey the inequality. But superdeterminism generically predicts that measurement outcomes in quantum mechanics are actually determined, and not random.

Now, any theory that solves the measurement problem has to be non-linear, so the reason we haven’t noticed superdeterminism is almost certainly that all our measurements so far have been well in the chaotic regime. In that case trying to make a prediction for a measurement outcome is like trying to make a weather forecast for next year. The best you can do is calculate average values. That’s what quantum mechanics gives us.

But if you want to find out whether measurement outcomes are actually determined, you have to get out of the chaotic regime. This means looking at small systems at low temperatures and measurements in a short sequence, ideally on the same particle. Those measurements are currently just not being done. However, there is a huge amount of progress in quantum technologies at the moment, especially in combination with AI which is really good for finding new patterns. And this makes me think that at some point it’ll just become obvious that measurement outcomes are actually much more predictable than quantum mechanics says. Indeed, maybe someone already has the data, they just haven’t analyzed it the right way.

I know it’s somewhat boring coming from a German but I think Einstein was right about quantum mechanics. Call me crazy if you want but to me it’s obvious that superdeterminism is the correct explanation for our observations. I just hope I’ll live long enough to see that all those men who said otherwise will be really embarrassed.