Pages

Showing posts sorted by relevance for query superdeterminism. Sort by date Show all posts
Showing posts sorted by relevance for query superdeterminism. Sort by date Show all posts

Saturday, December 18, 2021

Does Superdeterminism save Quantum Mechanics? Or Does It Kill Free Will and Destroy Science?

[This is a transcript of the video embedded below. Some of the explanations may not make sense without the animations in the video.]


Superdeterminism is a way to make sense of quantum mechanics. But some physicists and philosophers have argued that if one were to allow it, it would destroy science. Seriously. How does superdeterminism work, what is it good for, and why does it allegedly destroy science? That’s what we’ll talk about today.

First things first, what is superdeterminism? Above all, it’s a terrible nomenclature because it suggests something more deterministic than deterministic and how is that supposed to work? Well, that’s just not how it works. Superdeterminism is exactly as deterministic as plain old vanilla determinism. Think Newton’s laws. If you know the initial position and velocity of an arrow, you can calculate where it will land, at least in principle. That’s determinism: Everything that happens follows from what happened earlier. But in quantum mechanics we can only predict probabilities for measurement outcomes, rather than the measurement outcomes themselves. The outcomes are not determined, so quantum mechanics is indeterministic.

Superdeterminism returns us to determinism. According to superdeterminism, the reason we can’t predict the outcome of a quantum measurement is that we are missing information. This missing information is usually referred to as the “hidden variables”. I’ll tell you more about those later. But didn’t this guy what’s his name Bell prove that hidden variables are wrong?

No, he didn’t, though this is a very common misunderstanding, depressingly, even among physicists. Bell proved that a hidden variables theory which is (a) local and (b) fulfills an obscure assumption called “statistical independence” must obey an inequality, now called Bell’s inequality. We know experimentally that this inequality is violated. It follows that any local hidden variable theory which fits to our observations, has to violate statistical independence.

If statistical independence is violated, this means that what a quantum particle does depends on what you measure. And that’s how superdeterminism works: what a quantum particle does depends on what you measure. I’ll give you an example in a moment. But first let me tell you where the name superdeterminism comes from and why physicists get so upset if you mention it.

Bell didn’t like the conclusion which followed from his own mathematics. Like so many before and after him, Bell wanted to prove Einstein wrong. If you remember, Einstein had said that quantum mechanics can’t be complete because it has a spooky action at a distance. That’s why Einstein thought quantum mechanics is just an average description for a hidden variables theory. Bell in contrast wanted physicists to accept this spooky action. So he had to somehow convince them that this weird extra assumption, statistical independence, makes sense. In a 1983 BBC interview he said the following:
“There is a way to escape the inference of superluminal speeds and spooky action at a distance. But it involves absolute determinism in the universe, the complete absence of free will. Suppose the world is super-deterministic, with not just inanimate nature running on behind-the-scenes clockwork, but with our behavior, including our belief that we are free to choose to do one experiment rather than another, absolutely predetermined, including the “decision” by the experimenter to carry out one set of measurements rather than another, the difficulty disappears.”
This is where the word “superdeterminism” comes from. Bell called a violation of statistical independence “superdeterminism” and claimed that it would require giving up free will. He argued that there are only two options: either accept spooky action and keep free will which would mean that Bell was right, or reject spooky action but give up free will which would mean that Einstein was right. Bell won. Einstein lost.

Now you all know that I think free will is logically incoherent nonsense. But even if you don’t share my opinion, Bell’s argument just doesn’t work. Spooky action at a distance doesn’t make any difference for free will because the indeterministic processes in quantum mechanics are not influenced by anything, so they are not influenced by your “free will,” whatever that may be. And in any case, throwing out determinism just because you don’t like its consequences is really bad science.

Nevertheless, the mathematical assumption of “statistical independence” has since widely been called the “free will” assumption, or the “free choice” assumption. And physicists stopped questioning it to the point that today most of them don’t know that Bell’s theorem even requires this additional assumption.

This is not a joke. All the alleged strangeness of quantum mechanics has its origin in nomenclature. It was forced on us by physicists who called a mathematical statement the “free will assumption”, never mind that it’s got nothing to do with free will, and then argued that one must believe in it because one must believe in free will.

If you find this hard to believe, I can’t blame you, but let me read you a quote from a book by Nicolas Gisin, who is Professor for Physics in Geneva and works on quantum information theory.
“This hypothesis of superdeterminism hardly deserves mention and appears here only to illustrate the extent to which many physicists, even among specialists in quantum physics, are driven almost to despair by the true randomness and nonlocality of quantum physics. But for me, the situation is very clear: not only does free will exist, but it is a prerequisite for science, philosophy, and our very ability to think rationally in a meaningful way. Without free will, there could be no rational thought. As a consequence, it is quite simply impossible for science and philosophy to deny free will.”
Keep in mind that superdeterminism just means statistical independence is violated which has nothing to do with free will. However, even leaving that aside, fact is, the majority of philosophers either believe that free will is compatible with determinism, about 60% of them, or they agree with me that free will doesn’t exist anyway, about 10% of them.

But in case you’re still not convinced that physicists actually bought Bell’s free will argument, here is another quote from a book by Anton Zeilinger, one of the probably most famous physicists alive. Zeilinger doesn’t use the word superdeterminism in his book, but it is clear from the context that he is justifying the assumption of statistical independence. He writes:
“[W]e always implicitly assume the freedom of the experimentalist. This is the assumption of free will… This fundamental assumption is essential to doing science.”
So he too bought Bell’s claim that you have to pick between spooky action and free will. At this point you must be wondering just what this scary mathematical expression is that supposedly eradicates free will. I am about to reveal it, brace yourself. Here we go.

I assume you are shivering in fear of being robbed of your free will if one ever were to allow this. And not only would it rob you of free will, it would destroy science. Indeed, already in 1976, Shimony, Horne, and Clauser argued that doubting statistical independence must be verboten. They wrote: “skepticism of this sort will essentially dismiss all results of scientific experimentation”. And here is one final quote about superdeterminism from the philosopher Tim Maudlin: “besides being insane, [it] would undercut scientific method.”

As you can see, we have no shortage of men who have strong opinions about things they know very little about, but not like this is news. So now let me tell you how superdeterminism actually works, using the double slit experiment as an example.

In the double slit experiment, you send a coherent beam of light at a plate with two thin openings, that’s the double slit. On the screen behind the slit you then see an interference pattern. The interference isn’t in and by itself a quantum effect, you can do this with any type of wave, water waves or sound waves for example.

The quantum effects only become apparent when you let a single quantum of light go through the slits at a time. Each of those particles makes a dot on the screen. But the dots build up… to an interference pattern. What this tells you is that even single particles act like waves. This is why we describe them with wave-functions usually denoted psi. From the wave-function we can calculate the probability of measuring the particle in a particular place, but we can’t calculate the actual place.

Here’s the weird bit. If you measure which slit the particles go through, the interference pattern vanishes. Why? Well, remember that the wave-function – even that of a single particle – describes probabilities for measurement outcomes. In this case the wave-function would first tell you the particle goes through the left and right slit with 50% probability each. But once you measure the particle you know 100% where it is.

So when you measure at which slit the particle is you have to “update” the wave-function. And after that, there is nothing coming from the other slit to interfere with. You’ve destroyed the interference pattern by finding out what the wave did. This update of the wave-function is sometimes also called the collapse or the reduction of the wave-function. Different words, same thing.

The collapse of the wave-function doesn’t make sense as a physical process because it happens instantaneously, and that violates the speed of light limit. Somehow the part of the wave-function at the one slit needs to know that a measurement happened at the other slit. That’s Einstein’s “spooky action at a distance.”

Physicists commonly deal with this spooky action by denying that wave-function collapse is a physical process. Instead, they argue it’s just an update of information. But information about… what? In quantum mechanics there isn’t any further information beyond the wave-function. Interpreting the collapse as an information update really only makes sense in a hidden variables theory. In that case, a measurement tells you more about the possible values of the hidden variables.

Think about the hidden variables as labels for the possible paths that the particle could take. Say the labels 1 2 3 go to the left slit and the labels 4 5 6 go to the right slit and the labels 7 to 12 go through both. The particle really has only one of those hidden variables, but we don’t know which. Then, if we measure the particle at the left slit, that simply tells us that the hidden variable was in the 1 2 3 batch, if we measure it right, it was in the 4 5 6 batch, if we measure it on the screen, it was in the 7 - 12 batch. No mystery, no instantaneous collapse, no non-locality. But it means that the particle’s path depends on what measurement will take place. Because the particles must have known already when they got on the way whether to pick one of the two slits, or go through both. This is just what observations tell us.

And that’s what superdeterminism is. It takes our observations seriously. What the quantum particle does depends on what measurement will take place. Now you may say uhm drawing lines on YouTube isn’t proper science and I would agree. If you’d rather see equations, you’re most welcome to look at my papers instead.

Let us then connect this with what Bell and Zeilinger were talking about. Here is again the condition that statistical independence is violated. The lambda here stands for the hidden variables, and rho is the probability distribution of the hidden variables. This distribution tells you how likely it is that the quantum particle will do any one particular thing. In Bell’s theorem, a and b are the measurement settings of two different detectors at the time of measurement. And this bar here means you’re looking at a conditional probability, so that’s the probability for lambda given a particular combination of settings. When statistical independence is violated, this means that the probability for a quantum particle to do a particular thing depends on the detector settings at the time of measurement.

Since this is a point that people often get confused about, let me stress that it doesn’t matter what the setting is at any earlier or later time. This never appears in Bell’s theorem. You only need to know what’s the measurement that actually happens. It also doesn’t matter how one chooses the detector settings, that never makes any appearance either. And contrary to what Bell and Zeilinger argued, this relation does not restrict the freedom of the experimentalist. Why would it? The experimentalist can measure whatever they like, it’s just that what the particle does depend on what they measure.

And of course this won’t affect the scientific method. What these people were worrying about is that random control trials would be impossible if choosing a control group could depend on what you later measure.

Suppose you randomly assign people into two groups to test whether a vaccine is efficient. People in one group get the vaccine, people in the other group a placebo. The group assignment is the “hidden variable.” If someone falls ill, you do a series of tests to find out what they have, so that’s the measurement. If you think that what happens to people depends on what measurement you will do on them, then you can’t draw conclusions about the efficiency of the vaccine. Alrighty. But you know what, people aren’t quantum particles. And believing that superdeterminism plays a role for vaccine trials is like believing Schrödinger’s cat is really dead and alive.

The correlation between the detector settings and the behavior of a quantum particle which is the hallmark of superdeterminism only occurs when quantum mechanics would predict a non-local collapse of the wave-function. Remember that’s what we need superdeterminism for: that there is no spooky action at a distance. But once you have measured the quantum state, that’s the end of those violations of statistical independence.

I should probably add that a “measurement” in quantum mechanics doesn’t actually require a measurement device. What we call a measurement in quantum mechanics is really any sufficiently strong or frequent interaction with an environment. That’s why we don’t see dead and alive cats. Because there’s always some environment, like air, or the cosmic microwave background. And that’s also why we don’t see superdeterministic correlations in people.

Okay, so I hope I’ve convinced you that superdeterminism doesn’t limit anyone’s free will and doesn’t kill science, now let’s see what it’s good for.

Once you understand what’s going on with the double slit, all the other quantum effects that are allegedly mysterious or strange also make sense. Take for example a delayed choice experiment. In such an experiment, it’s only after the particle started its path that you decide whether to measure which slit it went through. And that gives the same result as the usual double slit experiment.

Well, that’s entirely unsurprising. If you considered measuring something but eventually didn’t, that’s just irrelevant. The only relevant thing is what you actually measure. The path of the particle has to be consistent with that. Or take the bomb experiment that I talked about earlier. Totally unsurprising, the photon’s path just depends on what you measure. Or the quantum eraser. Of course the path of the particle depends on what you measure. That’s exactly what superdeterminism tells you!

So, in my eyes, all those experiments have been screaming us into the face for half a century that what a quantum particle does depends on the measurement setting, and that’s superdeterminism. The good thing about superdeterminism is that since it’s local it can easily be combined with general relativity, so it can help us find a theory of quantum gravity.

Let me finally talk about something less abstract, namely how one can test it. You can’t test superdeterminism by measuring violations of Bell’s inequality because it doesn’t fulfil the assumptions of Bell’s theorem, so doesn’t have to obey the inequality. But superdeterminism generically predicts that measurement outcomes in quantum mechanics are actually determined, and not random.

Now, any theory that solves the measurement problem has to be non-linear, so the reason we haven’t noticed superdeterminism is almost certainly that all our measurements so far have been well in the chaotic regime. In that case trying to make a prediction for a measurement outcome is like trying to make a weather forecast for next year. The best you can do is calculate average values. That’s what quantum mechanics gives us.

But if you want to find out whether measurement outcomes are actually determined, you have to get out of the chaotic regime. This means looking at small systems at low temperatures and measurements in a short sequence, ideally on the same particle. Those measurements are currently just not being done. However, there is a huge amount of progress in quantum technologies at the moment, especially in combination with AI which is really good for finding new patterns. And this makes me think that at some point it’ll just become obvious that measurement outcomes are actually much more predictable than quantum mechanics says. Indeed, maybe someone already has the data, they just haven’t analyzed it the right way.

I know it’s somewhat boring coming from a German but I think Einstein was right about quantum mechanics. Call me crazy if you want but to me it’s obvious that superdeterminism is the correct explanation for our observations. I just hope I’ll live long enough to see that all those men who said otherwise will be really embarrassed.

Monday, December 16, 2019

The path we didn’t take


“There are only three people in the world who understand Superdeterminism,” I used to joke, “Me, Gerard ‘t Hooft, and a guy whose name I can’t remember.” In all honesty, I added the third person just in case someone would be offended I hadn’t heard of them.

What the heck is Superdeterminism?, you ask. Superdeterminism is what it takes to solve the measurement problem of quantum mechanics. And not only this. I have become increasingly convinced that our failure to solve the measurement problem is what prevents us from making progress in the foundations of physics overall. Without understanding quantum mechanics, we will not understand quantum field theory, and we will not manage to quantize gravity. And without progress in the foundations of physics, we are left to squeeze incrementally better applications out of the already known theories.

The more I’ve been thinking about this, the more it seems to me that quantum measurement is the mother of all problems. And the more I am talking about what I have been thinking about, the crazier I sound. I’m not even surprised no one wants to hear what I think is the obvious solution: Superdeterminism! No one besides ‘t Hooft, that is. And that no one listens to ‘t Hooft, despite him being a Nobel laureate, doesn’t exactly make me feel optimistic about my prospects of getting someone to listen to me.

The big problem with Superdeterminism is that the few people who know what it is, seem to have never thought about it much, and now they are stuck on the myth that it’s an unscientific “conspiracy theory”. Superdeterminism, so their story goes, is the last resort of the dinosaurs who still believe in hidden variables. According to these arguments, Superdeterminism requires encoding the outcome of every quantum measurement in the initial data of the universe, which is clearly outrageous. Not only that, it deprives humans of free will, which is entirely unacceptable.

If you have followed this blog for some while, you have seen me fending off this crowd that someone once aptly described to me as “Bell’s Apostles”. Bell himself, you see, already disliked Superdeterminism. And the Master cannot err, so it must be me who is erring. Me and ‘t Hooft. And that third person whose name I keep forgetting.

Last time I made my 3-people-joke was in February during a Facebook discussion about the foundations of quantum mechanics. On this occasion, someone offered in response the name “Tim Palmer?” Alas, the only Tim Palmer I’d heard of is a British music producer from whose videos I learned a few things about audio mixing. Seemed like an unlikely match.

But the initial conditions of the universe had a surprise in store for me.

The day of that Facebook comment I was in London for a dinner discussion on Artificial Intelligence. How I came to be invited to this event is a mystery to me. When the email came, I googled the sender, who turned out to be not only the President of the Royal Society of London but also a Nobel Prize winner. Thinking this must be a mistake, I didn’t reply. A few weeks later, I tried to politely decline, pointing out, I paraphrase, that my knowledge about Artificial Intelligence is pretty much exhausted by it being commonly abbreviated AI. In return, however, I was assured no special expertise was expected of me. And so I thought, well, free trip to London, dinner included. Would you have said no?

When I closed my laptop that evening and got on the way to the AI meeting, I was still wondering about the superdeterministic Palmer. Maybe there was a third person after all? The question was still circling in my head when the guy seated next to me introduced himself as... Tim Palmer.

Imagine my befuddlement.

This Tim Palmer, however, talked a lot about clouds, so I filed him under “weather and climate.” Then I updated my priors for British men to be called Tim Palmer. Clearly a more common name than I had anticipated.

But the dinner finished and our group broke up and, as we walked out, the weather-Palmer began talking about free will! You’d think it would have dawned on me then I’d stumbled over the third Superdeterminist. However, I was merely thinking I’d had too much wine. Also, I was now somewhere in London in the middle of the night, alone with a man who wanted to talk to me about free will. I excused myself and left him standing in the street.

But Tim Palmer turned out to not only be a climate physicist with an interest in the foundations of quantum mechanics, he also turned out to be remarkably persistent. He wasn’t remotely deterred by my evident lack of interest. Indeed, I later noticed he had sent me an email already two years earlier. Just that I dumped it unceremoniously in my crackpot folder. Worse, I seem to vaguely recall telling my husband that even the climate people now have ideas for how to revolutionize quantum mechanics, hahaha.

Cough.

Tim, in return, couldn’t possibly have known I was working on Superdeterminism. In February, I had just been awarded a small grant from the Fetzer Franklin Fund to hire a postdoc to work on the topic, but the details weren’t public information.

Indeed, Tim and I didn’t figure out we have a common interest until I interviewed him on a paper he had written about something entirely different, namely how to quantify the uncertainty of climate models.

I’d rather not quote cranks, so I usually spend some time digging up information about people before interviewing them. That’s when I finally realized Tim’s been writing about Superdeterminism when I was still in high school, long before even ‘t Hooft got into the game. Even more interestingly, he wrote his PhD thesis in the 1970s about general relativity before gracefully deciding that working with Stephen Hawking would not be a good investment of his time (a story you can hear here at 1:12:15). Even I was awed by that amount of foresight.

Tim and I then spent some months accusing each other of not really understanding how Superdeterminism works. In the end, we found we agree on more points than not and wrote a paper to explain what Superdeterminism is and why the objections often raised against it are ill-founded. Today, this paper is on the arXiv.


Thanks to support from the Fetzer Franklin Fund, we are also in the midst of organizing a workshop on Superdeterminism and Retrocausality. So this isn’t the end of the story, it’s the beginning.

Sunday, July 28, 2019

The Forgotten Solution: Superdeterminism

Welcome to the renaissance of quantum mechanics. It took more than a hundred years, but physicists finally woke up, looked quantum mechanics into the face – and realized with bewilderment they barely know the theory they’ve been married to for so long. Gone are the days of “shut up and calculate”; the foundations of quantum mechanics are en vogue again.

It is not a spontaneous acknowledgement of philosophy that sparked physicists’ rediscovered desire; their sudden search for meaning is driven by technological advances.

With quantum cryptography a reality and quantum computing on the horizon, questions once believed ephemeral are now butter and bread of the research worker. When I was a student, my prof thought it questionable that violations of Bell’s inequality would ever be demonstrated convincingly. Today you can take that as given. We have also seen delayed-choice experiments, marveled over quantum teleportation, witnessed decoherence in action, tracked individual quantum jumps, and cheered when Zeilinger entangled photons over hundreds of kilometers of distance. Well, some of us, anyway.

But while physicists know how to use the mathematics of quantum mechanics to make stunningly accurate predictions, just what this math is about has remained unclear. This is why physicists currently have several “interpretations” of quantum mechanics.

I find the term “interpretations” somewhat unfortunate. That’s because some ideas that go as “interpretation” are really theories which differ from quantum mechanics, and these differences may one day become observable. Collapse models, for example, explicitly add a process for wave-function collapse to quantum measurement. Pilot wave theories, likewise, can result in deviations from quantum mechanics in certain circumstances, though those have not been observed. At least not yet.

A phenomenologist myself, I am agnostic about different interpretations of what is indeed the same math, such as QBism vs Copenhagen or the Many Worlds. But I agree with the philosopher Tim Maudlin that the measurement problem in quantum mechanics is a real problem – a problem of inconsistency – and requires a solution.

And how to solve it? Collapse models solve the measurement problem, but they are hard to combine with quantum field theory which for me is a deal-breaker. Pilot wave theories also solve it, but they are non-local, which makes my hair stand up for much the same reason. This is why I think all these approaches are on the wrong track and instead side with superdeterminism.

But before I tell you what’s super about superdeterminism, I have to briefly explain the all-important theorem from John Stewart Bell. It says, in a nutshell, that correlations between certain observables are bounded in every theory which fulfills certain assumptions. These assumptions are what you would expect of a deterministic, non-quantum theory – statistical locality and statistical independence (together often referred to as “Bell locality”) – and should, most importantly, be fulfilled by any classical theory that attempts to explain quantum behavior by adding “hidden variables” to particles.

Experiments show that the bound of Bell’s theorem can be violated. This means the correct theory must violate at least one of the theorem’s assumptions. Quantum mechanics is indeterministic and violates statistical locality. (Which, I should warn you has little to do with what particle physicists usually mean by “locality.”) A deterministic theory that doesn’t fulfill the other assumption, that of statistical independence, is called superdeterministic. Note that this leaves open whether or not a superdeterministic theory is statistically local.

Unfortunately, superdeterminism has a bad reputation, so bad that most students never get to hear of it. If mentioned at all, it is commonly dismissed as a “conspiracy theory.” Several philosophers have declared superdeterminism means abandoning scientific methodology entirely. To see where this objection comes from – and why it’s wrong – we have to unwrap this idea of statistical independence.

Statistical independence enters Bell’s theorem in two ways. One is that the detectors’ settings are independent of each other, the other one that the settings are independent of the state you want to measure. If you don’t have statistical independence, you are sacrificing the experimentalist’s freedom to choose what to measure. And if you do that, you can come up with deterministic hidden variable explanations that result in the same measurement outcomes as quantum mechanics.

I find superdeterminism interesting because the most obvious class of hidden variables are the degrees of freedom of the detector. And the detector isn’t statistically independent of itself, so any such theory necessarily violates statistical independence. It is also, in a trivial sense, non-linear just because if the detector depends on a superposition of prepared states that’s not the same as superposing two measurements. Since any solution of the measurement problem requires a non-linear time evolution, that seems a good opportunity to make progress.

Now, a lot of people discard superdeterminism simply because they prefer to believe in free will, which is where I think the biggest resistance to superdeterminism comes from. Bad enough that belief isn’t a scientific reason, but worse that this is misunderstanding just what is going on. It’s not like superdeterminism somehow prevents an experimentalist from turning a knob. Rather, it’s that the detectors’ states aren’t independent of the system one tries to measure. There just isn’t any state the experimentalist could twiddle their knob to which would prevent a correlation.

Where do these correlations ultimately come from? Well, they come from where everything ultimately comes from, that is from the initial state of the universe. And that’s where most people walk off: They think that you need to precisely choose the initial conditions of the universe to arrange quanta in Anton Zeilinger’s brain just so that he’ll end up turning a knob left rather than right. Besides sounding entirely nuts, it’s also a useless idea, because how the hell would you ever calculate anything with it? And if it’s unfalsifiable but useless, then indeed it isn’t science. So, frowning at superdeterminism is not entirely unjustified.

But that would be jumping to conclusions. How much detail you need to know about the initial state to make predictions depends on your model. And without writing down a model, there is really no way to tell whether it does or doesn’t live up to scientific methodology. It’s here where the trouble begins.

While philosophers on occasion discuss superdeterminism on a conceptual basis, there is little to no work on actual models. Besides me and my postdoc, I count Gerard ‘t Hooft and Tim Palmer. The former gentleman, however, seems to dislike quantum mechanics and would rather have a classical hidden variables theory, and the latter wants to discretize state space. I don’t see the point in either. I’ll be happy if the result solves the measurement problem and is still local the same way that quantum field theories are local, ie as non-local as quantum mechanics always is.*

The stakes are high, for if quantum mechanics is not a fundamental theory, but can be derived from an underlying deterministic theory, this opens the door to new applications. That’s why I remain perplexed that what I think is the obvious route to progress is one most physicists have never even heard of. Maybe it’s just a reality they don’t want to wake up to.


Recommended reading:
  • The significance of measurement independence for Bell inequalities and locality
    Michael J. W. Hall
    arXiv:1511.00729
  • Bell's Theorem: Two Neglected Solutions
    Louis Vervoort
    FoP, 3,769–791 (2013), arXiv:1203.6587

* Rewrote this paragraph to better summarize Palmer’s approach.

Saturday, February 26, 2022

An update on the status of superdeterminism with some personal notes

In December I put out a video on superdeterminism that many of you had asked for. I hesitated with this for a long time. As you have undoubtedly noticed, I don’t normally do videos about my own research. This is because I can’t lay out all the ifs and buts in a 10 minutes video, and that makes it impossible to meet my own scientific standards.

I therefore eventually decided to focus the video on the most common misunderstandings about superdeterminism, which is (a) that superdeterminism has something to do with free will and (b) that it destroys science. I sincerely hope that after my video we can lay these two claims to rest.

However, on the basis of this video, a person by name Bernado Kastrup chose to criticize my research. He afterwards demanded on twitter that we speak together about his criticism. I initially ignored him for several reasons.  

First of all, he got things wrong pretty much as soon as he started writing, showing that he either didn’t read my papers or didn’t understand them. Second, a lot of people pick on me because they want to draw attention to themselves and that’s a game I’m not willing to take part in. 

Third, Kastrup has written a bunch of essays about consciousness and something-with-quantum and “physicalism” which makes him the kind of person I generally want nothing to do with. Just to give you an idea, let me quote from one of his essays:
“Ordinary phenomenal activity in cosmic consciousness can thus be modelled as a connected directed graph. See Figure 1a. Each vertex in the graph represents a particular phenomenal content and each edge a cognitive association logically linking contents together.”
And here is the figure: 



Hence, in contrast to what Kastrup accused me of, the reason I didn’t want to talk to him was not that I hadn’t read what he wrote, but that I had read it.

The fourth and final reason that I didn’t want to talk to him is that I get a lot of podcast request, and I don’t reply to most of them simply because I don’t have the time.

I consulted on this matter with some friends and collaborators, and after that decided that I’d talk to Kastrup anyway. Mostly because I quite like Curt Jaimugal who offered to host the discussion and who I’d spoken with before. He’s a smart young man and if you take away nothing else from this blogpost, then at least go check out his YouTube channel which is well worth some of your time. Also, I thought that weeding out Kastrup’s understandings might help other people, too.

A week later, the only good thing I can report about my conversation with Kastrup is that he didn’t bring up free will, which I think is progress. Unfortunately, he didn’t seem to know much about superdeterminism even after having had time to prepare. He eventually ran out of things to say and then accused me of being “combative” after clearly being surprised to hear that an interaction with a single photon isn’t a measurement. Srsly. Go listen to it.


Instead of concluding that he’s out of his depth, he then wrote another blogpost in which he accused me of “misleading, hollow, but self-confident, assertive rhetoric”, claimed that “Sabine has a big mouth and seems to be willing to almost flat-out lie in order to NOT look bad when confronted on a point she doesn’t have a good counter for.” And, “Her rhetorical assertiveness is, at least sometimes, a facade that hides a surprising lack of actual substance.”

Keep in mind that this is a person who claimed to “model” the “phenomenal activity in cosmic consciousness” with 16 circles. Speak of lack of substance.

Lesson learned: I was clearly too optimistic about the possibility of rational discourse, and don’t think it makes sense to further communicate with this person.

Having said that, I gather that some people who watched the exchange were genuinely interested in the details, so I want to add some explanations that didn’t come across as clearly as I hoped they would.

First of all, the reason I am interested in superdeterminism has nothing whatsoever to do with physicalism or realism (I don’t know what these words mean to begin with). It’s simply that the collapse postulate in quantum mechanics isn’t compatible with general relativity because it isn’t local. That’s a well-defined mathematical problem and solving such problems is what I do for a living.

Note that simply declaring that the collapse isn’t a physical process doesn’t explain what happens and hence doesn’t solve the problem. We need to have some answer for what happens with the expectation value of the stress-energy-tensor during a measurement. I’m an instrumentalist; I am looking for a mathematical prescription that reproduces observations, one of which is that the outcome of a measurement is a detector eigenstate.

The obvious solution to this problem is that the measurement process which we have in quantum mechanics is an effective, statistical description of an underlying local process in a hidden variables theory. We know from Bell’s theorem (or its observed violations, respectively) that if a local theory underlies quantum mechanics then it has to violate statistical independence. That’s what is commonly called “superdeterminism”. In such theories the wave-function is an average description, hence not “real” or “physical” in any meaningful way.

So: Why am I interested in superdeterminism? Because general relativity is local. It is beyond me why pretty much everybody else wants to hold onto an assumption as problematic and unjustified as statistical independence, and is instead willing to throw out locality, but that’s the situation.

Now, the variables in this yet-to-be-found underlying theory are only “hidden” in so far as that they don’t appear in quantum mechanics; they may well be observable with suitable experiments. This brings up the question what a suitable experiment would be.

It is clear that Bell-type tests are not the right experiments, because superdeterministic theories violate Bell inequalities just like quantum mechanics. In fact, superdeterministic theories, since they reproduce quantum mechanics when averaged over the hidden variables, will give the same inequality violations and obey the same bounds as quantum mechanics. (Some people seem to find this hard to understand and try to impress me by quoting other inequalities than Bell’s. You can check for yourself that all those inequalities assume statistical independence, so they cannot be used to test superdeterminism.)

This is why, in 2011, I wrote a paper in which I propose a mostly model-independent test for hidden variables that relies on repeated measurements on non-commuting variables. I later learned from Chris Fuchs (see note at end of paper) that von Neumann made a similar proposal 50 years ago, but the experiment was never done. It still hasn’t been done.

A key point of the 2011 paper was that one does not need to make specific assumptions about the hidden variables. One reason I did this is that Bell’s theorem works the same way: you don’t need to know just what the hidden variables are, you just need to make some assumptions about their properties.

Another reason is, as I have explained in my book “Lost in Math”, that math alone isn’t sufficient to develop a new theory. We need data to develop the underlying hidden variables theory. Without that, we can only guess models and the chance that any one of them is correct is basically zero. 

This is why I did not want to develop a model for the hidden variables – it would be a waste of time. It didn’t work for phenomenology beyond the standard model and it won’t work here either. Instead, we have to identify the experimental range where evidence could be found, collect the data, and then develop the model.

Unfortunately and, in hindsight, unsurprisingly, the 2011 paper didn’t go anywhere. I think it’s just too far off the current mode of thinking in physics, which is all about guessing models and then showing that those guesses are wrong, a methodology that works incredibly badly. Nevertheless, I have since spent some time on developing a hidden variables model, but it’s going slowly, partly because I think it’s a waste of time (see above), but also because I am merely one person working four jobs while raising two kids and my day only has 24 hours.

However, in contrast to what Kastrup accuses me of, I have repeatedly and clearly stated that we do not have a satisfactory superdeterministic hidden variables model at the moment. I say this in pretty much all of my talks, it’s explicitly stated in my paper with Tim (“These approaches [...] leave open many questions and it might well turn out that none of them is the right answer.”). I also said this in my conversation with Kastrup. 

But I want to stress that the reason I (and quite possibly others too) didn’t write down a particular hidden variables model is not that it can’t be done, but that there are too many ways it could be done.

Next thing he got confused about is that two years ago, Sandro an I cooked up a superdeterministic toy model. The point of this model was not to say that it should be experimentally tested. We merely put this forward to demonstrate once and for all that superdeterministic models do not require “finetuning” or any “conspiracies”.

The toy model reproduces quantum mechanics exactly, but – in contrast to quantum mechanics – it’s local and deterministic, on the expense of violating statistical independence. Since it reproduces quantum mechanics it’s as falsifiable as quantum mechanics, hence the claim that superdeterminism somehow ruins science is arguably wrong. 

But besides this, it is a rather pointless and ad hoc toy model that I don’t think makes a lot of sense for a number of reasons (which are stated in the paper). Still, it demonstrates that of course if you want to then can define your hidden variables somehow. I should also mention that our model is certainly not the first superdeterministic hidden variables model. (See references in paper.)

There are a lot of toy models in quantum foundations like this with the purpose of shedding light on one particular assumption or another, and my model falls in this tradition. I could easily modify this model so that it would make predictions that deviate from quantum mechanics, so that one could experimentally test it. But the predictions would be wrong, so why would I do this.

Having said that, my thinking about superdeterminism has somewhat changed since 2011. I was at the time thinking about the hidden variables the way that they are usually portrayed as some kind of extra information that resides within particles. I have since become convinced that this doesn’t work, and that the hidden variables are instead the degrees of freedom of the detector. If that is so, then we do know what the hidden variables are, and we can estimate how likely they are to change. Hence, it becomes easier to test the consequences.

This is why in my later papers and in my more recent talks I mention a simpler type of experiment that works for this case – when the hidden variables are the details of the detector – specifically. I have to stress though that there are other models of superdeterminism which work differently and that can’t be tested this way.

Just what the evolution law looks like I still don’t know. I think it can’t be done with a differential equation, which is why I have been looking at path integrals. I wrote a paper about this with Sandro recently in which we propose new path-integral formalism that can incorporate the required type of evolution law. (It was just accepted for publication the other day.)

What we do in the paper is to define the formalism and show that it can reproduce quantum mechanics exactly – a finding I think is interesting in and by itself. As Kastrup said entirely correctly, there are no hidden variables in that paper. I don’t know why he thought there would be. The paper is just not about hidden variables theories. 

I have a number of ideas of how to include the detector degrees of freedom as hidden variables into the path integral. But again the problem isn’t that it can’t be done, but that there are too many ways it can be done. And in none of the ways I have tried can you still calculate something with the integral. So this didn’t really go anywhere – at least so far. It doesn’t help that I have no funding for this research.

I still think the best way forward would be to just experimentally push into this region of parameter space (small, cold system with quick repeated measurements on simple states) and see if any deviations from quantum mechanics can be found. However, if anyone reading is interested in helping with the path integral, please shoot me a note because I have a lot more to say than what’s in our papers.

Finally, I have given a number of seminars about superdeterminism, at least one of which is on YouTube here. I also some months ago did a discussion with Matt Leifer which is here. Leifer, I would say, is one of the leading people in the foundations of quantum mechanics at the moment. I may have some disagreements with him but he knows his stuff. You will learn more from him than from Kastrup.

In case you jumped over some of the more cumbersome paragraphs above, here is the brief summary. You can either go off the deep end and join people like Kastrup who complain about “physicalism”, claim that photons are observers, detectors can both click and not click at the same time, and other bizarre consequences you have to accept if you insist that quantum mechanics is fundamental. 

Or you conclude, like I have, that quantum mechanics is not a fundamental theory. In this case we just need to find the right experiment get a handle on the underlying physics. 

Saturday, May 21, 2022

The Superdetermined Workshop finally took place

In case you’re still following this blog, I think I owe you an apology for the silence. I keep thinking I’ll get back to posting more than the video scripts but there just isn’t enough time in the day. 

Still, I’m just back from Bonn, where our workshop on Superdeterminism and Retrocausality finally took place. And since I told you how this story started three years ago I thought I’d tell you today how it went.

Superdeterminism and Retrocausality are approaches to physics beyond quantum mechanics, at least that’s how I think about it – and that already brings us to the problem: we don’t have an agreed-upon definition for these terms. Everyone is using them in a slightly different way and it’s causing a lot of confusion. 

So one of the purposes of the workshop was to see if we can bring clarity into the nomenclature. The other reason was to bring in experimentalists, so that the more math-minded among us could get a sense of what tests are technologically feasible.

I did the same thing 15 years ago with the phenomenology of quantum gravity, on which I organized a series of conferences (if you’ve followed this blog for a really long time you’ll remember). This worked out beautifully – the field of quantum gravity phenomenology is in much better condition today than it was 20 years ago.

It isn’t only that I think we’ll quite possibly see experimental confirmation (or falsification!) of quantum gravity in the next decade or two, because I thought that’d be possible all along. Much more important is that the realization that it’s possible to test quantum gravity (without building a Milky-Way sized particle collider) is slowly sinking into the minds of the community, so something is actually happening.

But, characteristically, the moment things started moving I lost interest in the whole quantum gravity thing and moved on to attack the measurement problem in quantum mechanics. I have a lot of weaknesses, but lack of ambition isn’t one of them.

The workshop was originally scheduled to take place in Cambridge in May 2020. We picked Cambridge because my one co-organizer, Huw Price, was located there, the other one, Tim Palmer, is in Oxford, and both places collect a decent number of quantum foundations people. We had the room reserved, had the catering sorted out, and had begun to book hotels. Then COVID happened and we had to cancel everything at the last minute. We tentatively postponed the meeting to late 2020, but that didn’t come into being either.

Huw went to Australia, and by the time the pandemic was tapering out, he’d moved on to Bonn. We moved the workshop with him to Bonn, more specifically to a place called the International Center for Philosophy. Then we started all over again.

We didn’t want to turn this workshop into an online event because that’d have defeated the purpose. There are few people working on superdeterminism and retrocausality and we wanted them to have a chance to get to personally know each other. Luckily our sponsor, the Franklin Fetzer Fund, was extremely supportive even though we had to postpone the workshop twice and put up with some cancellation fees.

Of course the pandemic isn’t quite over and several people still have travel troubles. In particular, it turned out there’s a nest of retrocausalists in Australia and they were more or less stuck there. Traveling from China is also difficult at the moment. And we had a participant affiliated with a Russian university who had difficulties traveling for yet another reason. The world is in many ways a different place now than it was 2 years ago.

One positive thing that’s come out of the pandemic though is that it’s become much easier to set up zoom links and live streams and people are more used to it. So while we didn’t have remote talks, we did have people participating from overseas, from Australia, China, and Canada. It worked reasonably well, leaving aside the usually hiccups, that they partly couldn’t see or hear, the zoom event expired when it shouldn’t have, etc.

I have organized a lot of workshops and conferences and I have attended even more of them. This meeting was special in a way I didn’t anticipate. Many of the people who are working on superdeterminism and retrocausality have for decades been met with a mix of incredulity, ridicule, and insults. In fact, you might have seen this play out with your own eyes in the comment sections of this and other blogs. For many of us, me included, this was the first time we had an audience who took our work seriously.

All of this talk about superdeterminism and new physics beyond quantum mechanics may turn out to be complete rubbish of course. But at least at present I think it’s the most promising route to make progress in the foundations of physics. The reason is quite simple: If it’s right, then new physics should appear in a parameter range that we can experimentally access from two sides, by making measuring devices smaller, and by bringing larger objects into quantum states. And by extrapolating the current technological developments, we'll get there soon enough anyway. The challenge is now to figure out what to look for when the data come in.

The talks from the workshop were recorded. I will post a link when they appear online. We’re hoping to produce a kind of white paper that lays out the terminology that we can refer to in the future. And I am working on a new paper in which I try to better explain why I think that either superdeterminism or retrocausality is almost certainly correct. So this isn’t the end of the story, it’s just the beginning. Stay tuned. 

Saturday, February 27, 2021

Schrödinger’s Cat – Still Not Dead

[This is a transcript of the video embedded below.]


The internet, as we all know, was invented so we can spend our days watching cat videos, which is why this video is about the most famous of all science cats, Schrödinger’s cat. It is really both dead and alive? If so, what does that mean? And what has recent research to say about it? That’s what we’ll talk about today.

Quantum mechanics has struck physicists as weird ever since its discovery, more than a century ago. One especially peculiar aspect of quantum mechanics is that it forces you to accept the existence of superpositions. That are systems which can be in two states at the same time, until you make a measurement, which suddenly “collapses” the superposition into one definite measurement outcome.

The system here could be a single particle, like a photon, but it could also be a big object made of many particles. The thing is that in quantum mechanics, if two states exist separately, like an object being here and being there, then the superposition – that is the same object both here and there – must also exist. We know this experimentally, and I explained the mathematics behind this in an earlier video.

Now, you may think that being in a quantum superposition is something that only tiny particles can do. But these superpositions for large objects can’t be easily ignored, because you can take the tiny ones and amplify them to macroscopic size.

This amplification is what Erwin Schrödinger wanted to illustrate with a hypothetical experiment he came up with in 1935. In this experiment, a cat is in a box, together with a vial of poison, a trigger mechanism, and a radioactive atom. The nucleus of the atom has a fifty percent chance of decaying in a certain amount of time. If it decays, the trigger breaks the vial of poison, which kills the cat.

But the decay follows the laws of quantum physics. Before you measure it, the nucleus is both decayed and not decayed, and so, it seems that before one opens the box, the cat is both dead and alive. Or is it?

Well, depends on your interpretation of quantum mechanics, that is, what you think the mathematics means. In the most widely taught interpretation, the Copenhagen interpretation, the question what state the cat is in before you measure it is just meaningless. You’re not supposed to ask. The same is the case in all interpretations according to which quantum mechanics is a theory about the knowledge we have about a system, and not about the system itself.

In the many-worlds interpretation, in contrast, each possible measurement outcome happens in a separate universe. So, there’s a universe where the cat lives and one where the cat dies. When someone opens the box, that decides which universe they’re in. But for what observations are concerned, the result is exactly the same as in the Copenhagen interpretation.

Pilot wave-theory, which we talked about earlier, says that the cat is really always in only one state, you just don’t know which one it is until you look. The same is the case for spontaneous collapse models. In these models, the collapse of the wave-function is not merely an update when you open the box, but it’s a physical process.

It’s no secret that I myself am signed up to superdeterminism, which means that the measurement outcome is partly determined by the measurement settings. In this case, the cat may start out in a superposition, but by the time you measure it, it has reached the state which you actually observe. So, there is no sudden collapse in superdeterminism, it’s a smooth, deterministic, and local process.

Now, one cannot experimentally tell apart interpretations of mathematics, but collapse models, superdeterminism, and, under certain circumstances, pilot wave theory, make different predictions than Copenhagen or many worlds. So, clearly, one wants to do the experiment!

But. As you have undoubtedly noticed, cats are usually either dead or alive, not both. The reason is that even tiny interactions with a quantum system have the same effect as a measurement, and large objects, like cats, just constantly interact with something, like air or the cosmic background radiation. And that’s already sufficient to destroy a quantum superposition of a cat so quickly we’d never observe it. But physicists are trying to push the experimental boundary for bringing large objects into quantum states.

For example, in 2013, a team of physicists from the University of Calgary in Canada amplified a quantum superposition of a single photon. They first fired the photon at a partially silvered mirror, called a beam splitter, so that it became a superposition of two states: it passed through the mirror and also reflected back off it. Then they used one part of this superposition to trigger a laser pulse, which contains a whole lot of photons. Finally, they showed that the pulse was still in a superposition with the single photon. In another 2019 experiment, they amplified both parts of this superposition, and again they found that the quantum effects survived, for up to about 100 million photons.

Now, a group of 100 million photons not a cat, but it is bigger than your standard quantum particle. So, some headlines referred to this as the “Schrödinger's kitten” experiment.

But just in case you think a laser pulse is a poor approximation for a cat, how about this. In 2017, scientists at the University of Sheffield put bacteria in a cavity between two mirrors and they bounced light between the mirrors. The bacteria absorbed, emitted, and re-absorbed the light multiple times. The researchers could demonstrate that this way, some of the bacterias’ molecules became entangled with the cavity, so that is a special case of a quantum superposition.

However, a paper published the following year by scientists at Oxford University argued that the observations on the bacteria could also be explained without quantum effects. Now, this doesn’t mean that this is the correct explanation. Indeed, it doesn’t make much sense because we already know that molecules have quantum effects and they couple to light in certain quantum ways. However, this criticism demonstrates that it can be difficult to prove that something you observe is really a quantum effect, and the bacteria experiment isn’t quite there yet.

Let us then talk about a variant of Schrödinger’s cat that Eugene Wigner came up with in the nineteen-sixties. Imagine that this guy Wigner is outside the laboratory in which his friend just opens the box with the cat. In this case, not only would the cat be both dead and alive before the friend observes it, the friend would also both see a dead cat and see a live cat, until Wigner opens the door to the room where the experiment took place.

This sounds both completely nuts as well as an unnecessary complication, but bear with me for a moment, because this is a really important twist on Schrödinger’s cat experiment. Because if you think that the first measurement, so the friend observing the cat, actually resulted in a definite outcome, just that the friend outside the lab doesn’t know it, then, as long as the door is closed, you effectively have a deterministic hidden variable model for the second measurement. The result is clear already, you just don’t know what it is. But we know that deterministic hidden variable models cannot produce the results of quantum mechanics, unless they are also superdeterministic.

Now, again, of course, you can’t actually do the experiment with cats and friends and so on because their quantum effects would get destroyed too quickly to observe anything. But recently a team at Griffith University in Brisbane, Australia, created a version of this experiment with several devices that measure, or observe, pairs of photons. As anticipated, the measurement result agrees with the predictions of quantum mechanics.

What this means is that one of the following three assumptions must be wrong:

1. No Superdeterminism.
2. Measurements have definite outcomes.
3. No spooky action at a distance.

The absence of superdeterminism is sometimes called “Free choice” or “Free will”, but really it has nothing to do with free will. Needless to say, I think what’s wrong is rejecting superdeterminism. But I am afraid most physicists presently would rather throw out objective reality. Which one are you willing to give up? Let me know in the comments.

As of now, scientists remain hard at work trying to unravel the mysteries of Schrödinger's cat. For example, a promising line of investigation that’s still in its infancy is to measure the heat of a large system to determine whether quantum superpositions can influence its behavior. You find references to that as well as to the other papers that I mentioned in the info below the video. Schrödinger, by the way, didn’t have a cat, but a dog. His name was Burschie.

Tuesday, October 01, 2013

Testing Conspiracy Theories

I'm about to fly to Vienna where I'll be attending a conference on Emergent Quantum Mechanics. I'm not entirely sure why I was invited to this event, but I suspect it's got something to do with me being one of the three people on the planet who like superdeterministic hidden variables theories, more commonly known as "conspiracy theories".

Leaving aside some loopholes that are about to be closed, tests of Bell's theorem rule out local hidden variables theories. But any theorem is only as good as the assumptions that go into it, and one of these assumptions is that the experimenter can freely chose the detector settings. As you know, I don't believe in free will, so I have an issue with this. You can see though why theories in which this assumption does not hold are known as "conspiracy theories". While they are not strictly speaking ruled out, it seems that the universe must be deliberately mean to prevent the experimentalists from doing what they want, and this option is thus often not taken seriously.

But really, this is a very misleading interpretation of superdeterminism. All that superdeterminism means is that a state cannot be prepared independently of the detector settings. That's non-local of course, but it's non-local in a soft way, in the sense that it's a correlation but doesn't necessarily imply a 'spooky' action at a distance because the backwards lightcones of the detector and state (in a reasonable universe) intersect anyway.

That having been said, you might like or not like superdeterministic hidden variables theories, the real question is if there is some way to test if that's how nature works, because one can't use Bell's theorem here. After some failed attempts, I finally came up with a possible test that is almost model-independent, and it was published in my paper "Testing super-deterministic hidden variables theories".

I actually wrote this paper in the hospital when I was pregnant. The nurses kept asking me if I'm writing a book. They were quite disappointed to be drowned in elaborations on the foundations of quantum mechanics rather than hearing a vampire story. In any case, in the expectation that the readers on this blog are somewhat more sympathetic to the question whether the universe is fundamentally deterministic or not, here a brief summary of the idea.

The central difference between standard quantum mechanics and superdeterministic hidden variables theories is that in the former case two identically prepared states can give two different measurement outcomes, while in the latter case that's not possible. Unfortunately, "identically prepared" includes the hidden variables and it's difficult to identically prepare something that you can't measure. That is after all the reason why it looks indeterministic.

However, rather than trying to prepare identical states we can try to make repeated measurements on the same state. For that, take two non-commuting variables (for example the spin or polarization in two different directions) and measure them alternately. In standard quantum mechanics the measurement outcomes will be non-correlated. In a superdeterministric hidden variables theory, they'll be correlated - provided you can make a case that the hidden variables don't change in between the measurements. The figure below shows an example for an experimental setup.

A particle (electron/photon) is bounced back and forth between
two mirrors (grey bars). The blue and red bars indicate measurements
of two non-commuting variables, only one eigenvalue passes, the
other leaves the system. The quantity to measure is the average time
it takes until the particle leaves. In a superdeterministic theory,
it can be significantly longer than in standard quantum mechanics.


The provision that the hidden variables don't change is the reason why the test is only 'almost' model independent, because I made the assumptions that the hidden variables are due to the environment (the experimental setup) down to the relevant scales of the interactions taking place. This means basically if you make the system small and cool and measure quickly enough you have a chance to see the correlation between subsequent measurements. I made some estimates (see paper) and it seems possible with today's technology to make this test.

Interestingly, after I had finished a draft of the paper, Chris Fuchs sent me a reference to a 1970 article by Eugene Wigner where, in a footnote, Wigner mentions Von Neumann discussing exactly this type of experiment:
“Von Neumann often discussed the measurement of the spin component of a spin-1/2 particle in various directions. Clearly, the possibilities for the two possible outcomes of a single such measurement can be easily accounted for by hidden variables [...] However, Von Neumann felt that this is not the case for many consecutive measurements of the spin component in various different directions. The outcome of the first such measurement restricts the range of values which the hidden parameters must have had before that first measurement was undertaken. The restriction will be present also after the measurement so that the probability distribution of the hidden variables characterizing the spin will be different for particles for which the measurement gave a positive result from that of the particles for which the measurement gave a negative result. The range of the hidden variables will be further restricted in the particles for which a second measurement of the spin component, in a different direction, also gave a positive result...”
Apparently there was a longer discussion with Schrödinger following this proposal, which could be summarized with saying that the experiment cannot test generic superdeterminism, but only certain types as I already said above. If you think about it for a moment, you can never rule out generic superdeterminism anyway, so why even bother.

I'm quite looking forward to this conference, to begin with because Vienna is a beautiful city and I haven't been there for a while, but also because I'm hoping to meet some experimentalists who can tell me if I'm nuts :p

Update: Slides of my talk are here.

Monday, November 07, 2016

Steven Weinberg doesn’t like Quantum Mechanics. So what?

A few days ago, Nobel laureate Steven Weinberg gave a one-hour lecture titled “What’s the matter with quantum mechanics?” at a workshop for science writers organized by the Council for the Advancement of Science Writing (CASW).

In his lecture, Weinberg expressed a newfound sympathy for the critics of quantum mechanics.
“I’m not as happy about quantum mechanics as I used to be, and not as dismissive of the critics. And it’s a bad sign in particular that those physicists who are happy about quantum mechanics, who see nothing wrong with it, don’t agree with each other about what it means.”
You can watch the full lecture here. (The above quote is at 17:40.)


It’s become a cliché that physicists in their late years develop an obsession with quantum mechanics. On this account, you can file Weinberg together with Mermin and Penrose and Smolin. I’m not sure why that is. Maybe it’s something which has bothered them all along, they just never saw it as important enough. Maybe it’s because they start paying more attention to their intuition, and quantum mechanics – widely regarded as non-intuitive – begins itching. Or maybe it’s because they conclude it’s the likely reason we haven’t seen any progress in the foundations of physics for 30 years.

Whatever Weinberg’s motivation, he doesn’t like neither Copenhagen, nor Many Worlds, nor decoherent or consistent histories, and he seems to be allergic to pilot waves (1:02:15). As for qbism, which Mermin finds so convincing, that doesn’t even seem noteworthy to Weinberg.

I learned quantum mechanics in the mid-1990s from Walter Greiner, the one with the textbook series. (He passed away a few weeks ago at age 80.) Walter taught the Copenhagen Interpretation. The attitude he conveyed in his lectures was what Mermin dubbed “shut up and calculate.”

Of course I as most other students spent some time looking into the different interpretations of quantum mechanics – nothing’s more interesting than the topics your prof refuses to talk about. But I’m an instrumentalist by heart and also I quite like the mathematics of quantum mechanics, so I never had a problem with the Copenhagen Interpretation. I’m also, however, a phenomenologist. And so I’ve always thought of quantum mechanics as an incomplete, not fundamental, theory which needs to be superseded by a better, underlying explanation.

My misgivings of quantum mechanics are pretty much identical to the ones which Weinberg expresses in his lecture. The axioms of quantum mechanics, whatever interpretation you chose, are unsatisfactory for a reductionist. They should not mention the process of measurement, because the fundamental theory should tell you what a measurement is.

If you believe the wave-function is a real thing (psi-ontic), decoherence doesn’t solve the issue because you’re left with a probabilistic state that needs to be suddenly updated. If you believe the wave-function only encodes information (psi-epistemic) and the update merely means we’ve learned something new, then you have to explain who learns and how they learn. None of the currently existing interpretations address these issues satisfactorily.

It isn’t so surprising I’m with Weinberg on this because despite attending Greiner’s lectures, I never liked Greiner’s textbooks. That we students were more or less forced to buy them didn’t make them any more likable. So I scraped together my Deutsche Marks and bought Weinberg’s textbooks, which I loved for the concise mathematical approach.

I learned both general relativity and quantum field theory from Weinberg’s textbooks. I also later bought Weinberg’s lectures on Quantum Mechanics which appeared in 2013, but haven’t actually read them, except for section 3.7, where he concludes that:
“[T]oday there is no interpretation of quantum mechanics that does not have serious flaws, and [we] ought to take seriously the possibility of finding some more satisfactory other theory, to which quantum mechanics is merely a good approximation.”
It’s not much of a secret that I’m a fan of non-local hidden variables (aka superdeterminism), which I believe to be experimentally testable. To my huge frustration, however, I haven’t been able to find an experimental group willing and able to do that. I am therefore happy that Weinberg emphasizes the need to find a better theory, and to also look for experimental evidence. I don’t know what he thinks of superdeterminism. But superdeterminism or something else, I think probing quantum mechanics in new regimes is best shot we presently have at making progress on the foundations of physics.

I therefore don’t understand the ridicule aimed at those who think that quantum mechanics needs an overhaul. Being unintuitive and feeling weird doesn’t make a theory wrong – we can all agree on this. We don’t even have to agree it’s unintuitive – I actually don’t think so. Intuition comes with use. Even if you can’t stomach the math, you can build your quantum intuition for example by playing “Quantum Moves,” a video game that crowd-sources players’ solutions for quantum mechanical optimization problems. Interestingly, humans do better than algorithms (at least for now).

[Weinberg (left), getting some
kind of prize or title. Don't
know for what. Image: CASW]
So, yeah, maybe quantum physics isn’t weird. And even if it is, being weird doesn’t make it wrong, and therefore you don’t think it’s a promising research avenue to pursue. Fine, then don’t. But before you make jokes about physicists who rely on their intuition, let us be clear that being ugly doesn’t make a theory wrong either. And yet it’s presently entirely acceptable to develop new theories with the only aim of prettifying the existing ones.

I don’t think for example that numerological coincidences are problems worth thinking about – they’re questions of aesthetic appeal. The mass of the Higgs is much smaller than the Planck mass. So what? The spatial curvature of the universe is almost zero, the cosmological constant tiny, and the electric dipole moment of the neutron is for all we know absent. Why should that bother me? If you think that’s a mathematical inconsistency, think again – it’s not. There’s no logical reason for why that shouldn’t be so. It’s just that to our human sense it doesn’t quite feel right.

A huge amount of work has gone into curing these “problems” because finetuned constants aren’t thought of as beautiful. But in my eyes the cures are all worse than the disease: Solutions usually require the introduction of additional fields and potentials for these fields and personally I think it’s much preferable to just have a constant – is there any axiom simpler than that?

The difference between the two research areas is that there are tens of thousands of theorists trying to make the fundamental laws of nature less ugly, but only a few hundred working on making them less weird. That in and by itself is reason to shift focus to quantum foundations, just because it’s the path less trodden and more left to explore.

But maybe I’m just old beyond my years. So I’ll shut up now and go back to my calculations.

Wednesday, January 22, 2020

Travel and Book Update

My book “Lost in Math” has meanwhile also been translated to Hungarian and Polish. Previous translations have appeared in German, Spanish, Italian, and French, I believe. I have somewhat lost overview. There should have been a Chinese and Romanian translation too, I think, but I’m not sure what happened to these. In case someone spots them, please let me know. The paperback version of the US-Edition is scheduled to appear in June.

My upcoming trips are to Cambridge, UK, for a public debate on the question “How is the scientific method doing?” (on Jan 28th) and a seminar about Superdeterminism (on Jan 29). On Feb 13, I am in Oxford (again) giving a talk about Superfluid Dark Matter (again), but this time at the physics department. On Feb 24th, I am in London for the Researcher to Reader Conference 2020.

On March 9th I am giving a colloq at Brown University. On March 19th I am in Zurich for some kind of panel discussion, details of which I have either forgotten or never knew. On April 8, I am in Gelsenkirchen for a public lecture.

Our Superdeterminism workshop is scheduled for the first week of May (details to come soon). Mid of May I am in Copenhagen for a public lecture. In June I’ll be on Long Island for a conference on peer review organized by the APS.

The easiest way to keep track of my whatabouts and whereabouts is to follow me on Twitter or on Facebook.

Thursday, March 12, 2020

Essays, Elsewhere

Just a brief note that Tim and I have an essay up at Nautilus
How to Make Sense of Quantum Physics
Superdeterminism, a long-abandoned idea, may help us overcome the current crisis in physics.
BY SABINE HOSSENFELDER & TIM PALMER

Quantum mechanics isn’t rocket science. But it’s well on the way to take the place of rocket science as the go-to metaphor for unintelligible math. Quantum mechanics, you have certainly heard, is infamously difficult to understand. It defies intuition. It makes no sense. Popular science accounts inevitably refer to it as “strange,” “weird,” “mind-boggling,” or all of the above.

We beg to differ. Quantum mechanics is perfectly comprehensible. It’s just that physicists abandoned the only way to make sense of it half a century ago. Fast forward to today and progress in the foundations of physics has all but stalled. The big questions that were open then are still open today. We still don’t know what dark matter is, we still have not resolved the disagreement between Einstein’s theory of gravity and the standard model of particle physics, and we still do not understand how measurements work in quantum mechanics.

How can we overcome this crisis? We think it’s about time to revisit a long-forgotten solution, Superdeterminism, the idea that no two places in the universe are truly independent of each other. This solution gives us a physical understanding of quantum measurements, and promises to improve quantum theory. Revising quantum theory would be a game changer for physicists’ efforts to solve the other problems in their discipline and to find novel applications of quantum technology.

Head over to Nautilus to read the whole thing. It’s a great magazine, btw, and I warmly recommend you follow it.

If you found that interesting, you may also be interested in my contribution to this year’s essay contest from the Foundational Questions Institute on Undecidability, Uncomputability, and Unpredictability:
Math Matters
By Sabine Hossenfelder

Gödel taught us that mathematics is incomplete. Turing taught us some problems are undecidable. Lorenz taught us that, try as we might, some things will remain unpredictable. Are such theorems relevant for the real world or are they merely academic curiosities? In this essay, I first explain why one can rightfully be skeptical of the scientific relevance of mathematically proved impossibilities, but that, upon closer inspection, they are both interesting and important.

Friday, May 22, 2020

Book Update

The US version of my book "Lost in Math" is about to be published as paperback. You can now pre-order it, which of course you should.

I am quite pleased that what I wrote in the book five years ago has held up so beautifully. There has been zero further progress in the foundations of physics and, needless to say, there will be zero progress until physicists understand that they need to change their methodology. Chances that they actually understand this are not exactly zero, but very close by.

In other news, on Monday I gave an online seminar about Superdeterminism, which was recorded and is now available on YouTube. Don't despair if Tim doesn't quite make sense to you; it took me a year to figure out that he isn't as crazy as he sounds.

The How The Light Gets in Festival which is normally in Hay-on-Wye has also been moved online. I think that's great, because Hay-on-Wye is a tiny village somewhere in the middle of nowhere and traveling there has been somewhat of a pain. Indeed, I had actually declined the invitation months ago. But since I can now attend without having to sit on a car, a bus, a plane, a train, and a taxi, I will be in a debate about Supersymmetry tomorrow (May 23) at 11:30am BST (not CEST) and giving a 30 mins talk about my book at 2pm (again that's BST).

Sunday, June 16, 2019

Book review: “Einstein’s Unfinished Revolution” by Lee Smolin

Einstein’s Unfinished Revolution: The Search for What Lies Beyond the Quantum
By Lee Smolin
Penguin Press (April 9, 2019)

Popular science books cover a spectrum from exposition to speculation. Some writers, like Chad Orzel or Anil Ananthaswamy, stay safely on the side of established science. Others, like Philip Ball in his recent book, keep their opinions to the closing chapter. I would place Max Tegmark’s “Mathematical Universe” and Lee Smolin’s “Trouble With Physics” somewhere in the middle. Then, on the extreme end of speculation, we have authors like Roger Penrose and David Deutsch who use books to put forward ideas in the first place. “Einstein’s Unfinished Revolution” lies on the speculative end of this spectrum.

Lee is very upfront about the purpose of his writing. He is dissatisfied with the current formulation of quantum mechanics. It sacrifices realism, and he thinks this is too much to give up. In the past decades, he has therefore developed his own approach to quantum mechanics, the “ensemble interpretation”. His new book lays out how this ensemble interpretation works and what its benefits are.

Before getting to this, Lee introduces the features of quantum theories (superpositions, entanglement, uncertainty, measurement postulate, etc) and discusses the advantages and disadvantages of the major interpretations of quantum mechanics (Copenhagen, many worlds, pilot wave, collapse models). He deserves applause for also mentioning the Montevideo interpretation and superdeterminism, though clearly he doesn’t like either. I have found his evaluation of these approaches overall balanced and fair.

In the later chapters, Lee comes to his own ideas about quantum mechanics and how these tie together with his other work on quantum gravity. I have not been able to follow all his arguments here, especially not on the matter of non-locality.

Unfortunately, Lee doesn’t discuss his ensemble interpretation half as critically the other approaches. From reading his book you may get away with the impression he has solved all problems. Let me therefore briefly mention the most obvious shortcomings of his approach. (a) To quantify the similarity of two systems you need to define a resolution. (b) This will violate Lorentz-invariance which means it’s hard to make compatible with standard model physics. (c) You better not ask about virtual particles. (d) If a system gets its laws from precedents, where do the first laws come from? Lee tells me that these issues have been discussed in the papers he lists on his website.

As all of Lee’s previous books, this one is well-written and engaging, and if you liked Lee’s earlier books you will probably like this one too. The book has the occasional paragraph that I think will be over many reader’s head, but most of it should be understandable with little or no prior knowledge. I have found this book particularly valuable for spelling out the author’s philosophical stance. You may not agree with Lee, but at least you know where he is coming from.

This book is recommendable for anyone who is dissatisfied with the current formulation of quantum mechanics, or who wants to understand why others are dissatisfied with it. It also serves well as a quick introduction to current research in the foundations of quantum mechanics.

[Disclaimer: free review copy.]