You say theoretical physicists are doing their job all wrong. Don’t you doubt yourself?

This is me with John Horgan, yesterday.
This photo is only here so
the share widgets work properly.

One of the most frequent critical remarks I have gotten on my book is that I seem confident. I was supposed, it seems, to begin each paragraph with “I’m sorry, but.”

But I am not sorry. I mean what I say. Yes, in the foundations of physics we are financing some 15,000 or so theorists who keep producing useless scientific articles because they believe the laws of nature must be beautiful. That’s exactly what I am saying.

Let us leave aside for a moment that you have to skip half the book to not notice I question myself on every other page. Heck, if you ask me to sign the book, I’m afraid I’ll misspell my own name. I’m a walking-talking bag of self-doubt. Indeed that was the reason I ended up writing this book.

See, I don’t understand what’s going on with this community. Everyone knows there’s no reason that a scientific explanation must appeal to the human sense of beauty. Right? Doesn’t everyone know this? Science is about explaining observations, regardless of whether we like these explanations.

But if it’s clear that putting forward new hypotheses just because they are beautiful doesn’t mean they’re likely to be right, then why do theorists in these fields focus so much on beauty? Worse, why do they continue to focus on the same type of beauty, even though that method has demonstrably not worked for 40 years?

At first I considered there might be a mathematical basis to their arguments which I was missing. That there is a solid reason why a theory must be natural, or that the fundamental forces must be unified, or that the mathematics of a theory must be “fruitful” and “have deep connections” and be “rigid” – to quote some expressions people in the foundations of physics commonly use. But there is no mathematical basis. Arguments from beauty are additional assumptions, and they are unnecessary to make a theory work.

Indeed, some philosophers have suggested I speak of “metaphysical assumptions” rather than “aesthetic arguments”, but I think the latter captures the historical origin better. These arguments trace back to tales about God’s beautiful creations. Also, if I’d call it metaphysics no one would know what I am talking about.

I then considered that using criteria from beauty is justified because it has historically been successful. This would leave open the question why that would be so – I cannot think of a reason such a connection should exist. But in any case, history speaks against it. Relying on beauty has sometimes worked, and sometimes not. It’s just that many theoretical physicists prefer to recall only the cases where arguments from beauty did work. And in hindsight they then reason that the wrong ideas were not all that beautiful. Needless to say, that’s not a good way to evaluate evidence.

Finally, the use of criteria from beauty in the foundations of physics is, as a matter of fact, not working. Beautiful theories have been ruled out in the hundreds, theories about unified forces and new particles and additional symmetries and other universes. All these theories were wrong, wrong, wrong. Relying on beauty is clearly not a successful strategy.

So I have historical evidence, math, and data. In my book I lay out these points and tell the reader what conclusion I have drawn: Beauty is not a good guide to theory-development.

I then explain that this widespread use of scientifically questionable but productive methodology is symptomatic to the current organization of academic research, and a problem that’s not confined to physics.

Now, look, just because I cannot find a reason that beautiful theories are more promising than ugly ones doesn’t mean that relying on beauty cannot work. It may work, if we get lucky. Neither, for that matter, do I think that if we find a new law of nature it must be ugly. Chances are we will come to find a successful new idea beautiful simply because it works. But our sense of beauty changes and adapts, and therefore I do not think that using criteria of beauty from the past is a promising route to future progress.

Needless to say, making a case against a community of some thousands of the biggest brains on the planet has not been conducive to my self-confidence. But I have tried to find a scientific reason for the methods which my colleagues use in theory-development and could not. I wrote the book because I think it’s my responsibility as scientist to say clearly that I have come to the conclusion what goes on the foundations of physics is a waste of money, and that the public is being misinformed about the promise of this work.

I do not think that this will change the mind of people in the field. They have nothing to worry about because the way that academia is currently organized there is safety in numbers.

So, yes, I doubt myself. But I have written a whole book in which I explain why I have arrived at my conclusion. Rather than asking me, you should ask the people who work in these fields what makes them so certain that beautiful ideas are promising descriptions of nature.