As you know, I am presently at a workshop at Perimeter Institute about the Laws of Nature: Their Nature and Knowability. Yesterday, we had a talk by Marcelo Gleiser titled “What can we know of the world?”. It occurred to me somewhat belatedly that I recently read an article by Gleiser in New Scientist, “The imperfect universe: Goodbye, theory of everything.” In that article, he writes that after “Fiften years [as] a physicist hard at work hunting for a theory of nature that would unify the very big and the very small” he has come to the conclusion that “the very notion of a final theory is faulty.” In a nutshell, that was also what his talk was about.
The only thing that's interesting about this insight is that it took him 15 years to arrive there. And maybe, why it got printed in New Scientist. Of course the notion of a final, fundamental, theory of all and everything is faulty. For the simple reason that even if we had a theory that explained everything we know, we could never be sure it would eternally remain the theory of everything we know. As Popper already realized about a century ago, one cannot verify a theory, it can only be falsified. Thus, theories we have are forever out for test, always on the risk that some new data does not fit in. That's exactly what makes a theory scientific. It's also one of the points I made in my FQXi essay. You see, I'm an even Newer Scientist.
That we can never know whether a theory is truly fundamental and able to explain all observable phenomena of course does not mean there is no fundamental theory. It just means we can never know - so your believe that such a theory exists belongs in the realm of religion, not science.
In any case, in his talk (video and audio here), Gleiser touched on another topic that reminded me of something else. He had a sketch of our expanding knowledge, with a filled circle representing “The Known” in the middle, that is expanding into what is now the unknown (“perennial ignorance”) outside:
I used a similar, though slightly different analogy for the progress of science in my PI public lecture some years ago (which incidentally has the same title as the FQXi essay, I'm very into recycling). In this case though, I used a map of Middle Earth.
The message that I wanted to convey is that the process of knowledge discovery is very similar to exploring unknown territory. There are parts that you have already seen and that you know very well, though details may be missing. And let me be clear that with “The Known” (in contrast to Gleiser) I don't mean laws themselves but the data from which the laws were extracted. Otherwise you lose information that is possibly important about the range of applicability (information you at first possibly didn't think was relevant).
You try to explain the known by a theory, and if everything fits you point somewhere into the unknown (make a prediction). Ideally, experimentalists go there and find what you told them they would find. You don't want to point out too far because people today are quite impatient, and if your prediction is not measurable within their lifetime it won't help you get tenure. The other way progress happens is that there is data available for which a theoretical explanation is missing. Or a theory might be sketchy and not work very well. That's the situation of the experimentalist saying: we've seen something on the horizon, please explain that. The body of knowledge that we have is usually not neatly simply connected, but typically has some pieces that don't really match with anything else.
Which brings me back to Gleiser's article then. The essential question is not whether you do or don't believe in a fundamental theory of everything. The essential question is what is a good and promising way to expand what is known. You can believe in flying spaghetti monsters, reincarnation, or a theory of everything: if it helps you with your research, by all means, go ahead, just don't put your believes in the abstract of your paper.
Experimental input is of course essential to progress all along. On the theoretical side, the obvious reason why people are looking for a unification of the known forces is that unification has worked previously and has been tremendously successful. The same holds for symmetry principles. Sure, that doesn't mean these procedures will continue to be successful, but it's the obvious thing to try. It's the same reason why a band's second hit sounds like the first, and why, after my move to Sweden I first had to learn that asking to speak to a supervisor and complaining about lacking customer service is not a very successful tactic in this country. Similarly, we might have to reconsider our tactics and learn new ways of thinking if we remain unsuccessful making headway on today's big questions in physics. For example when it comes to resolving the apparent tension between General Relativity and quantum mechanics, or to explain the arrow of time, rspt the initial conditions of the universe: It's terry incognita and there may be dragons.
That's why I find meetings like the current one at PI very useful to become more aware of our standard mode of thinking, for awareness and acknowledgement of limitations of a procedure is the first step to improvement.