Research in theoretical physics generally does one of two things: Either we have some data that require explanation for which a theory must be developed. Or we have a theory that requires improvement, and the improved theory leads to a prediction which is then experimentally tested.
I have noticed that some people think theoretical physics is something special to the foundations of physics. But that isn’t so. All subdisciplines of physics have an experimental part and a theoretical part. How much the labor is divided into different groups of people depends strongly on the field. In some parts of astrophysics, for example, data collection, analysis, and theory-development is done by pretty much the same people. That’s also the case in some parts of condensed matter physics. In these areas many experimentalists are also theorists. But if you look at fields like cosmology or high energy particle physics, people tend to specialize either in experiment or in theory development.
Theoretical physics is pretty much a job like any other in that you get an education and then you put your knowledge to work. You find theoretical physicists in public higher education institutions, which is probably what you are most familiar with, but you also find them in the industry or in non-profit research institution like the one I work at. Just what the job entails depends on the employer. Besides the research, a theoretical physicist may have administrational duties, or may teach, mentor students, do public outreach, organize scientific meetings, sit on committees and so on.
When it comes to the research itself, theoretical physics doesn’t work any different from other disciplines of science. The largest part of research, ninetynine percent, is learning what other people have done. This means you read books and papers, go to seminars, attend conferences, listen to lectures and you talk to people until you understand what they have done.
And as you do that, you probably come across some open problems. And from those you pick one for your own research. You would pick a problem that, well, you are interested in, but also something that you think will move the field forward and, importantly, you pick a problem that you think you have a reasonable chance of solving with what you know. Picking a research topic that is both interesting and feasible is not easy and requires quite some familiarity with the literature, which is why younger researchers usually rely on more senior colleagues to pick a topic.
Where theoretical physics is special is in the amount of mathematics that we use in our research. In physics all theories are mathematical. This means both that you must know how to model a natural system with mathematics and you must know how to do calculations within that model. Of course we now do a lot of calculations numerically, on a computer, but you still have to understand the mathematics that goes into this. There is really no way around it. So that’s the heart of the job, you have to find, understand, and use the right mathematics to describe nature.
The thing that a lot people don’t understand is just how constraining mathematics is in theory development. You cannot just dream up a particle, because almost everything that you can think of will not work if you write down the mathematics. It’s either just nonsense or you find quickly that it is in conflict with observation already.
But the job of a theoretical physicist is not done with finishing a calculation. Once you have your results, you have to write them up and publish them and then you will give lectures about it so that other people can understand what you have done and hopefully build on your work.
What’s fascinating about theoretical physics is just how remarkably well mathematics describes nature. I am always surprised if people tell me that they never understood physics because I would say that physics is the only thing you can really understand. It’s the rest of the world that doesn’t make sense to me.