But disillusionment followed swiftly when I read the paper.
Gian Francesco Giudice is a theoretical physicist at CERN. He is maybe not the most prominent member of his species, but he has been extremely influential in establishing “naturalness” as a criterion to select worthwhile theories of particle physics. Together with Riccardo Barbieri, Giudice wrote one of the pioneering papers on how to quantify naturalness, thereby significantly contributing to the belief that it is a scientific criterion. To date the paper has been cited more than 1000 times.
Giudice was also the first person I interviewed for my upcoming book about the relevance of arguments from beauty in particle physics. It became clear to me quickly, however, that he does not think naturalness is an argument from beauty. Instead, Giudice, like many in the field, believes the criterion is mathematically well-defined. When I saw his new paper, I hoped he’d come around to see the mistake. But I was overly optimistic.
As Giudice makes pretty clear in the paper, he still thinks that “naturalness is a well-defined concept.” I have previously explained why that is wrong, or rather why, if you make naturalness well-defined, it becomes meaningless. A quick walk through the argument goes as follows.
Naturalness in quantum field theories – ie, theories of the type of the standard model of particle physics – means that a theory at low energies does not sensitively depend on the choice of parameters at high energies. I often hear people say this means that “the high-energy physics decouples.” But note that changing the parameters of a theory is not a physical process. The parameters are whatever they are.
The processes that are physically possible at high energies decouple whenever effective field theories work, pretty much by definition of what it means to have an effective theory. But this is not the decoupling that naturalness relies on. To quantify naturalness you move around between theories in an abstract theory space. This is very similar to moving around in the landscape of the multiverse. Indeed, it is probably not a coincidence that both ideas became popular around the same time, in the mid 1990s.
If you now want to quantify how sensitively a theory at low energy depends on the choice of parameters at high energies, you first have to define the probability for making such choices. This means you need a probability distribution on theory space. Yes, it’s the exact same problem you also have for inflation and in the multiverse.
In most papers on naturalness, however, the probability distribution is left unspecified which implicitly means one chooses a uniform distribution over an interval of about length 1. The typical justification for this is that once you factor out all dimensionful parameters, you should only have numbers of order 1 left. It is with this assumption that naturalness becomes meaningless because you have now simply postulated that numbers of order 1 are better than other numbers.
You wanted to avoid arbitrary choices, but in the end you had to make an arbitrary choice. This turns the whole idea ad absurdum.
That you have to hand-select a probability distribution to make naturalness well-defined used to be well-known. One of the early papers on the topic clearly states
“The “theoretical license” at one’s discretion when making this choice [for the probability distribution] necessarily introduces an element of arbitrariness to the construction.”
Anderson and Castano, Phys. Lett. B 347:300-308 (1995)
Giudice too mentions “statistical comparisons” on theory space, so I am sure he is aware of the need to define the distribution. He also writes, however, that “naturalness is an inescapable consequence of the ingredients generally used to construct effective field theories.” But of course it is not. If it was, why make it an additional requirement?
(At this point usually someone starts quoting the decoupling theorem. In case you are that person let me say that a) no one has used mass-dependent regularization schemes since the 1980s for good reasons, and b) not only is it questionable to assume perturbative renormalizability, we actually know that gravity isn’t perturbatively renormalizable. In other words, it’s an irrelevant objection, so please let me go on.)
In his paper, Giudice further claims that “naturalness has been a good guiding principle” which is a strange thing to say about a principle that has led to merely one successful prediction but at least three failed predictions, more if you count other numerical coincidences that physicists obsess about like the WIMP miracle or gauge coupling unification. The tale of the “good guiding principle” is one of the peculiar myths that gets passed around in communities until everyone believes it.
Having said that, Giudice’s paper also contains some good points. He suggests, for example, that the use of symmetry principles in the foundations of physics might have outlasted its use. Symmetries might just be emergent at low energies. This is a fairly old idea which goes back at least to the 1980s, but it’s still considered outlandish by most particle physicists. (I discuss it in my book, too.)
Giudice furthermore points out that in case your high energy physics mixes with the low energy physics (commonly referred to as “UV/IR mixing”) it’s not clear what naturalness even means. Since this mixing is believed to be a common feature of non-commutative geometries and quite possibly quantum gravity in general, I have picked people’s brains on this for some years. But I only got shoulder shrugs, and I am none the wiser today. Giudice in his paper also doesn’t have much to say about the consequences other than that it is “a big source of confusion,” on which I totally agree.
But the conclusion that Giudice comes to at the end of his paper seems to be the exact opposite of mine.
I believe what is needed for progress in the foundations of physics is more mathematical rigor. Obsessing about ill-defined criteria like naturalness that don’t even make good working hypotheses isn’t helpful. And it would serve particle physicists well to identify their previous mistakes in order to avoid repeating them. I dearly hope they will not just replace one beauty-criterion by another.
Giudice on the other hand thinks that “we need pure unbridled speculation, driven by imagination and vision.” Which sounds great, except that theoretical particle physics has not exactly suffered from a dearth of speculation. Instead, it has suffered from a lack of sound logic.
Be that as it may, I found the paper insightful in many regards. I certainly agree that this is a time of crisis but that this is also an opportunity for change to the better. Giudice’s paper is very timely. It is also merely moderately technical, so I encourage you to give it a read yourself.
Hi Sabine,
ReplyDeleteI still fail to see why a coupling in range 1 would be natural. As far as I know, none of those beasts come in this range - not any - so I think nature tells us the opposite.
One question where I do not find any answer in what I read (which is not much actually): what would it mean if there is nothing more to find in any energy range?
Best,
J.
This comment has been removed by the author.
ReplyDelete"Indeed, it is probably not a coincidence that both ideas became popular around the same time, in the mid 1990s."
DeleteI guess you meant the idea quantifying naturalness, as opposed to naturalness itself?
Your previous comments on naturalness I found unconvincing because you seemed to be saying that an appropriate probability distribution can not, even in principle, be defined, as there is no ensemble from which you could extract frequencies. But as a Bayesian statistician / computer scientist I work mostly with *epistemic* probabilities. As a non-physicist I'm sure I can't appreciate the difficulties of creating an appropriate epistemic probability distribution, but I balk at a claim that it can't even be done *in principle*.
ReplyDeleteThis blog post, however, makes a point that I can agree with 100%: it is the responsibility of naturalness proponents to define and defend the prior distribution they have in mind.
Theory defining experiment "naturally" excludes falsifying observations. Success is empirical failure demanding better equipment. The 500 + 500 = 1000 GeV International Linear Collider rebudgets to 125 + 125 = 250 GeV, bonused bureaucrats, and no offending outputs. Enormous volumes of liquid water, argon, xenon; 3/4 tonne of TeO2 single crystals at 0.01 kelvin punctiliously see nothing.
ReplyDeletehttp://bigthink.com/videos/eric-weinstein-after-einstein-we-stopped-believing-in-lone-genius-is-it-time-to-believe-again
...”Transcript” lower left, pale.
CVs fatten with citations as answers remain obscure for not being within theory.
“The Character of Physical Law” and the corresponding Messenger Lectures by Feynman at Cornell in 1964 could really use an update. But I gather that such an undertaking might not be able to produce material that is comprehensible by the interested public.
ReplyDeleteEsoterica like naturalness may play a part or not but it seems a popular working principle. If our universe is just one of many, then perhaps there is no fundamental explanation for our physics. In that case perhaps the best we could do is “The Local Character of Physical Law.”
The chapters in Feynman’s book:
1 ) The Law of Gravitation as Example: No quantum gravity yet to supplant the Newtonian example shown. Einstein’s GR has received enormous conformational data since this book was published.
2 ) The Relation of Math to Physics: A lot more sophisticated math is in use now but the experimental side hasn’t progressed as much. (Group theory, knots, conformal whatever.)
3 ) The Great Conservation Principles: Not much new here (is there?) except R-parity, maybe.
4 ) Symmetry in Physical Law. More symmetries (10^500 ha, ha) are possible with M-theory, Lisi’s E8 model. But where is the experimental confirmation?
5 ) The Distinction of Past and Future: Sean Carroll’s Past Hypothesis could be added and Externalism. (This is one of my favorite topics.)
6 ) Probability and Uncertainty: A lot has been done in the area of interpreting quantum mechanics. Is there any more agreement that is supported by experiment than in 1964?
7 ) Seeking New Laws: Physicists have proposed many new laws since 1964. How many have been confirmed. Are physicists' methods better? They appear to be more prolific.
"I believe what is needed for progress in the foundations of physics is more mathematical rigor"
ReplyDeletewhat do you mean by that? foundations of physics by its nature at this moment is speculative. There is a whole journal dedicated to it.
example paper:(
Multi-Time Wave Functions Versus Multiple Timelike Dimensions
Matthias Lienert, Sören Petrat, Roderich Tumulka
"... theoretical particle physics has not exactly suffered from a dearth of speculation. Instead, it has suffered from a lack of sound logic."
ReplyDeleteKa-BLAM!
qsa,
ReplyDeleteWhat's your problem with the paper? Do you think it's wrong? Funny you would pick this particular paper as Roderich happens to be the person who taught me how to lead watertight proofs. I'm sure he knows what he's doing.
Of course theoretical physics is speculative in its nature. Physics isn't math and creativity is needed. But if you look at the history of physics then progress came from hitting on actual mathematical problems, not aesthetic itches. Eg, Newtonian gravity is actually incompatible with special relativity. Special relativity is actually incompatible with the non-relativistic Schroedinger equation. The standard model without the Higgs actually violates unitarity somewhere at LHC energies. These are actual problems. Naturalness problems aren't. They are aesthetic misgivings.
You cannot, of course, derive new physical theories, because logic alone only gets you so far. Conclusions always depend on the assumptions and those you can't prove, therefore you always need experimental check. What happens if you forget that you can see in the quantum gravity communities.
But at least by using math you can make sure that you are trying to solve an actual problem. Naturalness problems aren't actually problems. That the perturbative expansion doesn't converge, otoh, is an actual problem. So is, eg, Haag's theorem. Then there is of course the need to quantize gravity, also an actual problem. Maybe you know of others?
See, what happened with naturalness is that people erroneously came to believe it is a problem that requires a solution. It's a mistake that could have been prevented had they paid more attention to the math. Best,
B.
Kevin,
ReplyDeleteThat's right, the problem is that you are dealing with a "virtual" space (both here and in the multiverse) so there is no way to ever learn anything about the probability distribution - you have only one event in the sample. There is likewise no way to argue for one or the other prior (on that space) if you want to do a Bayesian assessment for what a 'probable' theory is. There are actually a few papers in the literature on Bayesian naturalness, typically assuming a uniform log prior, presumably because that is 'natural'. But it's just pushing the issue of 'making a choice' around under the carpet.
What you *can* do of course is just Bayesian inference on the space to find out which one best explains present data (as opposed to being probable by fiat) which would give you back the standard model in the IR with high confidence, but then we already knew that. I have no problem with this type of inference, but it does not prefer "natural" theories and is hence a different story altogether. Best,
B.
Bee,
ReplyDeleteI remember quite clearly, back around 1980, listening to Helen Quinn of SLAC complaining to another SLAC theorist that the younger generation were in danger of believing that musing about the correct definition of "naturalness" was actually doing physics.
I have not talked with Helen since on the issue and cannot attest to her views almost forty years later. But, I thought that you might like to know that the obsession with "naturalness" and criticism of that obsession goes back that far.
Dave Miller in Sacramento
It seems to me that a lot of progress in physics has come from picking an appealing heuristic and trying to see where it leads (Faraday, Einstein, Dirac, Feynman). I have more trouble seeing where emphasis on mathematical rigor has made a difference. Do you have some examples?
ReplyDeleteCIP,
ReplyDeleteWhat motivates an individual researcher is one thing. The actual reason why their hunch worked out another thing entirely. I already listed examples above. Special and General relativity, relativistic quantum mechanics, QED, they all solve actual mathematical problems (which of course require assumptions that are based on evidence). That this might not have been the reason why someone worked on it to begin with is psychologically interesting but factually irrelevant. Best,
B.
Tevong,
ReplyDeleteI meant the idea of moving around in a theory-landscape.
I thought Gian Giudice's essay sounded like a cry for help myself. But I'm afraid to say that if Guido Altarelli was still around, I'm not sure Gian would be listening. Read Particle headache: Why the Higgs could spell disaster by Matthew Chalmers. Note this: "It's a nice story, but one that some find a little contrived. "The minimal standard model Higgs is like a fairy tale", says Guido Altarelli of CERN near Geneva, Switzerland. "It is a toy model to make the theory match the data, a crutch to allow the standard model to walk a bit further until something better comes along". His problem is that the standard model is manifestly incomplete". The problem for particle physics is that people like Gian Giudice will not admit it. Because they've painted themselves into a corner with discoveries which have been cemented into place by Nobel prizes.
ReplyDeleteSabine,
ReplyDeleteLet me offer a different view on naturalness.
Naturalness cannot be easily discarded and it's not necessarily driven by aesthetic considerations. Look at the clustering principle of relativistic QFT, which ensures that phenomena on largely separated energy scales decouple. It is the basis of the Renormalization Group flow, which assumes that the low and high momenta are fully separable and that the high momenta can be sequentially integrated out. Naturalness lies at the heart of the fine-tuning problem in the Standard Model, where perturbative corrections drive the Higgs mass up to the Planck scale, in direct violation of the clustering principle. Naturalness also underlies the cosmological constant (CC) problem, where quantum contributions bring the CC to abnormally high values and break the clustering principle.
Having said that, whether naturalness should be the ultimate guide for model building beyond the Standard Model remains an open issue. You are probably right that other foundational principles may very well come into play. Only time will tell.
Ervin,
ReplyDeleteI explain in my blogpost why what you say is irrelevant. I suggest you read what I wrote about effective field theories and naturalness before making further comments. Best,
B>
Sabine,
ReplyDeleteBut how can one avoid making assumptions about initial conditions near the UV limit of the Renormalization Group flow?
All experimental observations on the effective theories we have today confirm the decoupling theorem. Therefore they confirm that the high and the low energy scales are decoupled. And so they implicitly confirm the choice of initial conditions.
Ervin,
ReplyDeletePhysical phenomena also decouple in a finetuned theory, that's what effective field theory does for you, by construction. The sensitivity that you probe with naturalness is a dependence of the parameters at low energy on the parameters at high energy. This is an additional requirement. But changing the parameters is not a physical process. What observations tell you today (at least in the most straight-forward interpretation) is that the theory *is* finetuned. That being the very problem that Giudice's paper is about. Best,
B.
"What observations tell you today (at least in the most straight-forward interpretation) is that the theory *is* finetuned."
ReplyDeleteYes, I agree. And the issue will continue to haunt us as long as the root cause of this apparent fine-tuning is not understood. We know now that some solutions are likely to fail. For example, low-scale SUSY is probably not the mechanism protecting the electroweak scale.
But this (and other) failures should not deter the search for novel answers to the fine-tuning problem. In my opinion, revisiting Wilson's Renormalization Group using the universal transition to chaos in nonlinear dynamics of generic flows should be a viable starting point.
Results any real physical experiments contain elements of the synthesis of the observed properties.
ReplyDeleteWe see increase of the proportion of artificially synthesized properties at with the increase induced energy. We measure the nature of the filter to a greater extent and we measure the nature of the analyzed signal to a lesser extent - in this case. The nature of the observation changes the result.
The proportion of artificially synthesized properties increases with the degree increase of rationalization of the experiment.
We can not identify the "natural" background statistics with the help of monotonous physical experiments. We must use experiments of a various nature to construct a spectrum of "natural" properties. The experimental conditions must contradict each other to some extent. We will be able to build "natural" statistics using a system of irrational experiments only.
"Natural" statistics are the result of irrational experiments only, because reality is irrational (contradictory).
Ervin,
ReplyDeleteI just told you why there is no "fine-tuning problem". What about my explanation did you not understand?
Sabine,
ReplyDeleteIt's quite obvious that we are not on the same page.
Let's agree to disagree.
Ervin,
ReplyDeleteThis is science, not opinion. "Agree to disagree" is not an option. I told you why you are wrong. That's all we can agree on.
Sabine,
ReplyDeleteThe dismissive tone of your reply is rude and uncalled for. You are obviously unwilling to listen to counterarguments and this attitude precludes any meaningful conversation.
Good bye.
Kevin,
ReplyDeleteInteresting to hear your thoughts. I am aware of your careful treatment of Cox's theorem and thorough reading of Jaynes' works. I agree with your remark.
Sabine,
It's important to emphasise that Kevin asked about an (epistemic) prior for an unknown parameter - a prior that describes our knowledge or ignorance about it. Such priors are ubiquitous in Bayesian statistics. Our prior isn't a physical mechanism by which the parameter was chosen and the parameter isn't determined by sampling from our prior; rather, the prior reflects our state of knowledge.
You argued in your reply that priors for parameters in fundamental physics are problematic because we observe a single n = 1 draw from the prior. We need to remember the meaning of our priors - they aren't distributions from which our Universe was sampled - and that priors can never be determined from experiment. This is true of every prior in a Bayesian analysis.
When I formulate a prior for Kevin's age, for example a flat distribution between 30 - 50 years with some tails, I am not suggesting that Kevin's age was determined by a random draw from such a distribution! I am describing my knowledge. Since there is only one actual age, of which I am incognizant, you could say, in a similar vein, there are only n = 1 possible samples with which to verify my prior. We quickly see that there are no conceptual differences in applying priors to someone's age and a parameter in fundamental physics, and no 'n = 1' problems that only appear in the latter.
Lastly let me clarify a detail about works on this topic in the literature. Log priors, p(log x) = const, are indeed often chosen. The motivation isn't that anyone believes them to be 'natural'; they are discussed in introductory textbooks and are completely unrelated to any notions of naturalness in particle physics. They are chosen because they represent our ignorance of the magnitude of an unknown scale parameter.
Ervin,
ReplyDeleteYou have not provided any counterargument. You have not provided any argument whatsoever.
Andrew,
ReplyDeleteAs I said, you can of course do a Bayesian analysis to infer the best-fit parameters of the theory. But this has nothing to do with naturalness. The "most likely" parameters that fit present observations are unnatural, which means that if you believe in naturalness they are supposedly unlikely. Best,
B.
Andrew,
ReplyDeleteForgot to say about the log priors. Even those, needless to say, require an assumption. You say people don't use them because they think they are "natural", but this is just picking on words. They believe they are a good starting point. It doesn't matter which way you put it, you have to chose them. You're the Bayesian, so now tell me how choosing a prior - a function in an infinite space - is any less of an assumption than choosing a constant. How do you want to do that? Introducing priors for priors?
As I have said elsewhere, you are trying to solve a problem that has no solution. If you want to find a mathematical theory that describes nature you always have to chose assumptions "just because" they explain what we see. These will be infinitely finetuned in the sense that they are one specific set of axioms among uncountably many. The whole idea of naturalness is a logical non-starter.
Best,
B.
Sabine,
ReplyDelete'As I said, you can of course do a Bayesian analysis to infer the best-fit parameters of the theory. But this has nothing to do with naturalness. The "most likely" parameters that fit present observations are unnatural, which means that if you believe in naturalness they are supposedly unlikely.'
That's not how any of this works.
'You're the Bayesian, so now tell me how choosing a prior - a function in an infinite space - is any less of an assumption than choosing a constant.'
I'll gladly try to explain. The prior for an unknown parameter reflects our state of knowledge. If we don't know the parameter, it would be contradictory to use a prior that selected a single permissible value i.e. a constant.
"Natural" properties is a properties of infinite phenomena.
ReplyDeleteYou use rational approximations of infinite phenomena.
You are trying to think in terms of infinite phenomena, but you are using finite tools in fact. You do not have tools with infinite properties. You must recognize the finite nature of your mathematical logic. You must evaluate the consequences of translating your knowledge system on rational rails.
You use postulates (starting conditions) instead of causes.
We can solve this task. We can get away from the postulates at the start of the model. We must solve the task for any conditions at the start. We must find a universal invariant at the level of the abstract form of the solution. We can build a universal theory. But we must solve a task with infinite conditions - we must solve a not rational or irrational task.
The solution of an irrational task is not a problem. The solution of the irrational task is possible. We must overcome of the Faith in the monopoly of rational methodology at this stage of the evolution of science. We will continue to use the rational methodology, it has specific effectiveness, but it will become part of the universal theory.
Andrew,
ReplyDelete"I'll gladly try to explain. The prior for an unknown parameter reflects our state of knowledge. If we don't know the parameter, it would be contradictory to use a prior that selected a single permissible value i.e. a constant."
No one ever uses a single value for anything because determining that would require infinite measurement precision. You are always assuming some prior on some space and assign special relevance to a certain choice of basis in that space etc. You have a log-distribution in one basis on that space, it's not a log-distribution in some other basis. Either way you put it, you make a choice, you add assumptions.
Having said that, I don't understand what you are arguing about to begin with. Are you just saying the standard model is natural because it's among the most likeliest fits? In which case we don't disagree to begin with.
Mathematics for any modern dynamical theories is finite by definition. Modern mathematics has no tools for describing phenomena with infinite properties.
ReplyDeleteYou are obliged to use boundaries - postulates - initial conditions. You do not have a universal invariant for any initial conditions.
I see the reason for the ban on solving the problem of an infinite in a fundamental error. I see the reason in the ban of contradictions at the level of rational methodology.
Sabine,
ReplyDeleteI mentioned constants because you expressly asked me about them. I don't understand your subsequent text.
'Having said that, I don't understand what you are arguing about to begin with. Are you just saying the standard model is natural because it's among the most likeliest fits? In which case we don't disagree to begin with.'
To begin with, I wanted to clarify a conceptual misunderstanding of epistemic probability in your writing, as already mentioned by Kevin, and explain the motivation for log priors, which you wrongly presumed were motivated by naturalness.
I'm certaintly not saying that and in fact politely disagree with almost everything non-trivial you've written on this topic.
Andrew,
ReplyDeleteClearly something is going wrong in this exchange because I can't figure out what you are even disagreeing with. Let me repeat that I have no problem whatsoever with using Bayesian inference to extract the best-fit parameters (at any energy).
Maybe you could start by explaining what you refer to by "naturalness problem". If you disagree with what I said, then please tell me what it is that you disagree with.
Since you say you don't understand my above text, let me rephrase this as follows. For all we presently know about the SM, the low-energy parameters sensitively depend on the UV parameters. That's not natural according to the present definition, but that's just a fact and not a problem. The relevant argument for why that is supposedly a problem comes from assuming that such a set of parameters is unlikely in a quantifiable way. My argument is simply that if you want to say this is "unlikely" you have to chose a probability distribution and there is no justifiction for that. It requires the exact choice that you didn't want. You could say that the probability distribution is itself unlikely.
For all I can tell, what you want to do by adding Bayesianism is that you say you had some prior for the parameters at high energies, log-normal or what have you. That prior expresses your lack of knowledge. Fine. Then you go and use the existing data to update that prior. Awesome. Upon which you'll end up with a distribution in the UV that's highly peaked around some values. That restates the above mentioned sensitivity. The parameters that are likely to fit present data in the IR correspond to a very focused distribution in the UV. Am I making sense so far?
But there is nothing problematic with this and the whole naturalness problem is about quantifying the supposed trouble of the SM. If you now want to claim that there is any problem here, you'll have to make a case that your initial prior should have been a good prior (in the sense of being close to the updated ones). That is the assumption I am saying is unjustified. As you say, the priors in this game express ignorance, not a property of the theory. You can start with whatever you want, but that starting guess has no fundamental relevance.
My previous comment explained why, if that is what you claim - that the log prior should have been close to what you get after taking into account data - this is just wrong. If this is not what you say, then I have nothing to disagree with, but then you're not saying the SM has a naturalness problem to begin with. So please clarify. Best,
B.
Bayes' formula allows us to rearrange position the cause and effect: we can calculate the probability some reason according to the known fact of the event.
ReplyDeleteThe Bayes formula is based on the reversibility of cause and effect. The Bayes formula uses the properties of rational logic. Rational logic is reversible.
But we know the difference between real logic and rational logic. Between the real effect and the real reason lies the space-time interval. The interval of real space-time obeys the second law of thermodynamics. The second law of thermodynamics describes an asymmetrical phenomenon. We can not use rational logic to describe the relationship between real cause and real effect.
If we can not use rational logic, then we must use a model of non-rational relations to describe the second law of thermodynamics.
Path to the creation of a universal theory is complex, but it lies through creation and use of non-rational mathematics.
Statistical mathematics filters probabilistic fenomena. A universal model must filter deterministic fenomena, for exaple, - among others. We can not build a universal model on the basis of laws of one (statistical) type.
ReplyDeleteThe conclusion of this thread is the following equation: naturalness=crisis
ReplyDeleteMy view on this subject is that naturalness is an irresponsible and disrespectful (to human intellect) way to accept a theory by simultaneously discard of classical logical consisteny for the sake of aesthetics (e.g. prestige). Unfortunately, Max Planck quote still hunts science evolution: "Science evolves one funeral at a time".
All problems in Physics either major or not can be solved using fundamental classical logical consistency something that is discarded since long (last 100 years at least).
What I mean about logical consistency.
An example: Casimir effect presupposes uncharged and conductive plates where the effect is justified over the quantum vacuum fluctuations.
What argument decides to follow such notion? Is there a counterargument? I would say "yes, there is and it can be demonstrated and proved classically without referring to harmonic quantum oscillator or generally to quantum vacuum fluctuations".
There are countless such examples.
"..now want to quantify how sensitively a theory at low energy depends on the choice of parameters at high energies, you first have to define the probability for making such choices. This means you need a probability distribution on theory space."
ReplyDeleteHi Sabine,
is such a statistical approach to quantum field theories already established or you are proposing the invention of new mathematics?
All the best for your lecture at PI...
I'm afraid that your understanding of Bayesian model selection is somewhat garbled. You first described inference of parameters by updating a prior with data resulting in a posterior. You then guessed that one evaluates a model by comparing the posterior and prior. Under this guess, a model would be considered unnatural or implausible if the posterior and prior were markedly different by eye.
ReplyDeleteThis is indeed complete baloney. It shouldn't be particularly surprisingly that one in fact judges a model by directly calculating its plausibility relative to a different model. In practice this amounts to calculating Bayes factors,
p(data | first model) / p(data | second model)
The integrand and denominator are individually refered to as evidences. This is treated in introductory material.
If one calculates a Bayes factor for the SM with Planck quadratic corrections versus a supersymmetric model, you find that the latter is favoured by a colossal factor by measurements of the weak scale. This conclusion holds so long as priors are constructed honestly, i.e. without reference to measurements of the weak scale, and permit sparticles at scales much less than the Planck scale. This is naturalness reborn in the language of Bayesian statistics - models traditionally regarded as unnatural are indeed relatively implausible in a coherent logical framework.
I will conclude by noting that the spirit of Bayesian statistics is that one updates beliefs in light of new information. I hope that in light of my taking time to correct elementary misapprehensions in your understanding of Bayesian statistics and naturalness, you might update your opinion.
Dear Sabine,
ReplyDelete"But there is nothing problematic with this and the whole naturalness problem is about quantifying the supposed trouble of the SM. If you now want to claim that there is any problem here, you'll have to make a case that your initial prior should have been a good prior (in the sense of being close to the updated ones)."
the problem is then is not with the choice of the prior, the problem is with the structure of your theory model that requires large cancellations between the model parameters in order to explain the observable effects. The statistical analysis needed to define confidence regions on the parameters is then inhibited by large derivatives with respect to the model parameters that reduce predictive power of an "unnatural" model. That's why an "unnatural" model is "ugly" -- it may describe nature successfully, but it is inferior in its predictive power because its probability distribution is poorly behaving.
Thus effective phenomenological models built to describe complex systems often impose a special constraint to discourage large cancellations between the model's parameters. As nature is presumably described well by several dual theories, if one of these descriptions has no cancellations, it would be preferred all other factors being equal.
I'm not a fan of the multiverse. It's flaws are obvious to non-physicists, like me. However, I want to point out that comments like this: "This is very similar to moving around in the landscape of the multiverse. Indeed, it is probably not a coincidence that both ideas became popular around the same time, in the mid 1990s." aren't very logical either.
ReplyDeleteIt's kind of the opposite to the "appeal to authority" logical fallacy.
"That the perturbative expansion doesn't converge, otoh, is an actual problem. So is, eg, Haag's theorem."
ReplyDeleteThese issues are well understood in 1+1d toy models. If you have locally correct dynamics you can patch together a global theory, but it is represented in a different Hilbert space. That's (one of the reasons) why the expansion doesn't converge. This is just math. The physics problem is to define the locally correct dynamics (not just a perturbation approximation to an ill-defined theory).
This is science, not opinion. "Agree to disagree" is not an option. I told you why you are wrong. That's all we can agree on.
ReplyDeleteThere is a story that, after a talk, a young researcher was destroyed by a comment from a senior scientist. Whether the latter was correct or not is beside the point. The junior scientist tried to avoid more embarrassment by uttering a line which is often used by more senior people to get the chairman to move on to the next question: "Yes, we should discuss that". The senior scientist: "I thought we just did". :-(
“naturalness is a well-defined concept.”
ReplyDeleteMy maths professor (yes, she even has her own (German) Wikipedia page) used to say that "well defined" is not well defined. :-)
For the record, let me say that I completely agree with you in your criticism of the naturalist argument in particle physics.
ReplyDeleteNaturalism claims "dimensionless parameters should be of order 1". This is opposite from fine-tuning arguments which, in contrast, claim that dimensionless parameters of order 1 (such as the ratio of the critical cosmological density to the observed density) need an explanation. (Whether the argument is bogus, misleading, or wrong---that is, whether someone has overlooked something which makes the ratio "natural"---or whether there is really some "additional" explanation needed (i.e. something which goes beyond the science in which the observation was made) is a different question. I would argue that in the case of the flatness problem, the traditional formulation is misleading and the answer is completely within the context of the science in which the problem was posed while in the case of fine-tuning for life some explanation is needed. Again, the puzzle is not "why this number and not some other" since all are a priori equally likely, but rather "why this number, which we knew was interesting even before we had any data".)
Andrew,
ReplyDeleteI asked you several time to state your argument. You didn't but instead left me guessing. Then you complain that my guess is "baloney." That's a transparently stupid mode of argumentation, though I will give it some creativity points since I haven't encountered it often.
For all I can tell the measure you suggest just rephrases the usual sensitivity of the IR on the UV parameters by weighting different models against each other. But no one doubts that, say, susy models are less sensitive to the UV than the vanilla SM, and they are in that sense more predictive. Or maybe one could say more forgiving. The question is, why do you think this means there is anything wrong with the SM? To state the obvious, renaming "natural" into "honest" doesn't make much of a difference. As everyone else who is in favor of naturalness, you have an idea how a fundamental theory "should" be and you believe that this bears relevance.
Yes, I appreciate your comments and I am updating my information. I get the impression though you aren't actually thinking about what I say, otherwise you wouldn't attempt what's impossible just by rather basic logical reasoning. Best,
B.
Liralen,
ReplyDeleteThis comment is not a "logical conclusion," it's a historic side remark. I have no idea what you thinks is wrong with pointing out that two similar ideas became popular around the same time.
TransparencyCNP,
ReplyDeleteTo state the obvious, we don't live in 1+1 dimensions. But I'm glad that somewhere someone is thinking about it ;) More seriously, every once in a while I do see a paper about these topics, so I'm certainly not saying no one works on that. Clearly though it doesn't get as much attention as naturalness "problems".
Joe,
ReplyDeleteThat's how the usual measures work. Except that people don't usually specify the probability distribution. They implicitly assume it's an almost uniform distribution over an interval of length of order 1. Ie, put in 1, get out 1. No, I am not saying one should develop this approach better because it's futile. You are just replacing one guess (what's the parameter) by another guess (what's the probability distribution).
Sabine,
ReplyDeleteI think Andrew's (and Kevin's) argument is this:
Obviously, a claim that there is a naturalness problem based on a posterior / prior comparison for one model - the SM - would be baloney. No matter what the prior. But there are 'Bayesian' ways of choosing priors that would not be appeals to naturalness in disguise, and model comparison - SM versus whatever - based on such a prior would be justifiable. So if it's been done in that principled way* the preference for one model over the other would not involve an appeal to naturalness or what anyone thinks a fundamental theory "should" look like.
* You say "There are actually a few papers in the literature on Bayesian naturalness, typically assuming a uniform log prior, presumably because that is 'natural'", suggesting that it has not been.
Paul,
ReplyDeleteI was guessing they wanted to compare different sets of parameters to find out what is a preferred set of parameters. That's a reasonable thing to do.
Comparing different models in this fashion is, in contrast to what you say, not justifiable because this doesn't quantify the probability of the model's assumptions themselves.
Let me give you a simple example for how this goes wrong. The reason that supersymmetric models come out ahead in the model-comparison that Andrew sketches above is that susy models allow a larger range of UV parameters to be compatible with the IR data than models which are not-supersymmetric. The reason for that is of course that the symmetry enforces the cancellation of certain contributions to certain IR values, that being an echo of 't Hooft's original motivation to introduce the notion of 'technical naturalness' to begin with.
But if you wanted to actually compare both models you should take into account that the supersymmetric model has the additional assumption. Which is, well, supersymmetry. And what's the probability for that? Well, with any "honest" prior (in Andrews's words) the probability for this additional assumption is zero, because there are infinitely many assumptions you could have made. And even after you've assumed susy, there are more assumptions about having to break this symmetry and having to add R-symmetry in order to avoid conflict with data. If you'd do a Bayesian analysis between different models (as opposed to different parameters in an EFT expansion) you should take the additional assumptions into account. If you don't, then supersymmetry will come out ahead, rather unsurprisingly.
There is a different way to see what I am saying which is to simply ask yourself why no one who makes SM predictions uses a supersymmetric model instead. Well, that's because to correctly calculate what we observe you don't need susy. It's not supported by data. If you've managed to construe a measure which says susy is actually preferred by the data, you have screwed yourself over in your model assessment.
(I don't understand your *-remark.)
Best,
B.
Sabine,
ReplyDeleteThanks - that clarifies matters.
My (now redundant) *-remark was just noting that what you said suggested that even the (uniform log) prior choice hadn't been made on principled, 'Bayesian' grounds.
Sabine,
ReplyDeleteyour reply to Paul is enlightening. I would use the Bayesian analysis in the other way around, though. I would not compare the Bayesian priors/probabilities to decide which model is ultimately "true" -- which only an entity with an infinite amount of information ("a deity") can answer. Such question is in the realm of metaphysics.
Instead, let us ask which model is consistent with the existing data and gives robust testable (non-trivial falsifiable) predictions for future measurements. Which model can systematically guide new experiments in order to incorporate new information into our shared picture of the world?
Say, we replace the model-building of the 20th century, driven by human speculations, by a neural net trained to make predictions for future colliders based on the existing data. The neural net learns by updating the probability distribution in the parametric space via recursively applying the Bayes theorem:
do when [new data]
{
p(theory posterior) = const*p(new data | theory)*p(theory prior).
p(theory prior) = p(theory posterior).
}
The probability reflects the knowledge state of the neural net, not the ultimate likelihood of the neural net model. Which models will facilitate relevant, stable, and converging learning process? They should be comprehensive and self-consistent, and it is preferable that they have a regularly behaving likelihood p(data|theory). This can be used to define a "natural" model. An 'unnatural' likelihood that is not smooth, e.g., that has 'landscape' features with haphazard variations of probability, or that requires large cancellations between unrelated parameters to describe the data, generally inhibits the learning process. Large unexplained variations of the likelihood disrupt convergence of the recursive learning. Unnatural models are less predictive, because the global structure of the landscape cannot be easily inferred from the local structure.
This gives a very intuitive definition of naturalness. The unnatural features of SM are therefore not desirable -- they may be what they are, but they limit progress in systematically expanding our knowledge of the microworld. In the absence of a confirmed natural BSM theory, a recourse to random testing of alternatives to SM mentioned at the end of your blog may be a reasonable strategy.
I did not leave you guessing - the rudiments of Bayesian statistics exist in textbooks and undergraduate classrooms. Before embarking on writing blogs, books and a lecture series criticising the foundations of naturalness, you should have familiarised yourself with it.
ReplyDelete'The question is, why do you think this means there is anything wrong with the SM?'
We have concluded that theories traditionally regarded as natural are, in light of data, vastly more plausible than the Standard Model (i.e., they are favored by enormous Bayes factors). This means that we should consider the SM relatively implausible; that is what is wrong with it.
'To state the obvious, renaming "natural" into "honest" doesn't make much of a difference.'
I have not renamed natural into honest. I introduced the word honest when remarking that we must pick priors that honestly reflect our state of knowledge prior to seeing experimental data. This was unconnected to ideas about naturalness in high-energy physics.
Your responses to Paul Hayes revealed further confusion. You argue that the prior probability of any model, including a supersymmetric one, is zero. As mentioned already, we, however, consider relative plausibility. Relative plausibility is updated by the aforementioned Bayes factor
Posterior odds = Bayes factor * prior odds
where prior odds is a ratio of priors for two models. The Bayes factor updates our prior beliefs with data. It is conventional to calculate and report only a Bayes factor and permit a reader to supply their prior odds. It remains the case, then, that data increase the plausibility of traditionally natural models relative to the SM. You could, nevertheless, insist that you a priori disfavoured supersymmetric models by a colossal factor that overpowers the Bayes factor. That is your prerogative. I don't think it is justified by our state of knowledge, though, and in any case wish to focus upon the factor containing experimental data - the Bayes factor. This is, once again, introductory material on this topic.
'If you've managed to construe a measure which says susy is actually preferred by the data, you have screwed yourself over in your model assessment.'
Flattering though it is to be credited with construing the measure (the Bayes factor etc), I must clarify that it was pioneered by Harold Jeffreys in the 1930s, and grown in popularity especially in the last 30 years. May I earnestly beg that before lecturing or writing any more criticisms of the logical foundations of naturalness, you learn the relevant basic material.
Would not truth always require mathematical rigor as a point of departure, as anything less just won’t get you there. Besides, the failing (of string theory, dark “matter,” etc.) is not lack of rigor but unexperiential physicality being (arrogantly) imposed on the world to serve the overly complex maths (so of course the world won).
ReplyDeleteNow, in order to light the darks and resolve residual mysteries besides, it’s good to wildly speculate as to ways that might explain everything. It’d be better to derive truest holistic model expecting to find deficiencies rather than bending to provisional--considering said outstanding dark mysteries--measures of a world so imperfectly understood.
That the world is ugly is not settled science. So best that simplicity yet find beauty.
I'm trained as a mathematician and I find physics papers hard to read - which is strange because mathematics is said to the language of physics. I like the Anthropic principle because it's easy to understand and obviously true. Whereas with naturalness, I'm still struggling to understand what this means.
ReplyDeleteAndrew,
ReplyDeleteYou are now being outright condescending even though you either do not understand or willfully ignore what I say.
First, needless to say I didn't say you left me guessing about what Bayesian inference is, but about what you believe it has to say about naturalness. You are still being vague about it. You now say that the Bayes factor (assuming the one you referred to earlier) tells you which model is more "plausible". Please explain what makes such a model more plausible. Do you just use the word "plausible" to replace "probably a correct description of observations"?
Second, the foundations of naturalness have nothing to do with Bayes, but that's only tangentially relevant.
Third, you seem to have not understood my reply to Paul, so let me iterate this once again. Yes, the prior for any model is zero, but that doesn't prevent you from comparing different models provided their priors are "equally zero" in the sense of factoring out and I never said so, so stop putting words into my mouth. You can indeed do that if you have two models that rest on the same assumptions. But you cannot do that if you are comparing models for which this isn't the case.
Forget about Bayes for a moment, Occam already tells you that a model with additional assumptions that are unnecessary to describe observations should be strongly disfavored.
"You could, nevertheless, insist that you a priori disfavoured supersymmetric models by a colossal factor that overpowers the Bayes factor. That is your prerogative. I don't think it is justified by our state of knowledge"
The reason why eg, supersymmetric models come out ahead in such an analysis is that the parameter cancellations are enforced by the symmetry requirement. If you do a Bayesian analysis comparing the predictive power of susy vs non-susy for the IR couplings, ignoring the symmetry assumption, this tells you nothing about whether susy actually describes nature, it merely tells you that it's a more rigid framework. We already knew that. You haven't told us anything new. More importantly, this bears no relevance for the question whether supersymmetry is or isn't correct.
If you have any other "natural" theories in mind, you'd have to give me examples.
"Flattering though it is to be credited with construing the measure (the Bayes factor etc), I must clarify that it was pioneered by Harold Jeffreys in the 1930s, and grown in popularity especially in the last 30 years. May I earnestly beg that before lecturing or writing any more criticisms of the logical foundations of naturalness, you learn the relevant basic material."
Bayes would turn over in his grave if he knew what nonsense you are construing out of his theorem. You seem to be seriously saying that a model with additional assumptions that are entirely superfluous to explain currently available data is preferred by a Bayesian analysis. Look, just because you can calculate something doesn't mean it's relevant. As they say, garbage in, garbage out.
Best,
B.
Pavel,
ReplyDeleteThis is an interesting comment, which I am guessing probably captures formally what many people in the community mean informally with naturalness, or why they dislike unnaturalness. As you say, this tells us nothing about whether natural theories are more likely to be correct but it quantifies the reason why unnatural theories are undesirable - they prevent us from making progress because learning (about high energies) becomes more difficult.
Best,
B.
Alas we must agree to disagree. Please accept my apologies if my manner was rude. I share your passion for this topic (though not your views). Best wishes.
ReplyDeletePavel,
ReplyDeleteI would not compare the Bayesian priors/probabilities to decide which model is ultimately "true" -- which only an entity with an infinite amount of information ("a deity") can answer. Such question is in the realm of metaphysics.
No Bayesian would.
The Standard Model is "unnatural" if one starts with uniform log priors.
ReplyDeleteThe Standard Model may be unnatural if one starts with the larger class of priors upon which neural networks work well. So what?
Has anyone tried feeding a neural net with collider data to see what predictions it makes? My guess is that neural nets, fed with data of collisions of energy 0-E will make excellent predictions of what is considered to be the background in collisions in the energy range E - E+ΔE. Predictions of new physics - not so much.
Observation 1: From the discussion here, it seems that a theory of low energy physics that is completely decoupled from every single detail of high energy physics is Bayesianly speaking, the most plausible theory of all.
ReplyDeleteObservation 2:
...by a neural net trained to make predictions for future colliders based on the existing data. The neural net learns by updating the probability distribution in the parametric space via recursively applying the Bayes theorem:
We already have such a neural net. E.g., see Gordon Kane and his successive predictions for BSM physics, updated each time colliders fail to find supersymmetry.
Arun,
ReplyDeleteThere's a lot of data analysis with neural networks, and it's not even a recent thing. Not only the LHC data is analyzed that way, but I also know that it's quite common for many small-scale colliders.
Naturalism claims "dimensionless parameters should be of order 1".
ReplyDeleteNatural "dimensionless parameters" are not rational parameters. Any rational relationship has dimensions. Irrational relations are dimensionless only.
"But note that changing the parameters of a theory is not a physical process. The parameters are whatever they are"
ReplyDeleteThanks for declaring this so clearly. This is true when you follow what smolin and unger calls the newtonian schema. If we also accept this as beeing right the most of what you say here are easy to agree with.
I however think this thinking is exactly what leads to fallacious reasoning om "non physical landscspes" or "non physical" probability spaces.
This is where an evolution abstraction than learns has something to add. Here the random walk in theory space IS a physical process - but not one that allows to be captured by a timeless metalaw.
So i think the core of this discussion is - can we find a way to make senes of - and make somewhat rigorous - the idea to unify dynamics as per law with the idea of evolving law?
If you say no here, then i at least understand most of your critique.
/Fredrik
I was reading about the new axion mass limits from nEDM measurements in your reposting today on Twitter. So I went and looked at the nEDM background story. I had forgotten that the strong CP problem seems to be classified as a 'problem' because the implied value of theta (quantifying the coupling of the strong CP violating term in QCD) from nEDM measurements, e.g., is quite small whereas 'naturalness' implies it should be about 1. Since (if I'm not mistaken) the axion was originally created to 'explain this deviancy' from naturalness, and given your argument in this post, would you say the entire axion search industry is prefaced on a misunderstanding?
ReplyDeleteI think the motivations to believe in the existence of this axion are questionable.
ReplyDeleteIn my humble understanding, a good physics theory should explain existing observations and make accurate predictions of future ones. Thus, to me the basis of deciding which theory is better for the purpose of progress in physics should be in this explanatory/predictive power.
ReplyDeleteThe idea of comparing theories within a landscape of theories to decide that one theory is more or less *plausible* than another, makes no sense to me -- I dare say the concept of *plausibility* of a theory is a useless one.
I think it would be much more productive to develop a metric for explanatory/predictive power for theories that the fog of "naturalness".
Strangely appropriate:
ReplyDeletehttp://www.smbc-comics.com/comic/how
Don't forget the mouseover. :-)
The criterion of truth is strictness. Strictness as a phenomenon is primary. A rigorous model is equivalent to the original. We will not compare different models in terms of greater or less naturalness in the presence of a rigorous model.
ReplyDeleteA strict model has unique capabilities. The behavior of the strict model is similar to the behavior of the original. We will observe the behavior of the model and we will see the behavior of the original in this.
We can construct a universal theory and localize part of the theory with a rigorous description of the model. We localize the uncertainty in another part of the theory.
You are exploiting the existing paradigm. We can examine the history of the development of this paradigm and build a uni-versal theory on the basis of this work. Scientists use the paradigm, but they did not investigate the causes of the paradigm in its present form.
The universal model will be non-rational in the indefinite part and irrational in the strict part.
We will build new rational models in the development of the universal model. Rational models bring an applied result. But we will see a fee for this result in another part of the universal model. The universal model will be based on the balance of benefit and harm.
I dont see insoluble problems on the way to creating a universal theory. I see the main obstacle in the monopoly of the rational paradigm to the truth. We will maintain a rational methodology also, but we will deprive it of the right to monopoly. Belief in this monopoly has a religious character. We can not break this barrier with the help of scientific arguments only. I see the need for using external resources - as well.
Boerhaave said: "Simplex sigillum veri"; "Simplicity is the sign of truth". You can only understand this truth when you have actually "seen" (experienced or gone through) it.
ReplyDeleteThen, in my view, automatically beauty shows itself.
Not as over-simplification or naievity which are equally natural.
Populating a distribution with personal insights sounds naive to me.
By the way, thinking and beleiving are different animals although not many persons refer to these differences
and use the words indiscriminately.
I very much enjoyed this conversation.
Frans de Wolff
We should note that the Planck scale is the "natural" scale at which we expect something significant to happen quantum-gravity-wise. The whole naturalness discussion above suggests that we can't expect Nature to adhere to the "natural" scale.
ReplyDeleteI’m an engineer and love ‘naturalness’ in all its manifestations. Arguing about naturalness is likely to be an exceedingly difficult nut to crack. People like what they like and that’s that. The only way a change in thinking could possibly be effected would be overwhelming success with non naturalistic thinking. Maybe that’ll occur, maybe not. For my 2c I dearly hope not, but, what happens down the road, happens. Even then there’ll be stubborn holdouts like me. I love so much of the older engineering - e.g. clipper ships, the most beautiful machines ever built, steam locomotives, large reciprocating engine powered aircraft... it goes on and on. They’ve all been eclipsed, Nothing to do but forge on and see where things go. Changing people’s attitudes is exceedingly difficult. Indeed, politics (99.9999...% bs, at least) is a continuing attempt to do just that.
ReplyDeleteQuote From Screams for Explanation: Finetuning and Naturalness in the Foundations of Physics, conclusions:
ReplyDeleteI will not deny that I too feel that finetuning is ugly and natural theories are more beautiful. I do not, however, see any reason for why this perception should be relevant for the discovery of more fundamental laws of nature.
Unquote.
From my point of view this is not the point. Of course you do not see it. You only see it when you have understood it. The seeing is an event after the relisation of what has been presented.
Or in Dutch: je ziet het pas als je het doorhebt.
Or, an attempt to translate: you only see it when you really got it.
The truth is hidden until recovery. It is certainly not self-explanetory. That is the next phase.
Best regards, Frans
Frans Hendrik. I have a good rational education. I could deny the existence of the loop phenomenon some time ago, but I observe the loop phenomenon in my model today. I built the solution using unique technologies, but I used the key selection method to verify the models. I jumped into the loop of a nontrivial solution.
ReplyDeleteI see the answers to many questions in my model, but I do not know about the existence of these answers a priori.
I can not predict the existence of a loop phenomenon using rational methodology. A rational methodology does not have the tools to investigate phenomena of this type. Phenomena of this type remain invisible to rational methodology.
Non-rational relationships (as the relationship between independent observers) can form amazing phenomena, such as a loop phenomenon.
The loop as a phenomenon is a source of information because of the non-rational relationships between the elements of the model. I observe this phenomenon as a fact.
Hi Sabine,
ReplyDeleteI am late to this party, but I wanted to comment on a point of disagreement you seemed to have with Andrew. You seemed to say that Bayes factors (which "automatically" include penalties for fine-tuning, if done properly) are not a valid basis for model comparison, because they neglect probability penalties associated with the assumptions required to construct a theory in the first place. Actually I partially agree with you here; those factors *are* vitally important! I would argue, however, that they should be "packaged" into the prior odds, not the Bayes factor. Typically one leaves the prior odds out of a model comparison, because as you say, assessing them is an extremely subjective matter. But it is a trivial matter for the reader to multiply their own personal prior odds back in, to obtain their own subjective posterior odds. So to summarize: naturalness does indeed arise as an automatic penalty factor in Bayesian model comparison, but it can be "out-competed" by sufficiently implausible model assumptions.
I have to ask a question about the subject of the discussion. We can discuss physics as a model of material reality, we can discuss the reality in the material and non-material parts, and we can discuss some mathematics with the localization of some part of the total entropy - outside of the connection with reality.
ReplyDeleteThe answer is discouraging. We do not have tools for localizing the model in some area. An experiment is necessary, but it is not proof. A particle does not prohibit a wave; a wave does not prohibit a particle, for example. A particle's hybrid and a wave do not have the properties of a particle or wave separately.
We do not have fundamental physics today, but we can do science, we can build a foundation.
Ben,
ReplyDeleteYou write:
"I would argue, however, that they should be "packaged" into the prior odds, not the Bayes factor. Typically one leaves the prior odds out of a model comparison, because as you say, assessing them is an extremely subjective matter."
Yes, you can do that. But in this case you then cannot claim that your Bayesian measure of naturalness tells you anything about the probability of the theory to begin with (or relative probability, respectively). It's just another measure of naturalness that fails to identify what the problem is supposed to be.
Sabine,
ReplyDelete"Yes, you can do that. But in this case you then cannot claim that your Bayesian measure of naturalness tells you anything about the probability of the theory to begin with (or relative probability, respectively)."
Why not? It arises automatically as a term in the odds calculation. It isn't the *only* term, but it would nevertheless be wrong to ignore it. As for relative probability, well that is exact what the posterior distributions over the parameter space are, and things work especially well here because probabilities related to model assumptions cancel out when you compare points within the one model.
"It's just another measure of naturalness that fails to identify what the problem is supposed to be."
I don't follow you here. It identifies precisely what the problem is; it is about the difficulty of reproducing observed data from a model, about how apriori unlikely certain precise parameter combinations are.
Overall, I see no way of rejecting naturalness arguments unless you also want to reject Bayesian statistics entirely (I agree that there are too many ad-hoc measures floating around, but Bayesian probability tells you what you should be doing, and it is pretty similar to the "guesses" that theorists make). But if you reject Bayesian statistics then how should we compare models instead? We can compute p-values, but those suffer even worse from the criticism you raise (they take zero account of the plausibility of model assumptions, and there isn't even any way to include these if you wanted to).
Ben,
ReplyDeleteThe model assumptions are not the same, that's the whole point. The very reason that (eg) supersymmetry is more restrictive (or more "natural" or more predictive, if you wish) is that it has additional assumptions. If you don't take into account the prior for the additional assumption your relative probability is meaningless.
" it is about the difficulty of reproducing observed data from a model,"
No, it's not. Supersymmetric models reproduce the Higgs mass equally badly as the standard model, which is not at all.
Sabine,
ReplyDelete"The model assumptions are not the same, that's the whole point. The very reason that (eg) supersymmetry is more restrictive (or more "natural" or more predictive, if you wish) is that it has additional assumptions. If you don't take into account the prior for the additional assumption your relative probability is meaningless."
A Bayes factor isn't a relative probability. It is *part* of a relative probability, an odds ratio. It isn't meaningless though, it tells you how much the data is pushing you one way or the other towards either model. Which is useful to know, because then you know how strongly you would have to apriori prefer one model or the other in order to counteract the data. So yes, SUSY has more restrictive assumptions, and should absolutely be penalised for this, but don't you want to know how larger a penalty is required to overcome the data? If the Bayes factor is 1:10^4, say, then you would require a prior penalty of 1/10^4 against the SUSY model to still prefer the Standard Model. Individuals can make up their own mind whether that is enough, but it nevertheless non-trivial information.
"No, it's not. Supersymmetric models reproduce the Higgs mass equally badly as the standard model, which is not at all."
I don't follow you here. One can obviously get the right Higgs mass in both SUSY models and the Standard Model. The issue is about how much tuning is required to do it in each case, and what penalty this logically implies.
Ben,
ReplyDeleteIndeed, if you don't take into account the priors for the model assumptions it's not the relative probability of the models.
"Which is useful to know, because then you know how strongly you would have to apriori prefer one model or the other in order to counteract the data."
Please tell me what's the use of knowing this.
"I don't follow you here. One can obviously get the right Higgs mass in both SUSY models and the Standard Model. The issue is about how much tuning is required to do it in each case, and what penalty this logically implies."
You were talking about what's observable. I am telling you that you cannot calculate the supposedly fine-tuned observable either way, so what is the problem with it? You measure it and that's that.
Sabine,
ReplyDelete"Please tell me what's the use of knowing this."
Well most theory people are going to have some feelings about prior probabilities of various models, subjective though they may be. It seems worthwhile to me to at least provide them with this less subjective piece of information so that they can better prioritize their activities towards the most promising models. Likewise experimentalists have to spend a lot of time and money designing experiments to look for this or that new physics, so it seems again worthwhile to try and point them in the direction of the models with the most promise. If we could study every model and build every experiment and search for everything at once then indeed we could forget about fine-tuning etc, but in a world of limited resources we need to prioritize.
"You were talking about what's observable. I am telling you that you cannot calculate the supposedly fine-tuned observable either way, so what is the problem with it? You measure it and that's that."
But imagine you *did* have a theory that was already sufficiently constrained by other measurements that you could predict the Higgs mass correctly. Wouldn't you prefer that theory? There is a continuum between predicting the Higgs mass exactly, and being able to tune a theory to any Higgs mass, and the issue of naturalness is precisely about where along this continuum a theory lies.
Ben,
ReplyDelete"Well most theory people are going to have some feelings about prior probabilities of various models, subjective though they may be. It seems worthwhile to me to at least provide them with this less subjective piece of information so that they can better prioritize their activities towards the most promising models. Likewise experimentalists have to spend a lot of time and money designing experiments to look for this or that new physics, so it seems again worthwhile to try and point them in the direction of the models with the most promise. If we could study every model and build every experiment and search for everything at once then indeed we could forget about fine-tuning etc, but in a world of limited resources we need to prioritize."
Well, at least we have reached the origin of our disagreement. No, scientists should not use their "feelings" and subjective assessments to decide what is and isn't a promising theory. That's exactly what has gone wrong with all the naturalness talk that resulted in tens of thousands of wrong predictions.
Naturalness is a subjective criterion. It's not a scientific criterion. You can call it metaphysical or maybe philosophical. I call it aesthetic because that's where I think it comes from historically. Either which way you turn it, it's not a scientific criterion and scientists should not use it. Certainly not to decide which experiments to commission. Wrapping it up in Bayesian inference doesn't change anything about this.
What these calculations do is merely covering up subjectivity with lots of math. The cynic may say theorists do this deliberately to convince taxpayers to pour money into experiments that will keep otherwise useless particle physicists in their jobs. I try to be not quite as cynical and say that they don't understand what they are doing themselves.
Regarding the Higgs mass. Yes, I would prefer a theory that predicts the Higgs mass, but only if this theory is preferable relative to the standard model as qua its full formulation. It's easy enough to come up with theories that allow you to "calculate" the mass of the Higgs, eg by postulating some equation the solution of which is the mass of the Higgs. That doesn't count as "explanation" and you can quantify this in various ways.
But that's a hypothetical case. To stick with the case at hand, supersymmetry doesn't explain anything observable the standard model doesn't also explain. If you construe up some quantifier that says it's preferable, that's a scientifically meaningless measure.
Ben,
ReplyDelete"using Bayesian statistics explicitly acknowledges the subjective elements of hypothesis testing"
If you mean by "subjective elements" the priors for the model assumptions, then you are disagreeing with yourself, for we seem to have agreed that you cannot take those into account. No one has a clue what the prior is for "supersymmetry" hence all other relative calculations that you may be able to get published in a journal these days are utterly meaningless.
"What magical objective criteria do you suggest?"
For starters I am telling you what scientists should not do. That's using their "feelings" to decide what to spend taxpayers money on. Once you manage to understand this we can talk about what else to do. Though I don't know why that is a discussion you want to have with me; you should have it with your colleagues.
"So you just don't think anyone should search for new physics at all? "
I never said anything like that and I am not interested in dealing with your attempts at building a straw man. I have said criteria from naturalness are not scientific, regardless of whether you use Bayesian assessment or any other measure of finetuning. They should not be used, especially not since we have known since at least two decades that they are wrong and do not work. Why don't you try to quantify the probability for the hypothesis "naturalness is correct"?
"I challenge you to come up with a self-consistent methodology that *isn't* equivalent to Bayesian decision theory."
I have no problem with Bayesian inference. I have a problem with people who declare the conclusions mean something that the analysis doesn't support. I would prefer to quantify simplicity by means of calculational complexity. Be that as it may, you continue to miss my point. The point is that the standard model works perfectly fine to explain our observations and supersymmetry doesn't improve anything about it.
Sabine,
ReplyDelete"If you mean by "subjective elements" the priors for the model assumptions, then you are disagreeing with yourself, for we seem to have agreed that you cannot take those into account. No one has a clue what the prior is for "supersymmetry" hence all other relative calculations that you may be able to get published in a journal these days are utterly meaningless."
There is no "the" prior. Priors are subjective probability assessments. They are therefore different for everyone depending on their background knowledge and assumptions, and it doesn't make sense to say that no-one knows what they are. You have them whether you like it or not. And you seem to have misunderstood me here anyway. Indeed, the prior for the model assumptions is subjective. This is the factor that you said is neglected by considering only the Bayes factor (which is a common practice across many scientific fields, by the way). However, the naturalness component that contributes to the Bayes factor is *much* less subjective. That is, naturalness is not the subjective part! If you accept Bayesian probability theory, which you say you have no problem with, then it is inescapable; you *must* take naturalness/fine-tuning into account. A Bayesian analysis which neglects these effects will simply be wrong.
"For starters I am telling you what scientists should not do. That's using their "feelings" to decide what to spend taxpayers money on..."
As I explain above, the naturalness part is *not* about feelings. It is about the mathematical structure of a theory and what Bayesian probability says about that. Priors, the subjective part, can modulate the conclusions to some degree, but the naturalness contribution never goes away. And ok, you are saying what people *shouldn't* do, but that isn't very helpful unless you have a better idea (or could at the very least formulate what the problem is in a more concrete fashion).
"I never said anything like that and I am not interested in dealing with your attempts at building a straw man. I have said criteria from naturalness are not scientific, regardless of whether you use Bayesian assessment or any other measure of finetuning...?"
My apologies if I am straw-manning you, but I am having a hard time understanding exactly what it is that you want to occur here. As for your claim that naturalness isn't scientific, this directly conflicts with your later claim to have no problem with Bayesian inference. If you think naturalness is unscientific then you have no choice but to also say that Bayesian inference is unscientific, because there is no escaping the fact that naturalness automatically appears as a relevant consideration in Bayesian inference.
"I have no problem with Bayesian inference. I have a problem with people who declare the conclusions mean something that the analysis doesn't support. I would prefer to quantify simplicity by means of calculational complexity."
ReplyDeleteBut the analysis does support the notion of naturalness. There have been various papers describing this. I am all for attempting to explicitly quantify the simplicity of other elements of theory structure --good luck with that and let me know if you have a good idea-- but it would just be another component of the probability assessment to consider. The fine-tuning/naturalness factor would not disappear.
"Be that as it may, you continue to miss my point. The point is that the standard model works perfectly fine to explain our observations and supersymmetry doesn't improve anything about it."
Well, no, it doesn't. It explains the electroweak sector well enough, but it doesn't explain, say, dark matter (amongst a host of other things). SUSY can do this, though of course there are plenty of other candidates. But even if the Standard Model *did* explain everything, I'm sure we would still be interested in what other physics might be out there, and naturalness would remain a perfectly valid criterion for helping to look for it.
Ben,
ReplyDeleteWell, first I agree of course that the standard model doesn't explain dark matter, but I don't see how this is relevant.
Second, I get the impression we are talking past each other. Yes, the standard model is unnatural. Yes, the Higgs mass is finetuned. I am not saying anything to the contrary - these are just properties of the model. I am saying that this is not a problem in need of a solution.
If you want to claim that "everything is subjective anyway" then I just don't know why you are calculating anything to begin with.
Yes, I know it's common in other field to compare different models with Bayesian inference. No, I don't think it makes sense in these cases either.
"And ok, you are saying what people *shouldn't* do, but that isn't very helpful unless you have a better idea (or could at the very least formulate what the problem is in a more concrete fashion)."
A) This is a nonsensical argument and, for the record, one that I have addressed dozens of times already. If I point out that physicists are doing nonsense, it is not also my task to come up with something better for them to do. If they can't think of anything better themselves, maybe we just don't need them. Does that sound like a good solution to you?
B) I have said a seemingly endless amount of times that we should instead focus on resolving actual mathematical inconsistencies. I have written a whole damned book about this. I have said it in dozens of talks which have been recorded and that you can find online. I have even written on this very blog about it. As I said, however, since you (and all your friends) do not understand or do not want to understand that naturalness arguments are unscientific it seems I would merely waste my time to get into this.
Best,
B.
Sabine,
ReplyDelete"Well, first I agree of course that the standard model doesn't explain dark matter, but I don't see how this is relevant."
It's relevant because it is evidence that physics beyond the Standard Model exists, and is something worth searching for and trying to model.
"Second, I get the impression we are talking past each other. Yes, the standard model is unnatural. Yes, the Higgs mass is finetuned. I am not saying anything to the contrary - these are just properties of the model. I am saying that this is not a problem in need of a solution."
They are problems if you are a Bayesian.
"If you want to claim that "everything is subjective anyway" then I just don't know why you are calculating anything to begin with."
I'm not claiming everything is subjective. I am claiming that the piece of the probability calculation that is missing (the model priors) is the most subjective, and the part you are complaining the most about (fine-tuning) is not very subjective at all.
"Yes, I know it's common in other field to compare different models with Bayesian inference. No, I don't think it makes sense in these cases either."
I'm confused. I thought you said you had no problem with Bayesian analysis. Do you mean you only think it makes sense for parameter inference, not model comparison? Well, the fine-tuning effects automatically appear there as well, so even this is not an escape (posterior distributions will disfavour fine-tuned parameter regions, since these occupy a very small prior volume of the parameter space). You have to reject Bayesian analysis completely in order to reject fine-tuning.
"A) This is a nonsensical argument and, for the record, one that I have addressed dozens of times already. If I point out that physicists are doing nonsense, it is not also my task to come up with something better for them to do. If they can't think of anything better themselves, maybe we just don't need them. Does that sound like a good solution to you?
B) I have said a seemingly endless amount of times that we should instead focus on resolving actual mathematical inconsistencies. I have written a whole damned book about this. I have said it in dozens of talks which have been recorded and that you can find online. I have even written on this very blog about it. As I said, however, since you (and all your friends) do not understand or do not want to understand that naturalness arguments are unscientific it seems I would merely waste my time to get into this."
Sorry, I haven't read your work on this. I came across your post here by accident, I'm not a regular reader. From what you've said here though it is unclear what your problem with it is. As for "resolving actual mathematical inconsistencies", what do you mean? And why should this be the focus of the theory community? I would mention that I am more of a phenomenologist than a model-building sort of person, but I think fine-tuning is an important consideration on both fronts (we need to take it into account in statistical analysis of observations, and it is also import when cooking up models to explain new phenomena). It's not like it is the only consideration, people build models with all sorts of ideas in mind. I'm not arguing that naturalness should be the sole consideration or driving force of the community. But it makes no sense to dogmatically reject it either. Theory people are not crazy or unscientific to worry about this.
Ben,
ReplyDelete"They are problems if you are a Bayesian."
Then the Bayesian is still misinterpreting their mathematics. I thought we had agreed above that the quantification of naturalness/finetuning tells you nothing about the probabilty of the theory being correct. So what is the supposed problem?
"I'm not claiming everything is subjective. I am claiming that the piece of the probability calculation that is missing (the model priors) is the most subjective, and the part you are complaining the most about (fine-tuning) is not very subjective at all."
Indeed, and I have agreed on this several times. I am telling you that the non-subjective part is not a problem, and the entire "problem" you construe is in the subjective part.
"I think fine-tuning is an important consideration on both fronts (we need to take it into account in statistical analysis of observations, and it is also import when cooking up models to explain new phenomena). It's not like it is the only consideration, people build models with all sorts of ideas in mind. I'm not arguing that naturalness should be the sole consideration or driving force of the community. But it makes no sense to dogmatically reject it either. Theory people are not crazy or unscientific to worry about this."
Finetuning arguments are good to use if you have a probability distribution. The way that they are currently used in hep you don't have a probability distribution, and you cannot have one. That's unscientific procedure. It's bad methodology. I do not "dogmatically reject" it, I simply tell you it's unscientific. You should not do it. You are wasting your time. And you are wasting other people's money.
Book is here, paper is here.
"Then the Bayesian is still misinterpreting their mathematics. I thought we had agreed above that the quantification of naturalness/finetuning tells you nothing about the probabilty of the theory being correct. So what is the supposed problem?"
ReplyDeleteWe agreed on no such thing. It give you plenty of useful and interesting information about the probability of the theory being "correct" (I will put aside the issue of what that means for now). It just doesn't tell you *everything* about it. Would you also suggest that we don't even bother to look at how well a theory can reproduce experimental data? That also doesn't tell you everything about the probability of a theory being correct, because it also neglects theory priors, but no-one would argue that we therefore should ignore it.
"Indeed, and I have agreed on this several times. I am telling you that the non-subjective part is not a problem, and the entire "problem" you construe is in the subjective part."
Ok, but this is just bare assertion. *Why* should we ignore something that probability theory tells us is a relevant effect?
"Finetuning arguments are good to use if you have a probability distribution. The way that they are currently used in hep you don't have a probability distribution, and you cannot have one. That's unscientific procedure. It's bad methodology. I do not "dogmatically reject" it, I simply tell you it's unscientific. You should not do it. You are wasting your time. And you are wasting other people's money."
What do you mean if you "have" a probability distribution? Are you talking in the same way as in your paper (which I only skimmed just now, so apologies if I get it wrong):
"But it doesn’t matter which way one attempts to calculate probabilities, this will not put naturalness arguments on a solid mathematical footing. The reason is that, when it comes to the laws of nature, we don’t observe repeated events or sample over many outcomes. Any talk about probability distributions or priors refers to the distribution of theories in some mathematical space, almost all of which we cannot observe. We have only one set of laws of nature."
If this is what you mean, then I'm afraid you are just completely misunderstanding the nature of Bayesian probability. You have offered a frequentist criticism here. But Bayesian probability is completely different to frequentist probability. They are defined in fundamentally different ways. If you are unfamiliar with this then may I suggest this Stanford Encyclopedia of Philosophy article on the philosophy of probability: https://plato.stanford.edu/entries/probability-interpret/. It is a really great introduction to the topic, I highly recommend it.
Ben,
ReplyDeleteWow, did you just accuse me of not knowing the difference between the Bayesian and frequentist interpretation? Seriously? Here, I also have something that I recommend you read: https://en.wikipedia.org/wiki/Mansplaining
I use the frequentist interpretation in both my paper and my book, though in an earlier version I had a long elaboration on the Bayesian case. It doesn't make a difference because it simply moves the ambiguity from the probability into the prior, as I have told you several times. It is beyond me why some people think that Bayesian probabilities are somehow magically different from the frequentist interpretation.
" It give you plenty of useful and interesting information about the probability of the theory being "correct" (I will put aside the issue of what that means for now)."
You put aside the relevant point.
"Would you also suggest that we don't even bother to look at how well a theory can reproduce experimental data?"
Indeed. You shouldn't look how well a theory reproduces data, but how well it reproduces data compared to another theory. Absolute performance is entirely irrelevant. And if you do a relative performance, and you want to do a Bayesian assessment, you better make sure that the assumptions with unknown priors are the same. I told you above that for this reason I prefer computational complexity - in this case you have assumed you formulate the models in the same way, ie the priors are reasonably the same.
Generally, I am afraid I get the impression you are not thinking about what I am saying. Not that this surprises me. Look at the post on which you are commenting. And yet you are here trying to defend naturalness.
Sabine,
ReplyDelete"Wow, did you just accuse me of not knowing the difference between the Bayesian and frequentist interpretation? Seriously? Here, I also have something that I recommend you read: https://en.wikipedia.org/wiki/Mansplaining
I use the frequentist interpretation in both my paper and my book, though in an earlier version I had a long elaboration on the Bayesian case. It doesn't make a difference because it simply moves the ambiguity from the probability into the prior, as I have told you several times. It is beyond me why some people think that Bayesian probabilities are somehow magically different from the frequentist interpretation."
If it beyond you why people think that Bayesian probabilities are different from the frequentist interpretation then I'm afraid that you simply *don't* understand the difference. They are completely different, at a fundamental level. They describe entirely different concepts. If you do not understand this then it is not possible for us to make any progress in this conversation.
Sabine,
ReplyDeleteBy way, I just wanted to make it clear that it I intend no insult by suggesting that you don't understand this distinction regarding interpretations of probability correctly. I think that most physicists, most scientists, do not understand this distinction correctly. Philosophy of probability is not part of their general education, and even statistics is generally taught in a pragmatic way only, without much regard for the foundational issues, and often completely lacking a Bayesian component (though this is changing nowadays). And I think the great majority of scientists have never read anything like the Stanford Encyclopedia of Philosophy article that I posted earlier. It is not basic material. But I think it is essential background for the issues we are discussing.
Ben,
ReplyDeleteYour continued accusation that I supposedly don't understand what Bayesian inference is does not help your case. I have told you above, please explain why an odds ration of, say 10^-4 that doesn't take into account the full model priors is a problem. Just spell it out. Write it down. Come on, use your brain! Your previous answer was "They are problems if you are a Bayesian." What's this? An argument from popularity? You have since only made evasions.
What I take away from this exchange so far is that you clearly never thought about what the numbers you calculate actually mean, if anything, and now you refuse to acknowledge it.
Sabine,
ReplyDeleteI enjoy this topic and am generally happy to discuss it in an academic fashion at any time, however I don't plan to continue much longer if you are only interested in being belligerent about it. I find discussion to be a much better use of my time than debate.
"Your continued accusation that I supposedly don't understand what Bayesian inference is does not help your case. I have told you above, please explain why an odds ration of, say 10^-4 that doesn't take into account the full model priors is a problem. Just spell it out. Write it down. Come on, use your brain! Your previous answer was "They are problems if you are a Bayesian." What's this? An argument from popularity? You have since only made evasions."
No it is not an argument from popularity. The key idea behind naturalness/fine-tuning is simply built into Bayesian probability at a fundamental level. That is why I say you cannot simultaneously accept Bayesian analysis and also reject naturalness. This is a logically incoherent stance to take. I am happy to accept that Bayesian theory has its limitations, however you have said that you have no problem with Bayesian analysis, and this just cannot be true if you reject naturalness entirely.
As for "why an odds ration of, say 10^-4 that doesn't take into account the full model priors is a problem" (I suppose you mean *isn't*), it is as I said earlier, a partial answer is better than no answer. And the model prior part can be left to the judgement of the reader, and trivially multiplied in to the result. That is what priors *are* after all, they represent subjective states of knowledge of rational agents. Whatever they might be for any individual, the Bayes factor tells that individually how they should rationally update their beliefs upon examining new evidence. That is why people very commonly compute only Bayes factors. You cannot tell people what they should believe apriori (at least not within the Bayesian paradigm, you must look elsewhere for this), but you *can* tell them how they should rationally update their beliefs in light of new data. This is the whole idea of how Bayesian probability works.
You can *never* calculate model priors in Bayesian probability. That is simply not how it works. Sometimes you can make certain arguments based on other background assumptions (symmetries for example), but in general you cannot.
Sabine,
ReplyDeleteAs for the math, feel free to read my PhD thesis where I go into the full details: https://figshare.com/articles/Epistemic_probability_and_naturalness_in_global_fits_of_supersymmetric_models/4705438
Hi Ben,
ReplyDeleteThanks for the reference. I'll have a look at this.
As to your comment:
"That is why I say you cannot simultaneously accept Bayesian analysis and also reject naturalness. This is a logically incoherent stance to take."
Bayesian analysis has its uses. This isn't one of them. I don't know what you think is logically incoherent about this.
"As for "why an odds ration of, say 10^-4 that doesn't take into account the full model priors is a problem" (I suppose you mean *isn't*)"
You are claiming it is a problem. I am saying you should spell out what's the problem.
"as I said earlier, a partial answer is better than no answer. And the model prior part can be left to the judgement of the reader, and trivially multiplied in to the result."
If I tell you the probability that you die tomorrow is x*10^4, but don't tell you x, that's not a partial answer, that's no answer.
Sabine,
ReplyDeleteAs to your comment:
"Bayesian analysis has its uses. This isn't one of them. I don't know what you think is logically incoherent about this."
It's logically incoherent because you cannot do Bayesian analysis without accounting for naturalness. It happens automatically, unless you specifically (or accidentally) do the analysis in such a way to cancel out the effect.
""As for "why an odds ration of, say 10^-4 that doesn't take into account the full model priors is a problem" (I suppose you mean *isn't*)"
You are claiming it is a problem. I am saying you should spell out what's the problem."
Now I'm confused. I didn't say it is a problem, you did, you are saying that I need to account for the full model priors.
"If I tell you the probability that you die tomorrow is x*10^4, but don't tell you x, that's not a partial answer, that's no answer."
If I give you the result of a medical test that comes back positive 99.99% of the time *if* you are going to die tomorrow, then this is a key piece of information that you must use to update your beliefs about whether you will die tomorrow and what further actions you should take, regardless of what the apriori chance of you dying tomorrow was. You would be crazy to ignore this test just because you don't have a concrete grasp of how apriori likely you were to die tomorrow.
Ben,
ReplyDelete"I didn't say it is a problem, you did, you are saying that I need to account for the full model priors."
I am not saying any such thing. To repeat it once again: You can calculate part of the relative odds and get some number, say 10^4 or what have you. Fine with me. That's a property of the model. I am not saying it's incorrect. It's correct the same way it's correct that the SM w/o susy is sensitive to the UV parameters. No one doubts that this is the case.
The question is why do you want a theory that is natural, which does not have this sensitivity? Why is a natural theory better? You have give no answer to this. I am telling you this question cannot be answered because to claim that such a theory would be preferable you'd have to be able to calculate the full ratio. Which you can't because you're comparing models with different assumptions.
"If I give you the result of a medical test that comes back positive 99.99% of the time *if* you are going to die tomorrow, then this is a key piece of information that you must use to update your beliefs about whether you will die tomorrow..."
Sure, but this analogy fails on two points. First, in the case of naturalness you are not making a statement about the likelihood about anything that's observable. The Higgs mass is whatever it is and neither the SM nor any supersymmetric extension of the SM predicts it. It's a free parameter in both cases. Second, your analogy is about an absolute probability, whereas in the case of naturalness you are already discussing a relative probability.
Sabine,
ReplyDelete"I am not saying any such thing. To repeat it once again: You can calculate part of the relative odds and get some number, say 10^4 or what have you. Fine with me. That's a property of the model. I am not saying it's incorrect. It's correct the same way it's correct that the SM w/o susy is sensitive to the UV parameters. No one doubts that this is the case.
The question is why do you want a theory that is natural, which does not have this sensitivity? Why is a natural theory better? You have give no answer to this. I am telling you this question cannot be answered because to claim that such a theory would be preferable you'd have to be able to calculate the full ratio. Which you can't because you're comparing models with different assumptions."
The natural theory will receive less of a penalty in the Bayesian analysis due to fine tuning. Just as some other theory might receive more of a penalty in the analysis due to being unable to explain the data as well. You agreed earlier that the latter is good to know (we want to know the relative best goodness-of-fit that models can achieve), however this enters into the posterior odds calculation in *exactly* the same way as the naturalness penalty. Each Bayes factor can be factorised into a maximum likelihood ratio, times an "Occam factor" (see e.g. Mackay 2003: http://www.inference.org.uk/mackay/itprnn/ps/343.355.pdf) which penalises models for "wasting" parameter space, and which corresponds to what I call a "naturalness penalty" here. So if we care about one of these things, why should we ignore the other? Your criticism that the exist of an unknown makes the calculation meaningless applies equally well to consideration of the likelihood ratio. Why should we care about the relative goodness of fit of two models if we cannot compute their prior odds? If the prior odds are free then any arbitrarily bad fit can be completely cancelled by enormous prior odds. This is the problem with your criticism; it is far too nuclear, and destroys any possibility of Bayesian analysis under any circumstances (since the prior odds are always undetermined). In realistic cases we do *not* find all prior odds to be acceptable, even if we can argue about what they should exactly be. We may agree that the prior odds are not 1:1, and at least not 1:10^100. The Bayes factor therefore *does* tells us something, even if it isn't as much as we might wish for.
""If I give you the result of a medical test that comes back positive 99.99% of the time *if* you are going to die tomorrow, then this is a key piece of information that you must use to update your beliefs about whether you will die tomorrow..."
Sure, but this analogy fails on two points. First, in the case of naturalness you are not making a statement about the likelihood about anything that's observable. The Higgs mass is whatever it is and neither the SM nor any supersymmetric extension of the SM predicts it. It's a free parameter in both cases."
It's a free parameter, but in the case of compound hypotheses (i.e. those with free parameters), the Bayes factor is computed by integrating over all the free parameters. That is, one compares the marginal likelihoods of each model. So models that can only match data in a small region of parameter space get penalised in this integral. This is the "Occam factor" that Mackay refers to in my citation earlier. It is this property that favours models that make more successful predictions, in the Bayesian sense where a prediction is the prior predictive probability distribution for an observation. A model with a prior predictive distribution that favours the observed Higgs mass would be rewarded in the Bayes factor relative to a model which doesn't.
"Second, your analogy is about an absolute probability, whereas in the case of naturalness you are already discussing a relative probability."
ReplyDeleteThat is a trivial difference. We can reformulate the medical test case as "sick VS not sick", or as "you have disease 1 vs disease 2 vs diseases 3", the argument runs through just the same.
Ben,
ReplyDelete"So if we care about one of these things, why should we ignore the other? Your criticism that the exist of an unknown makes the calculation meaningless applies equally well to consideration of the likelihood ratio. Why should we care about the relative goodness of fit of two models if we cannot compute their prior odds?
You shouldn't.
"You shouldn't."
ReplyDeleteThen you reject Bayesian analysis completely, because one can never calculate prior probabilities. That is not how it works.
Ben,
ReplyDeleteI know you cannot calculate priors. I do not reject Bayesian analysis. I don't know what you even mean by that. It's a way to do calculations, how do you even reject it? What I reject is your interpretation of the results.
Look, let us stick with the case of susy. Neither susy nor the standard model predict the value of the Higgs mass. Could you please answer this question: How do you think that updating the data to take into account the LHC measurement of the Higgs mass makes it more likely that susy is correct given that susy requires additional assumptions whose priors cannot be larger than 1 and predicts the Higgs mass equally badly?
Sabine,
ReplyDelete"I know you cannot calculate priors."
Then I am having a hard time understanding your criticism of calculations that refrain from making assumptions about them.
"I do not reject Bayesian analysis. I don't know what you even mean by that. It's a way to do calculations, how do you even reject it? What I reject is your interpretation of the results."
Some people simply reject Bayesian interpretationa of probability (hardcore frequentists, for example). I think they are wrong to do this, but it is at least a self-consistent way to reject naturalness criteria, since it would release one from the epistemic obligation to follow Bayesian rules of inference, upon which all Bayesian analysis is based, and which automatically include the equivalent of naturalness penalties.
"Look, let us stick with the case of susy. Neither susy nor the standard model predict the value of the Higgs mass."
From a Bayesian perspective, yes, they do predict it. The quantities Pr(mH | SM) and, say, Pr(mH | MSSM) are calculable. These are predictive probability distributions for the Higgs mass. They are compute as Pr(mH|SM) = integral Pr(mH|x,SM) Pr(x|SM) dx, i.e. they are the model-averaged predictions for the Higgs mass. One needs priors for the free parameters (x), but unless one picks very strange priors that cancel out fine-tuning effects (and which would have to be post-hoc constructed to cause this to happen), these priors will encode notions of fine-tuning, whatever choice is made. Fine-tuned regions of parameter space will occupy a small prior volume, and so contribute very little to the model-averaged predictions.
"Could you please answer this question: How do you think that updating the data to take into account the LHC measurement of the Higgs mass makes it more likely that susy is correct given that susy requires additional assumptions whose priors cannot be larger than 1 and predicts the Higgs mass equally badly?"
It probably doesn't, I wrote a paper about this: https://arxiv.org/abs/1205.1568. But that was just one specific SUSY model. There are many, and one has to carefully analyse them to answer your question. And these analyses need to be done in such a way that they don't accidentally cancel out the fine-tuning effects (which can happen easily in the SUSY case because the mass spectrum is generally calculated in an iterative fashion, such that certain parameters are tuned to ensure the correct electroweak scale emerges. Accounting for this iterative calculation restores the appropriate penalty for the difficulty of reproducing the weak scale).
Ben,
ReplyDeleteI want to read your paper before responding, but I presently don't have the time. I'll get back to it. Sorry about that. Also, thanks for the discussion.
No problem, there is no rush.
ReplyDelete