But how unexpected was that experimental result really?
I learned only recently that by 1998 it might not have been so much of a surprise. Already in 1990, Efstathiou, Sutherland and Maddox, argued in a Nature paper that a cosmological constant is necessary to explain large scale structures. The abstract reads:
"We argue here that the successes of the [Cold Dark Matter (CDM)] theory can be retained and the new observations accommodated in a spatially flat cosmology in which as much as 80% of the critical density is provided by a positive cosmological constant, which is dynamically equivalent to endowing the vacuum with a non-zero energy density. In such a universe, expansion was dominated by CDM until a recent epoch, but is now governed by the cosmological constant. As well as explaining large-scale structure, a cosmological constant can account for the lack of fluctuations in the microwave background and the large number of certain kinds of object found at high redshift."By 1995 a bunch of tentative and suggestive evidence had piled up that lead Krauss and Turner to publish a paper titled "The Cosmological Constant is Back".
I find this interesting for two reasons. First, it doesn't seem to be very widely known, it's also not mentioned in the Wikipedia entry. Second, taking into account that there must have been preliminary data and rumors even before the 1990 Nature paper was published, this means that by the late 1980s, the cosmological constant likely started to seep back into physicists brains.
Weinberg's anthropic prediction dates to 1987, which likely indeed predated observational evidence. Vilenkin's 1995 refinement of Weinberg's prediction was timely but one is lead to suspect he anticipated the 1998 results from the then already available data. Sorkin's prediction for a small positive cosmological constant in the context of Causal Sets seems to date back into the late 80s, but the exact timing is somewhat murky. There is a paper here which dates to 1990 with the prediction (scroll to the last paragraph), which leads me to think at the time of writing he likely didn't know about the recent developments in astrophysics that would later render this paper a historically interesting prediction.
In 1996-8, at meetings of the Grand Challenge Cosmology Consortium, Ed Bertschinger repeatedly brought up the issue of the cosmological constant and that it could resolve the discrepancy between the stellar ages and the age of the universe, and the persistent inference from cluster data that Omega_m was ~ 0.3
ReplyDeleteIf I have my timeline correct, the reason initial condition codes for cosmological simulations with cosmological constant were at hand when the SNIa data came in is precisely because Ed and collaborators had anticipated this and started working on it.
De Vaucouleurs pushed for this for most of his career as the only way to reconcile the high values of the Hubble constant that he preferred, with the long ages he was forced to live with.
ReplyDeleteA quick few minutes on ADS (restricted to mentions of the cosmological constant before or during 1980) produced a 1980 paper by Occhionero et al in A&A pointing out that a modest cosmological constant would help successful structure formation compared to models with the same conventional energy density.
I think that at some level interest in the cosmological constant never went away, and many of the reasons why its such an attractive hypothesis had already been discussed by the mid-1980's (if not much earlier).
Ethan, Steinn,
ReplyDeleteThanks for these references! Best,
Sabine
I feel like there's one other point to make. One thing that doesn't come through from looking for references is that there was a widespread feeling that the cosmological constant was a needless complication during the 1970's. De Vaucouleurs talked about it, but he was seen as a pretty odd character. (In fact, there were several issues where he could properly be given credit for unusual foresight, but where his arguments were discounted at the time).
ReplyDeleteThe just before and during 1980 there was a sudden dramatic increase in interest in previously unpopular ideas in cosmology, including dark matter and the cosmological constant, driven by claims for a nonzero neutrino mass and the proposal of an early epoch of cosmic inflation. Suddenly it was not seen as crazy to include speculation on these topics in papers on cosmology. The SNe studies were driven by interest in pinning this down. If there weren't such widespread interest in the topic, I'm not sure that either of the high redshift SNe groups could have assembled the resources.
Would one assume there was a span of years that the cosmological constant was not of interest after Einstein's death in April of 1955? So 1955 to 1980?
ReplyDelete"dynamically equivalent to endowing the vacuum with a non-zero energy density" Interstellar medium is ~1.2 H atoms/cm³ with detectable optical properties. Photon vacuum symmetries, when assumed for fermionic matter, afford parity violations patched with unending symmetry breakings (re economics’ heteroskedasticity). Consider a pseudoscalar 1/length² cosmological constant - trace vacuum chiral background acting only on matter. Trace vacuum chiral anisotropy perturbs Noetherian coupling of vacuum isotropy and conservation of angular momentum. Discontinuous symmetry parity can ding Noether without contradiction. Fantastical dark matter is unremarkable Milgrom acceleration. Strop (jpeg) Occam's razor.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteWhen studying astronomy at Bonn University in the 1980s you were constantly reminded of the distinct possibility that Lambda was not zero by the late Prof. Wolfgang Priester. As evidence mounted for a high Hubble constant, it thus became an obvious solution to the age problem of the Universe in the late 1980s and early 1990s - and so, when the Lambda >> 0 result from the supernovae came out in 1998, I was not surprised but rather elated - Priester had been right after all (though he had favored a different value)! Incidentally I wrote a pro-cosmological-constant article in 1989 already, albeit in a self-published astronomy newsletter and thus not to be found by an ever-so-deep ADS search. :-)
ReplyDeleteThere is this paper published in Nature in 1978:
ReplyDeleteAccelerating universe revisited
Tinsley, B. M.
http://adsabs.harvard.edu/abs/1978Natur.273..208T
Best,
Christine
I come from a natural philosophy tradition so am less technically up-to-date than others contributing comments. However, it is interesting to see the slow tug of war between those advocating the accuracy of qft and those advocating the accuracy of GR. This tug of war is most clearly seen through the evolving view of the CC.
ReplyDeleteI hope I'm not being a nuisance here but the evolving CC really seems to show qft's problem. It does not breakout the split between potential and kinetic energy on a global scale, like the CC seems to. There is something in qft that is mathematically technically accurate but is being misinterpreted. At least so it seems to me.
I'll shut up now on this thread. Please proceed :-)
ReplyDelete
ReplyDeleteI have read several papers claiming that inhomogeneous cosmological models have the potential to explain the observed epoch of acceleration without dark energy or the Phoenix-like cosmological constant.
It seems like we have more theoretical possibilities than there are decisive predictions to test them.
Perhaps nature will offer some guidance eventually.
Robert,
ReplyDeleteYes, there are some people working on this, but it's quite controversial. If you have the same approach in mind as I, they're basically saying there's something about GR that we're not understanding. It's a hard sell because in the regime we're talking about gravity is very weak and esp structure formation is mostly numerical simulations anyway.
Having said that, I think that inhomogeneity plays a more important role than it is presently given, but that alone isn't going to make the cc go away. Best,
B.
Eric,
ReplyDeleteQFT in curved space is a very active area of research. There are many things that presently aren't well understood and I've heard talks from people claiming that if you renormalize correctly in a FRW background the problem goes away. I don't know enough about the subject to tell whether that's plausible or not. However, for me the most interesting one of the three cc puzzles is not why it's not huge, but why it's positive and not zero. Best,
B.
Skyweek, Christine,
ReplyDeleteThanks for the references. It's interesting that the topic apparently was on people's mind even more and earlier than I thought. It somehow reminds me of the status of technicolor today. It's another one of these ideas that seems almost dead and yet never really goes away. Best,
B.
The finding of dark energy is not Einstein's cosmological constant - and frankly, I don't understand, why it's called so. The original cosmological constant was inserted into general relativity for to make the expanding universe model steady state - whereas the new "cosmological constant" is making already expanding universe expanding faster and faster. It's evident, the new "cosmological constant" fits neither sign, neither derivation of the universe expansion introduced with older constant. These two constants actually have nothing in common.
ReplyDeleteMy paper interpreting the DMR results in 1992 included 3 models that fit the DMR data and the large scale structure data: hot and cold DM, open CMB, and Lambda CDM. So the cosmological constant was certainly in the mix of possibilities in 1992.
ReplyDeleteHi Sabine, Krauss and Turner actually wrote quite a few papers on this (pre 1998), as did Sean Carroll. But you're right it's not that widely known, pop accounts seem to like the story of the supernova shock!
ReplyDeleteBtw, there's also a mathematical reason, isn't there? I have always understood from Einstein's first paper on this that the cosmological constant is essentially a constant of integration - hence there is no reason for it to be exactly zero. It must be small to match observation, but it would be quite contrived to assume it to be zero!
Cormac
"First, it doesn't seem to be very widely known, it's also not mentioned in the Wikipedia entry."
ReplyDeleteWidely known by whom? (I wouldn't use Wikipedia as a point of reference here either. The quality of Wikipedia articles varies enormously.)
I started working at the Hamburg Observatory in 1992 and all of the papers you mentioned were discussed when they came out. Of course, it is only when hard experimental data come along that it trickles down to other fields.
As you say, there was some circumstantial evidence, but for every indication, there was an alternative model. Even in the sum it wasn't convincing. There were other ideas. For example, there was a serious paper by serious cosmologists suggesting that having a Hubble constant of 30 would solve many problems, including some which indicated a positive cosmological constant.
I put the blame on [drum role, please] particle physicists. Inflation types were so convinced that inflation meant a flat universe that they believed in the Einstein-de Sitter universe with no observational evidence whatsoever, and unfortunately many astronomers were in awe of them. Check out Dennis Overbye's Lonely Hearts in the Cosmos for a description of these times.
You might say that the current "standard model" is flat, but with a positive cosmological constant. But that was too much of a muchness then. People actually thought that if there was a flat universe with a zero cosmological constant, then surely Nature would use that and not something ugly. Check out The Deep Universe (proceedings of a Saas-Fee school) to see how much Allan Sandage (a traditional and respected astronomical cosmologist) was influenced by this.
"Vilenkin's 1995 refinement of Weinberg's prediction was timely but one is lead to suspect he anticipated the 1998 results from the then already available data."
ReplyDeleteNo way. Even the people involved in the collaborations didn't see this coming that early.
"When studying astronomy at Bonn University in the 1980s you were constantly reminded of the distinct possibility that Lambda was not zero by the late Prof. Wolfgang Priester."
ReplyDeleteI knew Wolf personally. A thoroughly nice chap. (Maybe he was nicer to me because I was an "early adopter" of the cosmological constant.) Yes, he believed in a positive cosmological constant. However, the actual values of lambda and Omega which he believed in were ruled out by other observations. Unfortunately, he never came to accept this.
"I have read several papers claiming that inhomogeneous cosmological models have the potential to explain the observed epoch of acceleration without dark energy or the Phoenix-like cosmological constant."
ReplyDeleteYes, you can find a host of suggestions in the literature. These are generally considered "fringe", and for good reason.
A general remark. One must distinguish between ideas in the literature and convincing observational evidence. Obviously the latter had to wait for the observations. On the theoretical side, since the 1930s it has been clear how the cosmological constant affects observational quantities. However, the observations weren't yet good enough.
ReplyDeleteNote that the cosmological constant is thus not some ad-hoc explanation for the observations; it is an old idea which only know has been confirmed since only now are the observations good enough.
Note that many people assumed it was zero not because of observations (no observations ever indicated it was zero), nor out of theoretical prejudice (though some did), but rather because it was difficult mathematically (especially before the widespread use of computers) so many people just set it to 0 to simplify calculations, which was justified because the observations weren't good enough. If the calculations are just order-of-magnitude anyway, there is no need to make them too complicated.
"Would one assume there was a span of years that the cosmological constant was not of interest after Einstein's death in April of 1955? So 1955 to 1980?"
ReplyDeleteEinstein's death had nothing to do with it. Yes, he introduced the cosmological constant in 1916, with a special value for a static universe. He later threw the baby out with the bathwater by calling it the biggest blunder of his life (though note that the only reference to this is a story told by Gamow in his autobiography). After his paper with de Sitter in the early 30s, in which Einstein argued in favour of a vanishing cosmological constant on purely practical grounds, he never worked anymore in this sort of classical cosmology. So, Einstein's death had nothing to do with interest in the cosmological constant.
John D. Barrow's The Book of Universes is highly recommended for a historical overview of stuff like this. (And look for my review of it in the August issue of The Observatory.)
"The finding of dark energy is not Einstein's cosmological constant - and frankly, I don't understand, why it's called so. The original cosmological constant was inserted into general relativity for to make the expanding universe model steady state - whereas the new "cosmological constant" is making already expanding universe expanding faster and faster. It's evident, the new "cosmological constant" fits neither sign, neither derivation of the universe expansion introduced with older constant. These two constants actually have nothing in common"
ReplyDeleteYou are completely wrong here. Read any introductory textbook on cosmology.
If it walks like a duck, and quacks like a duck then, until we have evidence to the contrary, we should assume it is a duck.
Both the currently observed value and Einstein's value are positive. In Einstein's case, it just cancelled the gravitational attraction, while today's value is larger and hence leads to accelerated expansion. The sign is the same. (Yes, Einstein's model is unstable, as was pointed out first by Eddington---it is an unstable fixed point in the dynamical phase space spanned by the cosmological parameters. However, the Einstein-de Sitter model is an unstable fixed point in the same way mathematically, but as far as I know this was never used as an argument against it.)
ReplyDeleteP. Helbig, esq.: Often wrong; almost never in doubt.
Beware those who pepper their opinions with absolutes. It reveals an unscientific emotionalism.
Zephir:
ReplyDeleteAs Phillip already said, it's the same constant. Just open a text book. It's a constant appearing in an equation. As I explained here, dark energy is a more general expression than cosmological constant. The CC is one type of dark energy, but there are other types of dark energy that look similar and have similar effects and we're waiting for experiments to distinguish between them. Best,
B.
Phillip,
ReplyDeleteIt's just my impression from talking to people, listening to talks, etc, that pre 1995 we "didn't know" of the CC. I looked at the Wikipedia entry before I wrote this blogpost to see if maybe I'm just dumb and "even Wikipedia knows". Alas it turned out the Wikipedia entry doesn't list the historical timeline either. It's not so much that I used it as a reference (for what?), but that I used it to get a sense of how widely spread knowledge of the earlier papers is. Best,
B.
Hi Phillip,
ReplyDeleteI knew there were others because of the information that was developed.
It was a way of saying that the quest even after Einstein proposed the CC the mathematical language was being developed lead to the issues of the cosmological constant being geometrically expressed I think with this in existence, this became a model with which to support Omega?
Always open to corrections.
Best,
What about this 1986 paper by C. Sivaram:
ReplyDelete"Uncertainty Principle Limits on the Cosmological Constant" ?
(If you don't have access to this paper, there several others by the author restating the results).
Best
If you'd rather see the cosmological constant go away, then you might want to check out my very modest "Tiny Alice" theory, circa 1975, in which the same evidence of accelerated expansion is explained rather elegantly (if I say so myself) by positing a situation in which the universe finds itself expanding into a black hole. It is the gravitational pull of the black hole that is the source of the acceleration, NOT any cosmological constant, or "dark energy." For details, see: http://fqxi.org/data/forum-attachments/Is_the_Universe_Expanding_Into_a_Black_Hole.pdf
ReplyDeleteThis comment has been removed by the author.
ReplyDelete"It's just my impression from talking to people, listening to talks, etc, that pre 1995 we "didn't know" of the CC."
ReplyDeleteYou were talking to the wrong people. :-)
"dark energy is a more general expression than cosmological constant. The CC is one type of dark energy, but there are other types of dark energy that look similar and have similar effects and we're waiting for experiments to distinguish between them"
ReplyDeleteRight. However, there is no observational evidence--and not because no-one has looked---that "dark energy" (a rather useless term---as Sean Carroll pointed out, many things are dark and everything has energy) is in any way distinguishable from the classical cosmological constant.
It's just my impression from talking to people, listening to talks, etc, that pre 1995 we "didn't know" of the CC.
ReplyDeleteCheck out this paper from the 1960s. This shows that the cosmological constant was certainly considered among cosmologists back then. By the way, this is probably my all-time favourite paper. Its information content is huge; there is more information in some footnotes than in other entire papers. If you read only one paper on classical cosmology, this should be it.
Hi Bee, I spent a bit of time on this a while ago, and the answer given in most of the history books is that, at least for theoreticians, the cc 'never really went away', partly because of the age problem, and partly because it arises naturally as a constant of integration.
ReplyDeleteReferences for this are North's 'the measure of the universe, Kragh's 'cosmology and controversy' Barrow's 'book of universes' and Longair's 'the cosmic century'
coraifeartaigh,
ReplyDeleteYes... but that's not what I meant. The interesting thing for me is not that there were theoreticians who never stopped believing in the CC, but that there was evidence in the data well before the supernova results from 98. Best,
B.
"there was evidence in the data well before the supernova results from 98"
ReplyDeleteNow I'm confused. It depends on what you mean by "evidence". Before the SNIa stuff, many people suggested (Krauss and Turner, Ostriker and Steinhardt etc)---and in prominent journals---that a positive cosmological constant would be a natural interpretation of the data. On the other hand, for each individual observation, there were competing and, to some, more plausible alternatives. This was around the time when people started producing "joint constraints" by combining different cosmological tests. This is really the best argument, but even here it was just possible to avoid the cosmological constant. I think the problem is that "extraordinary claims demand extraordinary evidence". There is nothing wrong with that per se; rather, the problem was that the cosmological constant was deemed to be an extraordinary claim, when in fact it was pretty much the most conservative approach one could take. However, the observational uncertainties were still large enough that it wasn't mandatory.
Again, I blame the particle physicists, or at least most of them. A bit earlier, the idea of inflation surfaced, which also involves accelerated expansion. Many people believe that the cosmological constant and inflation are somehow related. However, it seemed easier in inflationary models to have a zero cosmological constant NOW.
Keep in mind that the cosmological constant as introduced by Einstein appears in the context of classical field theory. It has nothing to do with quantum theory, particle physics etc. Whether the "cause" of the cosmological constant is something like an inflation field is a completely open question.
Note that Weinberg's idea is that the classical cosmological constant is large and negative which almost, but not quite, balances the vacuum-energy contribution from QFT, the fine-tuning having an anthropic explanation. Weinberg thus realizes that the two are separate things (though he invokes both), but of course Weinberg is far from the average particle physicist.
Maybe the confusion has to do with the way the different sciences work. Cosmology is not an experimental science (despite some observers speaking of "CMB experiments" and the like). It is more like history, or paleontology. Was there evidence for evolution before Darwin? Yes, in some sense, but it was Darwin's detailed investigation which made a convincing case.
Hi Phillip,
ReplyDeleteSure, it wasn't clear really what the data showed. All I was saying is that there were hints already. Maybe the word "evidence" that I used is too strong, if that's what you mean. Best,
B.
I think that's a good assessment. It's a gradual process, of course, and some people saw it coming more quickly than others. What is definitely not the case, though, is that someone saw hints in the supernovae data already in 1995.
ReplyDeleteThis comment has been removed by the author.
ReplyDelete"Again, I blame the particle physicists, or at least most of them. A bit earlier, the idea of inflation surfaced, which also involves accelerated expansion. Many people believe that the cosmological constant and inflation are somehow related. However, it seemed easier in inflationary models to have a zero cosmological constant NOW."
ReplyDeleteI too blame the particle physicists. But I disagree that we ever should have expected a zero CC. I think it was always a matter of mathematical convenience, which you mentioned in another comment. For GR people there was never a good reason to believe the CC would be zero, at least from a theoretical viewpoint. Now, if we had discovered dark matter 100 years before discovering an accelerated universe outside of our local galaxies, then I would agree with you. It would be correct then to use the factual data to override the theoretical naturalness of a non-zero CC.
I will continue to think that a zero CC was just too mathematically convenient and that other ideas, primarily from qft, had their essential motivation coming from that convenience. Of course the worst offenders by far were the particle physicists. But the GR people weren't completely innocent either in that mathematical convenience, along with pressure from others, overrode their common sense that the CC should NOT be zero.
Actually, contrary to what all the televised pundits say, dark matter in combination with dark energy is very theoretically consistent. First one should make a distinction along with an association. "Inflation" seems like it goes with "unitary", while "Inflationary" as used in the common cosmic inflation/multiverse scenario would be associated with "infinite" and zero "measured" background.
ReplyDeleteThe unitary concept would be consistent with extreme acceleration at the beginning that is consistent with the isotropic details on large scales that is seen later. It would also be consistent with acceleration that slows down along with the expansion.
The infinite viewpoint doesn't really tell you anything except that since energy is infinite you can assign it anything according to what you see locally. In that scenario you could call the local galaxies a "universe" with a CC of zero. And the farther galaxies another universe with a CC of close to zero. It gains you nothing to think in this way.
But if you think in terms of a unitary value in which the CC represents the splitting point of potential and kinetic energy through time, then it makes perfect sense. It would act like the central part of a new Hamiltonian of the universe. Just like nucleons condensed when it got cool enough, so cold dark matter congealed out of existing particles when it cooled much farther.
I should add one more thing and then I'll be quiet. The way the multiverse crowd is thinking, including Bee (I think?)one could go on infinitely adding more universes. Each would be incorporated into the existing universe retroactively as one observes them or gets concrete evidence of them. It's the opposite of reductionist philosophy where as you learn more things become simpler.
ReplyDeleteIf I was a quark in a nucleon I would definitely tell the other two quarks that the CC is surely zero and that the universe consisted of one nucleon. If I was an oxygen atom in a molecule of water I would definitly also tell my two cohorts that the the CC is surely zero.
But if a jolt of electricity hit us then I would say it wasn't zero. The CC is NOT what you feel inside of your home environment. It is the potential energy outside of it that creates other particles on different scales. And that energy density is reduced as condensed matter is created. It is no different in principle from evaporation/condensation cycles in condensed matter physics.
What is so difficult to understand?
Eric,
ReplyDelete"The way the multiverse crowd is thinking, including Bee (I think?)..."
You're thinking wrong. I've made it clear on this blog numerous times that I think the occurrence of "multiverses" of one way or the other is inevitable, and simply a consequence of us aiming to describe nature by mathematical law. Since I don't believe that nature, fundamentally, is mathematics, to me this just signals a shortcoming of the models we can presently construct. But for practical purposes this shortcoming is, at least right now, a philosophical one. Which is to say, I think all this discussion about multiverses is a giant waste of time. Best,
B.
Then in this particular case I'm glad I was thinking wrong...
ReplyDeleteBtw talking about history, can someone tell me
ReplyDeletewhy D. Kazanas's 1980 paper on inflation (in ApJ) is not as widely cited as some of the other papers by Guth, Linde etc? In fact it was not cited in the Planck paper on inflation. I think the paper by Kazanas is the first which mentioned that accelerated expansion could solve the horizon problem (although it doesn't mention flatness problem).
Were people in cosmology community aware of Kazanas paper before Guth's 1981 article on inflation?
Thanks
The expanding universe was recognized back in the 1920s, but the big bang theory wasn't really confirmed until radio astronomy was developed in the 1950s & 1960s. In the late 1960s and into the 1970s, the big focus was on completing and nailing down the standard model. Once the micro-theory (the standard model) and the macro-theory (the big bang theory) were in place, the focus turned to reconciliation, getting them both to work together. When I read you post, I was reminded of a 1981 article (Inflation and the Mysteries of the Cosmos) on Grand Unified Theories (GUTs), inflation and Guth in Science. Guth's inflation was one of the early approaches to dealing with the flatness and horizon problems, along with the baryon problem. The article never mentioned the cosmological constant, but discussed a number of proposed theories that would combine the micro and macro theories and get us the universe we observed.
ReplyDeleteIn other words, the cosmological constant was always in play. The success of the standard model and a host of new astronomical observations led to a reassessment. I'm not too happy with the various results. Physicists, unlike evolutionary biologists, seem to have a problem with the "just so" aspects of existing theories. Maybe they are right. One never knows where a theory may lead. Look at Copernicus who advocated the heliocentric theory because it provided a strength metric for the astrological influence of Mercury and Venus, unlike the geocentric theory.
Some slightly late comments from the middle author of Efstathiou, Sutherland and Maddox 1990 [here ESM],
ReplyDeletecited in Sabine's original post:
Firstly, ESM did not claim "the data requires a cosmological constant", but did show that
observations of large-scale galaxy clustering, (mainly data from the APM Galaxy Survey from
http://adsabs.harvard.edu/abs/1990MNRAS.242P..43M )
are a poor fit to Omega_m=1 CDM, but a good fit to a Lambda-CDM model with Omega_m * h \approx 0.2 .
Note this value is in good agreement with today's Planck + BAO best-fit of 0.208 .
[as usual h = H_0/(100 km/s/Mpc) ].
The new angle brought in by ESM was good data on large-scale structure; nearly all previous *observational*
evidence for Lambda had been on the age problem (H_0 * t_0 uncomfortably large), and/or
had not included CDM (i.e. trying baryons+Lambda flat models). That age problem
had rumbled on throughout the 1970s and 80s with no decisive conclusion, depending which
of the various H_0 results was chosen. My impression was, after the rise of non-baryonic DM
in the 1980s, there was a widespread belief that this "had to" make Omega_m = 1.
A few earlier papers studied Lambda-CDM phenomenology: see e.g. Peebles 1984, Vittorio & Silk 1985,
Bardeen, Bond & Efstathiou 1987, Holtzman 1989; but ESM was the first paper to fit the Lambda-CDM pair
to robust LSS data, extending well into the linear regime at large scales (~ 10 - 50 Mpc).
The basic line of argument was, if you assume n_S ~ 1 and the matter is mostly CDM, then the shape of the galaxy
correlation function at large scales mainly depends on Omega_m * h, and the best fit is ~ 0.2 as above.
All data agreed on a lower bound h > 0.4, so this pushes you to Omega_m < 0.5, typically 0.2-0.3 ;
on its own, this is relatively independent of a flat-Lambda or an open low-density universe, but
other evidence (including ages, CMB 1990-era upper limits, faint blue galaxy counts) favoured
the flat-Lambda over the open option. At the time, n_S ~ 1 and flatness were educated guesses based on inflation
since we had zero evidence for n_S ~ 1, and only moderate evidence for CDM existing, but it looks like we
guessed right. So, one could say Lambda is a successful prediction of CDM+inflation
models plus 1990-era data.
The ESM paper was noted at the time, but many/most people didn't like the double-dark aspect
(and indeed still don't! ); there was a huge expansion of interest in structure formation
after COBE discovered CMB anisotropy in Apr. 1992,
and it became increasingly accepted that inflation was very promising, but "original" CDM with
95% CDM , 5% baryons and h ~ 0.5 did not work quantitatively
(with new redshift surveys e.g. IRAS 1.2 Jy and CfA2 supporting the excess galaxy clustering).
Then, a reasonable handful of "CDM+X" variants emerged to fit COBE+large-scale structure data.
These included:
i) Lambda-CDM , basically as above
ii) Mixed dark matter (aka CHDM) with roughly 70% CDM + 30% HDM.
iii) Tilted CDM (TCDM) with a substantial red tilt, e.g. n_S ~ 0.7 .
iv) Open CDM (OCDM) with low Omega_m and Lambda=0.
v) tauCDM : Extra radiation models with late-decaying particles which shift the redshift of equality.
vi) CDM with Omega_m = 1 but H0 ~ 30 , as noted by Phillip H above.
vii) Non-power law initial spectrum.
and probably a couple more that I've forgotten.
( continued in next comment...)
(continued from above...)
ReplyDeleteSeveral of these were fitted in the Wright et al 1992 COBE paper, linked above.
All of these had some interest and a significant number of papers; it's hard to be quantitative,
but my general recollection is that the most popular was Mixed dark matter, with Lambda-CDM around
second or third; while in terms of actually fitting data the order was reversed.
Mixed DM was more popular since there was substantial reluctance
to accept Lambda (and/or a strong prejudice in favour of Omega_m = 1), but several observations
favoured Lambda-CDM, including the cluster baryon fraction.
A notable distinction (apart obviously from the accelerating expansion in Lambda-CDM) was H_0 :
as of the mid-90s Lambda-CDM could have tolerated any H_0 in [50,100]
since one could just tune Omega_m to ~ 0.2 / h.
However, the Mixed DM and tilted models ran into problems if H_0 moves above 60 or so,
since as H_0 goes up you need more HDM or tilt respectively and this became problematic:
if there's too much HDM then galaxy formation gets too late, or if the tilt is too big
you had trouble matching both COBE and galaxy clustering.
In around 1995/6 the early results from the HST H0 Key Project started to hint at H_0 ~ 70, and this
maybe started to nudge Lambda-CDM ahead of the alternatives.
An interesting snapshot of the pre-1998 situation is provided in the "Critical Dialogues in Cosmology"
conference held in Princeton in June 1996 (ed N. Turok, Princeton U.P. 1997). There Mike Turner
made the case for Lambda-CDM, Joel Primack for CHDM, Richard Gott for OCDM, among others.
I wasn't there but I have the book, and a quick scan shows roughly 1/3 of the talks give a mention of Lambda
(many but not all favourable !).
Then of course in 1998 the SNe acceleration results arrived and the community embraced Lambda-CDM with
remarkable speed. Personally I found the rapid community convergence surprising (Jim Peebles has
made this comment several times), since those early SN results were
only around 2.5 sigma from an open low-density model, and it's hard to be really sure about lack of SN
evolution, dust corrections etc. The SNe data have got much better since, but weren't that solid in 1998.
So, I think it's fair to say not many people would have been convinced in 1998 without all the
prior work on LambdaCDM.
I was a lot more confident after Boomerang and Maxima got the first acoustic peak in 2000; this
killed open models, and then in 2001-2 several results from 2dFGRS
(Peacock et al 2001, Efstathiou et al 2002, Verde et al 2002) gave convincing evidence for Omega_m ~ 0.3;
as is well known, a straight subtraction of Omega_tot - Omega_m gives you Omega_vac ~ 0.7
(though it's less sensitive to equation of state).
(third and last part...)
ReplyDeleteA few responses to comments above:
@Steinn: Re simulations, there is a small LambdaCDM sim actually in ESM (32k particles !) .
I think several N-body groups (Durham, Park & Gott, Cen & Ostriker ?) were running
big LambdaCDM simulations (and others) by around 1993/4, but I'll have to chase references.
@Phillip H: "for every observation there was an alternative explanation". This is true, but the
alternative explanations were often incompatible for different observations; probably, LambdaCDM was the only
fully satisfactory CDM+"X" model by around 1995, as was argued by Turner & Krauss and Ostriker & Steinhardt.
However, this left the "none of the above" or CDM + many extra parameters, so the case was not conclusive.
Also, for some further reading try Calder & Lahav, Physics World 2010,
http://physicsworld.com/cws/article/print/2010/jun/02/dark-energy-how-the-paradigm-shifted
Apologies for the long post but it's an interesting story, and more complicated than the usual journalist
hype that Lambda just fell out of the sky in 1998.
@Will: Great summary of what actually happened (based on the fact that it agrees with my recollection) by someone who lived through these exciting times and lived to tell the tale.
ReplyDeleteI agree on all of your points, especially the last one directed at me. The situation was similar to determining Avogadro's number a hundred years previously: no one measurement was conclusive, but collectively they were decisive. (Ernst Mach was playing the fin-de-siecle Rocky Kolb, holding out on the conservative position and being influential because of his large stature.) The problem was that experts in one field were aware of alternative explanations, but didn't know enough about other fields to realize that an alternative explanation in one field was incompatible with one in another field.
Yes, maybe some people were too quick to accept the cosmological constant, since as you say at the beginning the evidence wasn't that conclusive. However, there are always proponents (in some cases devil's advocates) to all sides of a question. Note that the people who believed in the Einstein-de Sitter universe did so on the basis of even less evidence, even though they were more convinced. Of course, it is this other camp Landau must have been referring to when he said that cosmologists are often wrong but never in doubt. :-)
@Will: A great summary by someone who lived through these exciting times and lived to tell the tale. I agree with all of your points, especially the one directed at me. There were always alternative explanations, but experts in one field couldn't see that their alternative explanation was incompatible with what was going on in other fields (e.g. the structure folks could favour a low Hubble constant---there was a paper pushing the value of 30---but this was incompatible with direct measurements, even those by Sandage).
ReplyDeleteIn some ways it was similar to the measurement of Avogadro's number a hundred years before: no one measurement was decisive, but collectively they were. (The role of the fin-de-siecle Rocky Kolb was played by Ernst Mach: hanging on to the conservative position and being influential while doing so since people respected/were afraid of him due to his great stature.)