 |
| Cover of New Scientist Nov 3rd 2018. |
The article is by Michael Brooks and it’s a summary of a claim I wrote about last year, that the original 2015 gravitational wave detection by the LIGO collaboration was not a real signal.
This claim was made by a Danish group around the physicist Andrew Jackson. This group tried to reproduce the data analysis of the LIGO collaboration with the publicly available data and could not.
The New Scientist article quotes Duncan Brown at Syracuse, who until recently was a member of LIGO, with reassuring the reader that the Danes are “credible scientists,” and Slava Mukhanov who likewise emphasizes that the Danes are people “with a high reputation.” Slava is also on record stating that “There is no mistake” in the analysis of the Danish group. Peter Coles chimes in to say that “I think their paper is a good one and it’s a shame that some of the LIGO team have been so churlish in response.”
The New Scientist article then draws a comparison between the LIGO case and the BICEP case. BICEP looked for the so-called primordial gravitational waves, which are in a different wavelength regime than LIGOs. Their supposed signal turned out to be merely noise.
The two measurements, however, work entirely differently because BICEP did not (attempt to) directly measure gravitational waves. Instead, it looked for a secondary signal that is the imprint of the primordial gravitational waves in the cosmic microwave background. The BICEP signal was contaminated by foreground from the Milky Way. The same problem does not exist for LIGO.
Michael Brooks in the New Scientist article then points out that this is the first time we are analyzing gravitational wave signals and it’s still early days, so if an independent analysis cannot reproduce the result that’s a problem.
Interestingly, Brooks seems to have found out that the key figure in the LIGO paper about the first discovery does not actually show the quantity that they used in the data analysis. I had been told about this previously, though I cannot now recall the details. (I believe it was something about the plotted quantity not actually showing the relevant significance. Anyone knows better, pls leave a comment.)
The way that I heard about it was that some members of the collaboration wanted a pretty plot that “could be printed on a T-shirt,” ie they opted for beauty over scientific relevance. I don’t know if that’s what really happened, but it sounds plausible enough. I recall thinking at the time that if that’s true it was a dumb decision; clearly this move pissed off some people in the collaboration and those had no reason to keep their mouth shut forever.
For me the issue with the Danish group’s criticism was not whether the signal is real. LIGO people pointed out problems with the Danes’ analysis to me that even I could understand. No, the issue for me was that the collaboration didn’t make an effort helping others to reproduce their analysis. They also did not put out an official response, indeed have not done so until today. I thought then – and still think – this is entirely inappropriate of a scientific collaboration. It has not improved my opinion that whenever I raised the issue LIGO folks would tell me they have better things to do.
We have here a group of researchers not associated with the collaboration which tries to follow the analysis methods that the collaboration reported and they cannot confirm the collaboration’s results. This should not happen. If the collaboration is not able to explain their procedures so that other scientists can find out what they’re doing, that is a problem that must be fixed. This is the first time anyone analyses data for gravitational wave signals and the methodology needs to be clearly documented. Evidently, this is not presently the case.
The Danes btw haven’t been the only ones who tried to redo the LIGO analysis and didn’t manage to. I know this not because I’m obsessed with LIGO, but because people send me references about this. I also get plenty of emails and comments from cranks who think that LIGO is a fraud and just wasting tax-money and so on. All this is reason why I think the LIGO collaboration is doing a disservice to science by ignoring the matter.
I was thus happy to read in the New Scientist article that some people from the LIGO collaboration are at least working on a response. But, well, it’s certainly taking some time.
What happened after the Danish group made their claim in June last year is that the VIRGO collaboration joined LIGO’s search for gravitational waves. So now the analysis draws on data from three detection sites. They have since seen a gravitational wave event with an optical counterpart recorded in several telescopes. Brooks reports that the Danish groups still doubts the detection because this event, which happened in August 2017, was originally labelled a “glitch”. The story about the glitches is indeed peculiar. The glitches are occasional false alarms in the detectors. They tend to not have the frequency spectrum of the real events however. So it seems to me like a stretch that the Danes are holding on to their claim, and I am not sure why New Scientist dug this up now.
If you cannot (or do not want to) access the New Scientist piece, Jennifer Ouellette has an excellent summary on Ars Technica.
Update: The LIGO collab has published a brief response to the New Scientist piece on their website:
“1 Nov 2018 -- Claims in a paper by Creswell et al. of puzzling correlations in LIGO data have broadened interest in understanding the publicly available LIGO data around the times of the detected gravitational-wave events. The features presented in Creswell et al. arose from misunderstandings of public data products and the ways that the LIGO data need to be treated. The LIGO Scientific Collaboration and Virgo Collaboration (LVC) have full confidence in our published results. We are preparing a paper that will provide more details about LIGO detector noise properties and the data analysis techniques used by the LVC to detect gravitational-wave signals and infer their source properties.”
ImIkind of a nobody and at the time of the original event I called the LIGO head engineer regarding possible simultaneous EM detection. lIGO people are incredibly open, transparent and approachable. They are professionals and outstanding people. Also, they're a little busy.
ReplyDeleteFascinating stuff! Love the science. Sorry to be "controversial" but I was immediately struck by how good these scientists are at sharing data so it can be checked. And I was unfortunately reminded of the quote by Phil Jones "We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try to find something wrong with it" ---EXACTLY! John
ReplyDeleteUnknown,
ReplyDeleteWhat the story should tell you is that you actually can not check the results from the shared data.
But at least they are sharing the data, right? That is the main point I meant. "Checking" may have been a "hopeful" comment. I am always reminded of Feynman's quote of course, something that "..belief in the ignorance of experts" to keep probing forever..... it seems. I am always thinking what we will "know" in 100 years compared to now... The longer I live, the more I think "know" is a questionable concept... Love your stuff. Fascinating stuff nevertheless.
ReplyDeleteThis looks a bit like a minority report that is less credible than the LIGO team. I am not familiar with the details of these analyses, and the gravitational wave phenomenology Thorne worked on is complicated. That is then used to compute estimated signals and compared with data. This is a huge effort.
ReplyDeleteI am not always that impressed with New Scientist, and I think Scientific American is become similarly "tabloidish." New Scientist is quick to jump on sensationalism.
The BICEP2 results were found to be not that distinguishable from the polarization of EM radiation by interstellar dust in the galaxy. This had the unfortunate effect of calling the find of B-modes into question. The results though are being worked on by BICEP3 and Keck array. Currently the null hypothesis is r < .07 and the results are above 3σ. So the claim is gaining traction again.
Good to hear, could you cite the BICEP3 paper?
DeleteThey're sharing part of the data, windows around the events. It's not enough to really understand the noise or the detector drift or the glitches.
ReplyDeleteBuild one more LIGO in South Australia desert, Pedirka and Tirari, average GPS -30, 132, near antipode to LIGO Livingston. The land is worthless, flat, tectonically quiet, and politically stable. Don't whine, trilaterate.
ReplyDeleteThis work:
ReplyDeletehttps://arxiv.org/abs/1711.00347
conducted an independent analysis and seems not to find anomalous correlations. Apparently it was possible to check the results from the available data.
Of course you can check the results from the available data. The question is whether that check gives you a reliable result. Also, please note that this story carried out in June 2017.
ReplyDelete"They're sharing part of the data, windows around the events. It's not enough to really understand the noise or the detector drift or the glitches."
ReplyDeleteSabine, this is simply not true. They have the whole analysis run data from 2015/2016 available on their website (called O1 data release): https://www.gw-openscience.org/data/
JKuipers,
ReplyDeleteWell, that was the status at the time when the Danes published their paper. Maybe things have changed since. The situation at the time was what is summarized in the paper mentioned above:
"The LOSC has made available time series records of
the measured strain data for each of the reported events.
The events are roughly central within short (32 s) and
long (4096 s) records, each available at 4096 samples per
second (sps) and 16384 sps."
Our paper, arXiv:1711.00347 (https://doi.org/10.1016/j.physletb.2018.08.009), shows that GW waveforms very similar to the templates published by the LIGO can be extracted from the published data using a completely independent method. It also identifies a major flaw in the work of the Copenhagen group. Quite simply, they failed to use proper window functions when processing finite length time series data.
ReplyDeleteOK, I don't know about before. Since I saw the article of New Scientist today I am wondering about the complaint if the data can be downloaded but I am not a specialist.
ReplyDeleteVery interesting developments. My hunch (backed up with some dialogue with various physicists working in this field who could never, in my view, adequately explain how grav waves in spacetime itself can be detected by apparatus in that same spacetime that is being "waved") is that we probably haven't actually detected grav waves at this point.
ReplyDeleteTam,
ReplyDeleteWell, wow, you should try to find some physicists who know what they talk about. This is one of the crackpot complaints that I get all the time and if you look at the calculation (or ask Google for that matter) the answer is very clear. Laser is light, the apparatus is not. They don't move on the same curves through space and time, hence they don't react the same way to a gravitational wave. Consequently you can measure the deformation caused by the wave. I will admit that the commonly shown illustrations don't quite capture what's going on.
Martin,
ReplyDeleteI mentioned the issue with the window functions already in my piece last year.
"Well, wow, you should try to find some physicists who know what they talk about."
ReplyDeleteTo be fair, if you re-read Tam Hunt's post, it is possible that the problem is more with the listener than the explainer.
Regarding the article itself. This may have been quite appropriate following the second event GW151226. This was still only using the two LIGO machines and the team really was dredging the noise to find the signal. Some independent, scientific skepticism at that point may have been quite healthy.
In the aftermath of GW170817 it all seems a little late.
Quite apart from the math, there is a physics problem with the LIGO claims. The issue is straightforward: a piece of classical machinery is claimed to have a design sensitivity capable of detecting a quantum scale signal (10^-20m). The claimed detection is via a variation in the fixed separation between two pairs of mirrors set at right angles to each other.
ReplyDeleteThe problem that arises with this claim is one of scale. The signal scale is one billion times smaller than the scale at which the mirrors' atomic components are vibrating chaotically. At a scale of 10^-11 the mirrors have no fixed separation. So the question is straightforward: how can the LIGO machinery detect a a variation in a fixed separation of two mirrors when there is not any meaning to a fixed separation of the mirrors at the claimed signal scale.
The only explanation I've seen offered for this is some math with a fixed L(separation) term - a fixed separation is assumed in the math. This casual assumption has no physical justification. LIGO's claimed detection is based on an assumption that is not physically meaningful.
Whatever the source of the detected signal, it cannot be a variation in a fixed mirror separation, that does not exist at the claimed signal scale.
bud rap,
ReplyDeleteIt is abundantly clear that you don't know the first thing about quantum noise to begin with. The only reason I am approving something as uninformed as your comment is to demonstrate the ridiculous amount of conspiracy theories that are ranking around the LIGO detection. For starters, I recommend you look at this and then read up on any of the noise sources mentioned therein.
Will it be possible to detect more wider wave forms with eLISA? Now there are restrictions due to GR-simulations what are wanted to see...
ReplyDeletePhysics tody is random, pointless people doing random pointless things.
ReplyDeleteI'll throw myself out.
P.S. stop saying spacetime! Not a thing.
To catch up I found the chapter 26.6 of Kip Thorne´s account very nice. He treats the interferometer (laser light and mirrors) both in TT gauge and in local Lorentz coordinates and also refers to what will change with LISA.
ReplyDelete@ bud rap,
ReplyDeleteThis problem you appear to be alluding to is removed by squeezing the QED vacuum of the laser photons. With quantum mechanics we have the ΔpΔx ≥ ħ/2. This would appear to make measurements LIGO performs impossible, and the Δx would make the position of the mirror impossible to pin down. However, this is the uncertainty principle at quadrature. The squeezed state operator is S = (a^†)^2 F^*(θ,φ) - a^2 F(θ,φ) for a and a^† the lowering and raising operators for removing or generating photons and F(θ,φ) a function of phases. The squeezed state operator has the effect of Δx → 0 and Δp → ∞. In this way the position of the mirrors can be measured with subatomic precision.
How does one actually squeeze quantum states of light? A photon can enter a sapphire crystal that has a birefringent property of splitting the polarization state of light. Thus a photon with frequency ν splits into two photons with frequencies ν' and ν' such that ν = ν' + ν". This means the sapphire crystal takes an input photon and splits it into an entangled state of two photons with reduced frequencies, called parametric down shifted frequencies, and where Δx → 0.
The next obvious question would then be, how is it then the mirror that is made of atoms etc, can have its motion detected down to a scale much smaller than an atom? This involves one of those oddities of quantum mechanics. Bohr made light of this, and is a centerpiece of his interpretation of quantum mechanics. It is also analogous to the Umklapp process in the Mossbauer effect. In that an atom in a crystal with an excited state can emit a photons so the the entire crystal lattice has a recoil. Since the crystal with say a mole of atoms is so much more massive the error in measuring this photon energy and the energy of the process is negligible. Similarly, the mirror acts as a single system that is essentially then classical with no quantum fuzziness.
The Danish group's paper "On the time lags of the LIGO signals" says: "With special focus on GW150914, we report correlations in the detector noise ... the other two events, GW151226 and GW170104, exhibits similar behavior."
ReplyDeleteDear Sabine (or dear readers): Do you know if there is similar correlation found in detector noise for GW170817, the neutron star event?
@bud rap
ReplyDeleteKio Thorn has recorded a whole online course on how the geavital wave detector works and how they reached this sensitivity. If you relly want to understand this, go to:
Lec 1 - Phys 237: Gravitational Waves with Kip Thorne
https://youtube.com/watch?v=Afq4b1HZcIc
@Lawrence Crowell LIGO did not used squeezed light in the experimental run that made the detections talked about here.
ReplyDeleteTypo: in here it should be from chapter 27.6.2 on.
ReplyDelete@ Peer, I am not able to find any reference to your statement the detections did not employ squeezed photon states. There is the reference below that states squeezed coherent states of light were employed in 2011.
ReplyDeleteThe LIGO Scientific Collaboration (2011). "A gravitational wave observatory operating beyond the quantum shot-noise limit". Nature Physics. 7 (12): 962. https://arxiv.org/abs/1109.2295
This objection to whether gravitational waves were detected also does not seem to address the GW170817 event, the first observed merger of binary neutron stars or kilonova that was accompanied by an optical signature.
@Lawrence Crowell The Caltech LIGO R&D webpage currently lists squeezed light as a possible future improvement to LIGO. It says that they have currently eliminated almost all other noise, so that using squeezed light is the most feasible way to improve detector resolution at this point. But it also says that it hasn't been done yet.
ReplyDeleteThe 2011 paper was reporting the results of an experimental study on whether it would be feasible to improve the resolution by using squeezed light. Apparently, it was successful, or this wouldn't be in their current plans.
The New Scientist article shows the raw data (red line with no signal visually emerging from the noise) and the filtered data magnified by 100 (blue line). Even with this magnification, the blue signal is much smaller than the raw data.
ReplyDeleteEven without knowing any of the details of the experiment, this gives me a very queasy feeling about this result. The NS article also explains that "The detectors are plagued by these short inexplicable bursts of noise, sometimes several per hour." I think this is meant to refer to the glitches, in which case such frequent glitches are surely a cause for further concern.
From reading the article, the decision not to use a windowing function in the re-analysis was deliberate, and I speculate that this was because varying the width of the Fourier window might cause this minute glitch to come and go!
I guess it appears I was wrong. I thought squeezing had been implemented with the upgrade or advances that were made a few years ago.
ReplyDeleteDavid,
ReplyDeleteIt's not the raw data that is shown. The window-function has nothing to do with the glitches. A badly performed windowing will leave you with periodic residuals that look neither like a glitch nor a signal. Best,
B.
I though additional gravitational waves have been detected by LIGO from Neutron Star mergers as well?
ReplyDeletepete,
ReplyDeleteAs I mention in the last paragraph.
The BICEP3/Keck effort is updated here.
ReplyDeletehttp://bicepkeck.org
"It's not the raw data that is shown. "
ReplyDeleteSo do you know what it is? Is it an artist's impression (analogous to the one originally presented)? Is the quoted scale magnification real - I mean if so we are still talking about an extracted signal that is about 500 times less than the raw signal. Is the filter basically a low-pass filter, or something more elaborate? If the red curve means anything, it seems to contain bursts of activity, none of which correspond in time to the claimed detection!
https://arxiv.org/abs/1602.03844
ReplyDeleteMaybe here?
David,
ReplyDeleteIt says in the caption below the figure how the data has been processed. The issue isn't the data (and not the documented processing of the data either) the issue is the model that's being subtracted (where does it come from?) and that the quantity shown is not the quantity that's actually used to determine the significance. As I said, I have not heard the rumor that the New Scientist article put into the world that the model was made "by eye".
@Lawrence Crowell, I appreciate your considered response but I don't think it addresses the problem I raised. The question regards the nature of the mechanical system employed and whether or not it is capable of detecting a 10^-20 signal. The mirrors employed are not crystals. The Mossbauer effect is dependent on the crystal lattice.
ReplyDeleteTo state my objection in a slightly different fashion: the premise of an interferometer is that a variation in the path lengths of two identical mirror systems set at 90 degrees to each other can be detected by a shift in the interference pattern of the recombined light from the two paths. The system is premised on the idea that there is a fixed path length between each of the two mirrors which depends in turn on there being a fixed surface to each mirror.
At the scale of 10^-20 there is no such fixed surface and therefore a fundamental premise of the interferometer mechanism is absent. The fixed surface is not undetermined or uncertain in the quantum sense, it is simply, physically nonexistent. So it is not a question of how to extract the data from the noise but rather how the data could possibly have been acquired by the system as built in the first place.
@ bud rap,
ReplyDeleteMy comment about the Mossbauer effect is that there is a similar classical behavior that occurs. For this reason there is the emergence of a clean surface. A part of what makes this is that the atomic aspects of the mirror, such as silicon etc, are smaller than the wavelength of the light.
Scientists are humans, I've heard. And scientists who working with expensive, publicly funded equipment are under hard pressure to succeed. Not so?
ReplyDeleteSo why not handle like good established: a result is only then a definit result if it was experienced and reported by several independent workers or groups.
Because the tools are very expensive, there can not be the consequence to believe any results of it. What is about cheating, mistakes, hopeful misinterpretations, ...?
@ Lawrence Crowell,
ReplyDelete...the emergence of a clean surface. A part of what makes this is that the atomic aspects of the mirror, such as silicon etc, are smaller than the wavelength of the light.
Well yes, the wavelength of the laser is of order 10^-6, but even allowing, for the sake of argument, the proposition that the laser can drive the atomic structure at 10-^10 to a "clean surface" why should that surface be expected to be "clean" at 10^20 where the detection has to take place? How can the combination of the 10^-6 laser and a "clean" 10^-10 atomic surface be sensitive to a 10^-20 signal. I still fail to see a physical pathway to the actual detector.
Not mentioned so far: there’s a citizen science/online crowd study of LIGO glitches; it’s called Gravity Spy, and has been running for quite some time now. I find it odd that none of the papers etc relevant to this discussion even mention this research, let alone how it might - or might not - help evaluate claims and analyses.
ReplyDeleteWhile I’m far from being a dab hand at Gravity Spy, I’d be astonished to learn that there are LIGO glitches which mimic gravitational wave radiation signals.
Dear Sabine, you wrote:
ReplyDelete"The BICEP signal was contaminated by foreground from the Milky Way. The same problem does not exist for LIGO."
Indeed it does, but LIGO-Virgo are expected to be less oblivious to foreground, which is clear as so many papers devoted to the failure and improvement of stochastic GW searches - due (mainly) to Schumann resonances - have been published.
Foreground contamination from instrumental noise, space weather, and thunderstorm-generated magnetic modes (or some combination) is mentioned in several recent (aLIGO-era) publications, and can produce similar SNR and higher likelihood NR template fits than weak transients like GW151226 and LVT151012.
Proper lag constraints and low coincident magnetometer noise in ASD, however, are the only barriers from conflation with NR chirps upon notching, bandpassing, whitening, and decimating h(t). These SNR>8 [purported] non-GW transients arrive at 0.01 Hz (1 per 100 seconds) trigger rate.
Magnetometers were reported to be generally malfunctioning, unplugged, or turned off throughout O1 and O2, not a single short-time analysis around signal in any publication. LIGO will not release magnetometer or power mains data, with the question avoided and redirected onto their open strain and flag data.
Phase-encoded strain is not picky with quasiperiodic transverse modes with nonstationary quasinormal enveleopes. Correlated noise with same lag as signal, but also PSD and ASD artifacts at 140-145 Hz (0.007s) also affects GW170817.
fulguritics,
ReplyDeleteBICEP did not measure gravitational waves. As I said above, they did an indirect measurement (or tried to do one) by measuring CMB polarization.
Sabine,
DeleteForgive me for not clarifying and for misunderstanding your intention. Foreground contamination for both projects is an analogous measurement-limiting condition, but LIGO has better press, more specialized Bayesian statisticians and programmers who are NOT cosmologists, astrophysicists, or astronomers, and less public demand for objectivity and transparency. ...I hope aLISA can pull it off, and LIGO couldn't have been done without LIGO being done, regardless of the nature of SNR>8 h(t) signals and their Keplerian/sawtooth cycle-synchronized multiscaled arrival time and phase correlations.