Tuesday, June 19, 2012

Did the OPERA affair harm or benefit science?

Last year around that time of the year I was working on a draft that re-investigated an old question, whether superluminal information exchange is compatible with special relativity, causality and locality. Just when I had finished the draft and had sent it to a few colleagues, I read the news on the alleged superluminal neutrinos in the OPERA experiment.

Suddenly, the arxiv was flushed with papers on superluminal propagation. My draft didn't have anything to say about neutrinos, but the last thing I wanted was for it to drown in a flood of papers I was convinced would become rapidly irrelevant. So I sat on the draft, but watched the appearance of papers on the arxiv, and the fate of the "OPERA anomaly" closely. Luckily, none of the papers that appeared had any resemblance with mine.

Now that the anomaly vanished into a loose cable that will probably become a running gag in the history of science, I want to return to a question that we already discussed two years ago: Did the attention of the press on what turned out to be a mistake benefit or harm the public perception of science?

A Nature Editorial from two months ago, titled "No Shame," boldly declared everything that happened went perfectly alright:
OPERA's handling of the incident, at least publicly, was a model for how scientists should behave. Ereditato and Auterio acted responsibly when speaking publicly by sticking close to their data and avoiding over-interpretation. They shared their work with their competitors, and did their best to quickly address outside criticism. In the end, it was OPERA's internal checks that found the loose cable. When the error was discovered, physicists on the team wasted no time in publicly announcing the problem, along with others they had exposed during their review.
This elaboration however misses the point that "sharing work" doesn't exactly require to hold a press conference. It would have been perfectly possible for the collaboration to share their results and trouble-shoot without making such a big fuzz about it. The press would probably have heard of it anyway, but the collaboration could have calmly explained them that they're working on it.

The GEO600 collaboration, for example, when faced with their "mystery noise" did not make a secret of it either. They had information on their website and in conference proceedings, and in fact a lot of people knew about it. There were a few reports in the media, but not even NewScientist managed to create a sensation with a collaboration member who just declared that everybody expected the noise to vanish in a rather mundane explanation. Which was exactly what happened.

A lot of my colleagues think that any attention physics receives in the press is good. I don't think so. I understand that it's certainly an ego-boost if you read in the news about a topic on which you are an insider, and suddenly friends and relatives want to hear your opinion. Ah, I'm so knowledgeable, so cool, I'm so up-to date. But there are downsides to this. In my earlier post I listed three points that one should take into account

  • First problem is that while it might draw interest in the short run, it erodes trust as well as interest in the long run. Science lives from accuracy more than any other field. The more often people read claims that something maybe was discovered, but then it wasn't, the less attention they'll pay if they read it again. Quantum gravity in cosmic rays! No wait, nothing to find there. Quantum gravity in gravitational wave interferometers! No wait, nothing to find there. Quantum gravity at the LHC. Sorry, nothing there either. This erosion of trust is exactly why I spent so much time on this blog deflating the headlines.
  • Second problem is one of principle. If rumors or measurement errors are considered a useful tool to draw attention, and attention is a good thing, why not make up a few? 
  • Third problem is that these rumors tend to circle around a few presently particularly popular topics or institutions, and if they dominate the news the vast majority of topics remains uncovered. This, I think, is clearly a disadvantage to education in general and also to the way researchers perceive the relevance of their work.
After having watched the OPERA anomaly come and go, I want to add a fourth point:
  • Fourth problem: The more public attention a topic receives, the more likely scientists in the field are to jump on the train and spend time coming up with contributions, eventually wasting their time and, essentially, taxpayers money.
It is not my intention to blame anybody for anything. It is always easier to point the finger after the facts are on the table. I also do not think people who made mistakes necessarily should resign over them, or should be forced to go. In many instances it seems better to me to keep people who have learned their lesson. If anything, the collaboration should maybe rethink their decision-making procedures. Clearly, they must have thought that it's a mistake that is not on their territory, something that they need outside help with. And they reported it before they had even run all the checks that they had at their disposal. That seems odd to me.

No, the reason for the post is that I think the above points should be taken into account in similar situations because it's not at all clear more attention is always better. Also, I am interested in your opinion.

Related: I saw coincidentally that Giovanni Amelino-Camelia has a paper on physics.hist-ph that discusses the relevance of the OPERA affair for the philosophy of science. I haven't read it, but if you're interested in the details, it might be worth a look.

35 comments:

Uncle Al said...
This comment has been removed by the author.
Uncle Al said...
This comment has been removed by the author.
Uncle Al said...

"a running gag in the history of science" Polywater, cold fusion, the Fifth Force; oscillating beta-decay rates. Expose an obscure footnote (beta-decay) or a weak founding postulate (Euclid vs. Bolyai, Newton vs. relativity and QFT). Grant-funding rejects empirical risk (discovery) while embracing publicity (fraud).

2012 Higgs excuses are already boiling over. Massless boson photon exact vacuum symmetries are not exact for fermionic matter, nor should they be. Vacuum trace chiral asymmetry toward fermionic matter sources SUSY empirical sterility to quantum gravitations' failure. Opposite shoes vacuum free fall non-identically in a trace chiral vacuum background. The experiment in existing apparatus tests spacetime geometry with enantiomorphic crystal structures. The worst it can do is succeed.

coraifeartaigh said...

I couldn't agree more Bee, especially point 1.
I think such announcements, soon followed by corrections, do erode trust in science and I think this erosion is a very serious matter.
A great deal of climate change skepticism, opposition to GM foods and other debates really come down to trust in the institutions of science..one cannot expect joe public, or journalist public, to be well versed in particular areas of science; yet we do expect them to trust the experts. That trust is easily eroded is every other artcile reports that red wine is good/bad/good for you etc

Arun said...

Sorry, "opposition to GM foods" - I trust science, I don't trust "science sponsored by large corporations". We have plenty of historical examples where corporations have claimed that their product is safe or will not produce such and such side effects based on "scientific" testing, and have proven to be false.

In general, "science" that is conducted with the explicit purpose to inform public policy needs an extra degree of skepticism.

Arun said...

E.g., watch this slowly unfolding herbicide fiasco: http://www.minnpost.com/earth-journal/2012/05/arms-race-against-weeds-new-weapons-extend-losing-strategy

What these stories will not tell you is where the corporations got the glyphosate-resistance gene in the first place (it came from nature).

Arun said...

Here is the history of asbestos safety: http://environmentalchemistry.com/yogi/environmental/asbestoshistory2004.html

Framing these questions as "trusting science" is exactly wrong. People rely on the prestige of science to support their schemes that exploit other people.

Arun said...

OPERA benefits science:

1. Even a well-established theory like special relativity can be challenged (unlike what many conspiracy theorists say).

2. The scientific process is shown to work.

OPERA harms science:

1. Scientists communicating to each other via press conference is a bad idea.

---

Anything which sets up a conflict of interest with the primary objectives of science is a bad idea.

- So I think Brian Greene's peddling of string theory is a really bad idea, or Lisa Randall her idea of extra dimensions. Their celebrity raises a potential concern about their own objectivity about their theories. Yeah, maybe they can handle it. Maybe not.

Teaching well established science is a different matter.

- Science in the hands of corporations used as a means to promote their products, change public policy, etc., also requires extra scrutiny. A lot of the "this is good for you" followed by "this is bad for you" comes from this kind of science.

- Science in service of political ideology - Soviet genetics or Apartheid South Africa's research into racial differences comes to mind - is also a problem.

In comparison with the above, the OPERA affair is really very minor.

Plato Hagel said...
This comment has been removed by the author.
Plato Hagel said...

Hi Bee,

I have an advantage that you do not have. I stand outside of academia looking in.:)

As the Opera events unfolded I was intrigued by all the information that seemed to flood the stage, that I saw the institutions such a PI weigh in from their panel of speakers. Other scientists too, that I follow. I thought this an enormous success because it brought out perspective on science that was right from the heart.

Having people weigh in now about the philosophy of science is nice too, as one can reflect in context of that situation.

Timing is everything isn't it, and looking back to to the subject of FTL it was not a subject one did not consider in the face of looking at all perspectives. Lee was most helpful by supporting João Magueijo on VSL.

The perspective for me was that such a foundations was already laid in mediums of earth, ice and water as such, that we could see the resulting Cerenkov. That we could see the relativity of Muon detection use in the discrimination of matter states of the earth. Volcanoes and magma flows?

I believe your views from inside need to be adjusted from people who can see the situation from the outside looking in. Just continue to be good scientists.

Best,

Georg Lentze said...

Hi Bee,

I'm interested to see that around this time last year you were working on the compatibility of superluminal signals with special relativity, causality and locality as I was doing exactly the same thing minus the locality aspect at the time. My thoughts are summed up in blog posts here:http://bit.ly/PpzIRE and here: http://bit.ly/LAbNQR, in case you're interested. I'm certainly interested in yours, have they been published yet?

Regarding the OPERA experiment, could it be that the scientists went public in the way they did simply because they were quite excited about their results? Yes, with hindsight it appears that they should have carried out more checks. But they were cautious enough not to claim anything. So I don't think much harm has been done.

Georg

Red C said...

A related, but much more important question would be:

"Do Blogs and internet forums harm or benefit science?"

I am raising this partly because blogs without any credentials start being cited in the mass media, as if they'd have any scientific competence.

Surely the answer will depend on the blog, and also on whether you ask laymen or actual scientists.

Bee said...

Hi Arun, Coraifeartaigh,

The issue of trust in science is of course more involved than just the reporting in the media. I'm only saying that reporting is part of it that, when not done wisely, can contribute to a lack of trust. Arguably if the science is bad to begin with that's worse.

Best,

B.

Bee said...

Hi Arun,

I agree with you that there are other examples that have a larger impact on research in general, but the OPERA affair makes a good example because it has generated such a lot of publicity and the situation is reasonably clear in that it's at this point not a matter of opinion any more. By and large I think people underestimate really the impact that widely spread, and often repeated, information from sources with a certain authority have on our thinking, not excluding the scientists themselves. Best,

B.

Bee said...

Hi Georg,

No, it's still not published. I guess I kind of lost interest in the whole topic at some point. Ah, anyway, I'll give a talk about this in two weeks, so that's why it's on my mind again. I guess sooner or later I'll dump it on the arxiv, I'll leave a link at your blog then. Best,

B.

Bee said...

Hi Red C,

The question you raise is important, but I don't think it's more important. One of the most relevant factors for the impact of a piece of information is the authority of the source. Which is exactly why it is not a good idea for a scientific collaboration to make a press conference on a result that they are not really really sure is correct, even if they try to be cautious about it, because just the fact that they make the press conference on the result is enough to gather attention. Sources without such an authority sometimes do have an impact but it's the exception rather than the norm, and that's likely also going to stay this way just because the way humans process information isn't going to change within a matter of years. Best,

B.

Phil Warnell said...

Hi Bee,

I think this whole episode’s lesson is an old one and that is when it comes to important matters it’s best to keep ones powder dry. That is scientific resource like any are not limitless and as such hyped stories like this one can amount to having the same effect as crying wolf once too often, with the consequences being fewer will listen and have even less to care. Moreover in a world where real science is increasingly competing with pseudo science, we can ill afford to have them to become indistinguishable from one another; particularly at a time like the present, where the promotion of good science may be all which differentiates having the whole human experiment succeed rather than fail.

“Do not trust the cheering, for those persons would shout as much if you or I were going to be hanged.’

-Oliver Cromwell (the same fellow who first advised to keep ones powder dry)


Best,

Phil

Bee said...

Hi Phil,

You said that very well. Best,

B.

Red C said...

Hi Bee,

well the question I raised is more important on the long term; a few years from now the Opera affair will be forgotten with little trace.

On the other hand, all those opinated blogs (not yours) and forums work in the direction of undermining scientific authority and integrity, and over time this will lead to an erosion of what is meant by an “expert". The general public cannot see the difference between informed statements and empty claims, it actually tends to distrust professional experts just for being experts.

Already now, in many discusions, the opinion of clueless laymen counts equal to, if not more than, the one of real scientists.

Bee said...

Hi Red C,

"The general public cannot see the difference between informed statements and empty claims, it actually tends to distrust professional experts just for being experts."

This is in conflict with what I have read, I am referring to some studies (forgot details) discussed in Surowiecki's book according to which the authority of the source is one of the most important factor by which people judge on the value of information (together with the number of people who hold the same information and how often that information is repeated). Essentially the same is explained in Sunstein's book, experts have an impact on the way people judge information that is usually *larger* than you'd want it to be (for the sake of good decision making) not smaller, as you seem to believe.

There is maybe the more subtle question how difficult it is to tell for somebody whether a professor who claims to be a professor is really a professor, but fraud unfortunately always happens. It is still the exception rather than the norm. Best,

B.

Edward Current said...

I think it harmed science, and next time, better discretion should be called for. The lay public doesn't know anything about science and increasingly distrusts it (conspiracy theory, frighteningly, tends to have more traction these days). Scientists are like baseball umpires and drummers -- people only notice them when they mess up.

Giotis said...

Why scientists are always crying for public's attention? Is it just vanity?

The appreciation of their work by their peers should be enough for them.

Anyway I wouldn't worry so much; 99,9% of the public simply don't care about that stuff. For them science is running somewhere in the background. They are interested only for the end product; what science can do to improve their lives...

chimpanzee said...

"Success is 90% failure"
-- Soichiro Honda

Scientific Method is a self-correcting method which seeks the truth. Whereas Religion (& other crackpottery) is based on dogmatic belief

The Superluminal Neutrino episode reminds me of the early foray in ESP (Extra Solar Planets) in Astronomy, where a researcher had to admit error:

http://www.wired.com/thisdayintech/tag/extrasolar-planets/

but it was the start of a new field (ESP .. alien planets is supposedly a good way to attract Public attention)

Here's a good link on "Birth of a New Theory" (Geophysics), where the old theory was overturned by Wegener. Whose general idea was right (Continental Drift aka Plate Tectonics), but the mechanism was wrong.

here

"Plate tectonics Theory fundamentally changed the way Earth Scientists view our world. Acceptance of the theory has taken hold worldwide, although a few skeptics remain. There are skeptics to every theory, and I guess you would have to say whether we are talking about evolution or a spherical Earth. You would have to say there are skeptics. There are people I am told who don't believes that the Apollo Mission actually went to the moon. [ Crackpots ] My belief is that it is a very healthy thing that there are such skeptics because they TEST OUR HONESTY basically. It's the emperor has no clothes sort of phenomenon. We all believe in a new hypothesis and accept it so completely that perhaps we don't test where we should, and it is very important that these skeptics exist. Although plate tectonics explains many geological phenomena, certain questions about how it actually operates remain."
-- Stanford Geophysics prof

The LHC could be a turning point in HEP (just like Geophysics was revolutionized by some new hardware..produced new data & thus new ideas)

chimpanzee said...

"Birth of Theory" video here:

scroll down, click on Birth of a Theory

Note Dr Tanya Atwater (aka Mrs Plate Tectonics), who was a young Scripps post-doc (?)leading the charge. A friend of mine (U of Arizona, Masters Geology) told me the older conservative (male) were too cowardly to stick their neck out, & let the young female "hang herself". Instead, it backfired on them

"The biggest risk is not taking a risk"

LHC is supposedly reinforcing the Standard Model, no sign of Supersymmetry. Using the Geophysics "model", is there new data (given the new hardware)..generating a new theory?

Zephir said...

/...A Nature Editorial from two months ago, titled "No Shame," boldly declared everything that happened went perfectly alright: .../

Why the project leader and speaker were fired, after then? Something wrong definitely happened after then, if these guys were actually innocent.

Bee said...

Zephir: He wasn't fired, he resigned. Best,

B.

Zephir said...

/* ...Zephir: He wasn't fired, he resigned. Best.. */
Yes, two main leaders of the whole project suddenly decided to resign after some 16 group leaders suddenly voted against the pair yesterday (while 13 voted in their favor) - everything in time of OPERA experiment failure. What a surprising coincidence...;-)

The first step, how to become an idiot is to consider the other people an idiots.. For some reading Phenomenology of Philosophy of Science: OPERA data

Zephir said...

I've two reasons for to consider the outcome of the OPERA experiments rather suspicious.

1) We have read, that the misinterpretation of OPERA results were caused with two errors, one has lead to the positive deviation, the second one has lead to the negative deviation. At the end of the whole story suddenly only one source of experimental failure (i.e. loose optical cable) was considered - why?

2) The poor connection of optical cable doesn't explain the [url=]published dependence[/url] of the neutrino delay on the neutrino frequency, which fits the previous experimental finding so "well". It seems for me, one of the sets of results was essentially fabricated and now the only question is, which one: the first published one - or the "correction", released six months later?

Arun said...
This comment has been removed by the author.
Arun said...

Sounds incredible, but this is what they are reporting (NOTE: they corrected their statement - the grass in question, Tifton 85, is a hybrid not a genetically modified variety):

http://www.cbsnews.com/8301-201_162-57459357/gm-grass-linked-to-texas-cattle-deaths/

"A mysterious mass death of a herd of cattle has prompted a federal investigation in Central Texas....The grass is a hybrid form of Bermuda known as Tifton 85 which has been growing here for 15 years, feeding Abel's 18 head of Corriente cattle.....Preliminary tests revealed the Tifton 85 grass, which has been here for years, had suddenly started producing cyanide gas, poisoning the cattle."

"What is more worrisome: Other farmers have tested their Tifton 85 grass, and several in Bastrop County have found their fields are also toxic with cyanide. However, no other cattle have died.

Scientists at the U.S. Department of Agriculture are dissecting the grass to determine if there might have been some strange, unexpected mutation."

Arun said...

The point of my previous post is as follows;

Tifton 85 turns out to be a hybrid of a South African grass and another hybrid, Tifton 68.

"Tifton 68, is a stargrass, a species that has potential for generating HCN, but hasn't apparently done so since the time the University of Florida starting using it for grazing in Ona, Florida in **1972**."

The speculation is that the drought conditions in Texas have triggered the HCN activity.

The point is that 40 years is not enough to establish that a **hybrid** is safe.

( The only reason we know regular stuff is safe is from long usage and because evolution has had time to work out the kinks. )

The underlying complexity of organisms is what leads me to doubt the short few years of testing of Genetically Modified crops can prove their safety.

Phil Warnell said...
This comment has been removed by the author.
Phil Warnell said...

Hi Arun,

Cyanide production is a naturally developed characteristic of many plants, which among other things serves as an insecticide. What you might find interesting, as explained in this article, is scientists have been able to remove this by way of genetic engineering to render former deadly species of plants safe. The truth is the plant kingdom has been using chemical warfare for eons against those that would eat them, which I think is the true lesson that should be learned here as it pertains to this story. However I’m not suggesting that GM should be taken lightly, yet as like dynamite it can either be used to create or destroy, which relates to matters of proper care and intent, rather than the fear of progress with the acquisition of knowledge.


”Fear is the main source of superstition, and one of the main sources of cruelty. To conquer fear is the beginning of wisdom, in the pursuit of truth as in the endeavour after a worthy manner of life.?”

-Bertrand Russell, “An Outline of Intellectual Rubbish” p-23 (1943).

Best,

Phil

P.S. One thing for the next time I eat an apple I will be certain to refrain from chewing the seeds.

Arun said...

Phil, what is troubling is, that if the story is true, it took 20+ years to find out that this grass can do that.

Phil Warnell said...

Hi Arun,

For me I find the fear of GM, as it is with many things these days, to be disproportional to what has been actually observed. I think a greater concern is that such fear is limiting our options at a time when we need more ingenuity in the world rather than less. I also find it curious that at a time when society is becoming increasingly secular, this hasn’t lessened the things that many people simply come to believe; that is including having many quickly assuming that one anomalous result in respect to SR should have all previous results to be disregarded.

Best,

Phil