By Cornelia Dean
Belknap Press (March 13, 2017)
It’s not easy, being a science journalist. On one hand, science journalists rely on good relations with scientists. On the other hand, their next article may be critical of those scientists’ work. On the one hand they want to get the details right. On the other hand they have tight deadlines and an editor who scraps that one paragraph which took a full day to write. That’s four hands already, and I wasn’t even counting the hands they need to write.
Like most scientists, I used to think if I see a bogus headline it’s the writers’ fault. But the more science writers I got to know, the better my opinion of them has become. Unlike scientists, journalists strongly adhere to professional guidelines. They want to get things right and they want the reader to know the truth. If they get something wrong, the misinformation almost always came from scientists themselves.
The amount of misinformation about research in my own discipline is so high that no one who doesn’t work in the field has a chance to figure out what’s going on. Naturally this makes me wonder how much I can trust the news I read about other research areas. Cornelia Dean’s book “Making Sense of Science” tells the reader what to look out for.
Cornelia Dean has been a science writer for the New York Times for 30 years and she knows her job. The book begins with a general introduction, explaining what science is, how it works, and why it matters. She then moves on to conflicts of interest, checking sources, difficulties in assessing uncertainty and risk, scientific evidence in court, pitfalls of statistical analysis and analytical modeling, overconfident scientists, and misconduct.
The book is full with examples, proceeds swiftly, and reads well. The chapters end with bullet-point lists of items to recall which is helpful if you, like I, tend to sometimes switch books half through and then forgot what you read already.
“Making Sense of Science” also offers quick summaries of topics that are frequently front-page news: climate change, genetically modified crops, organic food, and cancer risk. While I have found those summaries well-done they seem somewhat randomly selected. I guess they are mostly there because the author is familiar with those topics.
The biggest shortcoming of the book is its lacking criticism of the scientific disciplines and of journalism itself. While the author acknowledges that she and her colleagues often operate under time pressure and shit happens, she doesn’t assess how much of a problem it is or which outlets are more likely to suffer from it. She also doesn’t mention that even scientists who do not take money from the industry have agendas to push, and that both the scientists as well as the writers profit from big headlines.
In summary, I have found the book to be very useful especially for what the discussion of risk-assessment is concerned, but it presents a suspiciously clean and sanitized picture of journalism.
Hallo Mrs. Bee,
ReplyDeleteYou are using "scrapes" where i think you mean "scraps".
Rgds.
How do scientists spread wrong information? By giving explanations that are too coarse, in order to be understandable? Or by stating their victories with such enthusiasm that the picture they give is way off?
ReplyDeleteThanks, papa.
ReplyDeleteThank you Dr. H. a nice review. You might consider replacing 'scrapes' with 'scraps'. The first is plural of scrape (like scraping ice off a window) while the latter is plural of scrap which is to throw away or something to be discarded..
ReplyDeleteHope you are well
Matthew,
ReplyDeleteI changed that, but why is it a plural? It's a third person singular: "The editor scrap(e)s"
papa, Matthew,
ReplyDeleteI couldn't find it in my dictionary, but the Wiktionary for all I can see agrees with me that the third person singular of "to scrape" is "he/she/it scrapes". So I changed it back. Do you have evidence in defense of "scraps"?
Thank you Hossie for being what every science journalist should be: brave, honest, fierce, insightful and outraged by those who fall short. And it's not even your day job!
ReplyDeleteMaarten,
ReplyDeleteThey overstate the relevance of their findings and omit relevant information. I could list you dozens of examples for this. It results in headlines according to which the LHC will test string theory (that was never going to happen) and supersymmetry (all by itself an unfalsifiable idea) or that we live in a multiverse (we don't know and will never know) or that great progress has been made in quantizing gravity (unfortunately not in our universe).
I'm not just basing this on my own experience. There was a paper recently that tracked back misinformation in science news and found that in most cases it came from press releases. Now, these are written by some public relation people, but they almost always okay this with the researchers. And it makes sense. You see, everyone along the line - the researchers, the PR people, the writers - benefits from exaggerated claims.
I can dig up the reference if you're interested, I don't have it at hand right now.
Re: scrap/scrape. Wiktionary also gives the meanings for the two words https://simple.wiktionary.org/wiki/scrape and https://simple.wiktionary.org/wiki/scrap Basically, scrap is the word you're looking for, having the meaning of "get rid of", whereas scrape doesn't.
ReplyDeleteWhen all Media were local, American vaudeville performers perfected acts, town to town, over years. Radio then TV crushed that. The Internet is moment to moment. Industrial and academic research embraced decades. Now, every snippet is morning news promising to transcend the the limits of mankind (though more studies are needed).
ReplyDeleteSeduction was smothered by (faked) flashy orgasms. PIs don't touch Pyrex. PIs push administrative paper and pander publicity. Empirical discovery offends The War on Drugs, the War on Terror, the EPA, OSHA, ATF...activists, ethicists, religious "leaders," administrators, audits...Human Resources (the self-propagating sludge pit of mandatory diversity).
The universal fear is that something accomplished will crash the business model. Ask Elsevier about that.
Pontus,
ReplyDeleteThanks, that explains it! Right conjugation, but wrong verb. (So we are back to "scraps".)
One of the most extreme examples of a science news item with overstated relevance was the "Faster-than-light neutrino anomaly" (in the OPERA experiment) back in 2011. In the news (at least the media in my country) presented the finding as "maybe FTL is real, maybe not, no one knows". The context - an entire branch of science, a century of evidence against the possibility of FTL, all of the reasons to suspect a mistake - was entirely lost.
ReplyDeleteWhat saddens me most is not just the blown up news report itself, but the lack of attention that some really discoveries are getting, just because they don't sound as spectacular.
Sabine, I think the following aphorism describes the basic problem faced by any self-critical system, be it journalism or science, or simple self-examination.
ReplyDeleteAny system that claims to be able to evaluate itself accurately can only do so by cutting itself off from all other systems. This is otherwise known as circular logic.
It is a good part of the reason that flat-out lies (including to oneself) can gain so much traction.
The book wasn't written by a system, it was written by a person.
ReplyDeleteI am not saying that self-examination serves no purpose, quite the opposite. It’s just that no system or person working solely from inside itself can eliminate all bias or mistakes (illogic).
ReplyDeleteThey can. They just can't prove they did.
ReplyDeleteHilarious!!! ;-)
ReplyDeleteDr. H,
ReplyDeleteYeah, I expected that answer. I'm glad to see more details of this annoying phenomenon you describe. There are very many scientists, I hope, who want to be truthful, but the headlines of course tend to go to the other kind of scientists. Alas, I'm afraid every single interested person will need to learn somehow what's scientific content and what isn't. Math is nice because usually 'big discoveries' don't make such headlines there.
There are also biases in media coverage induced by the self-selection of who chooses to specialize in topic areas. Even other journalists have long noted the green zealotry of "environmental journalists:"
ReplyDeletehttp://niemanreports.org/articles/environment-journalists-dont-get-much-respect/
You said, "The chapters end with bullet-point lists of items to recall which is helpful if you, like I, tend to sometimes switch books half through and then forgot what you read already."
ReplyDeleteAnd that is one feature that I really liked about your book. And, the Introductory remark to each chapter.
Dr. H. you are right re the singular plural distinction as these are verbs not nouns. But the word you want is still 'scraps' to discard. Scrape means something completely different.
ReplyDeleteNice post.
ReplyDeleteTrue self-examination does require "stepping outside" of one's self. It is hard, but it is doable with the assistance of others. The problem is that few actually try; the results are often unlikeable.
Cornelia Dean is paid to write, so she wrote something. Not the deepest, most thoughtful thing, but something. One can't be Chaucer or Cervantes every day.
sean s.
An oldie-but-goodie:
ReplyDeleteThe Science News Cycle
http://phdcomics.com/comics/archive_print.php?comicid=1174
SteveB,
ReplyDeleteThe chapter summaries were my editor's suggestion. Or rather, she complained that it sometimes wasn't clear what she should take away from a chapter, so I added those. Proof that sometimes editors are good for something ;)
@sabine,
ReplyDelete"misinformation in science news ... in most cases it came from press releases".
This is how the Cold Fusion saga began. I wonder if you have ever researched that and have some thoughts about it.
...don't mind them Sabine; "what little things excite them most are a mere reflection of their limits"
ReplyDeleteyou can scrape me anytime ... not that I am into that kind of stuff ... oh dear god stop typing ... ... ...
In other parts of society there are increasing concerns that decisions are being influenced adversely by an increasing reliance on Computer Models especially Algorithms, whose justification can be open to much doubt. As an example it is possible that some financial sector crashes are caused or initiated by that.
ReplyDeleteSo this brings out the question as to whether there has been an undercurrent of believing computer models too much in areas of scientific theorising. Admittedly the scientists are meant to understand their computer models, but once a lot of effort has gone into a model it must become very hard to question it.
I've always held that ruthless application of the dialectic is the closest we can come to self-examination and self-correction. The hard trick lies in getting one's own ego out of the way. Practice helps.
ReplyDeleteThink of it as a kind of therapy, the shrink asking hard questions and not taking any hand-waving excuses in reply. It does take some courage.
It's like any form of fitness: you have to want it.
Writers have to learn to throw away words they worked hard to find and cherish. Scientists need to learn to throw away bad ideas. (Multiverses,... lookin' at you.)
But often the hardest things to throw out are one's own ideas.
Bob Whalen said:
ReplyDeleteDear Dr. Hossenfelder, many decades ago I read J.W.N.Thompson "The Limitations of Science" (1939) when it was still in print. Since I am no scientist I found broad explanations for lay people useful in trying to appreciate science. Then, in the 1950s, I became enamored of Dr. Fred Hoyle and his "Steady State Universe." It seemed to violate the conservation of mass and energy but so what? During my early middle age Drs. Penzias and Wilson (Bell Labs) seemingly provided proof that Dr. Hoyle was wrong -- the universe had a specific origin. Recently (the last few days) I read reports of the LHC being scheduled to test theories that suggest a "Big Bounce" universe. If "proven," this last hypothesis would pull us even with the ancient Hindus -- who even got the timing about right.
Many reputable authors, from John Horgan ("The End of Science:) to your recent "Lost In Math" warn readers that cosmologists have, for over a generation, been on a wild goose chase, what with Suzie, String Theory and the Multi-Verse. And, as I mentioned, now the Big Bang is to be reassessed.
The stock response might be "That is how science works." Well, it seems to be how cosmology works.
I wonder if you weren't a little curt with that reader who suggested that "systems" (she meant science) could not usefully evaluate themselves. After sixty-seventy years of befuddled but sincere interest in cosmology I have begun to wonder whether trying to be "objective" about the universe isn't akin to watching a live play from the audience while wanting to be up on the stage performing it.
As an experienced scientist of high standing are you confident that the origin and destiny of the universe and the nature of Reality are subject to definitive answers from science? If so, wonderful! If not -- what is the point?
As for being 42 and depressed,you are about on schedule. To fully understand futility, however, one needs about another forty years. Which is where I am at.
Thank you for your wonderful prose (is English a first language?) and kind attention to us liberal arts types.
Unknown,
ReplyDeleteThe task of the scientific system is not to evaluate itself, its task is to evaluate the fitness of hypotheses to describe nature. It is right that people within the system are not in a good position to evaluate how well the system is doing, but people can have several roles, only some of which are in the system. The system isn't defined by specific people, it's defined by its structure and the rules that the people in the system interact through.
My above comment merely stated what's rather obvious, namely that someone within the system may be able to eliminate their bias, but they'll never know they did and can't prove they did. This merely reflects the common-sense knowledge that you shouldn't trust people to judge their own kind.
English is my second language.
Sabine,
ReplyDeleteI think that's an "Ooopsie"! If we can't trust the judgement of people within the field, and we can't trust the judgement of people outside the field (because it can be said they don't understand it), where does that leave us?
Many years ago, in Laws of Form, G. Spencer Brown wrote:
• Let us consider, for a moment, the world as described by the physicist. It consists of a number of fundamental particles which, if shot through their own space, appear as waves, and are thus... of the same laminated structure as pearls or onions, and other wave forms called electromagnetic which it is convenient, by Occam’s razor, to consider as travelling through space with a standard velocity. All these appear bound by certain natural laws which indicate the form of their relationship.
• Now the physicist himself, who describes all this, is, in his own account, himself constructed of it. He is, in short, made of a conglomeration of the very particulars he describes, no more, no less, bound together by and obeying such general laws as he himself has managed to find and to record.
Thus we cannot escape the fact that the world we know is constructed in order (and thus in such a way as to be able) to see itself. This is indeed amazing.
Not so much in view of what it sees, although this may appear fantastic enough, but in respect of the fact that it can see at all.
• But in order [for the universe to see itself], evidently it must first cut itself up into at least one state which sees, and at least one other state which is seen. In this severed and mutilated condition, whatever it sees is only partially itself. We may take it that the world undoubtedly is itself (i.e. is indistinct from itself), but, in any attempt to see itself as an object, it must, equally undoubtedly, act so as to make itself distinct from, and therefore false to, itself. In this condition it will always partially elude itself.
What's your take on the possibility of corrective systems if they are inherently incomplete (divided into pieces that can't see themselves clearly)?
Lying,
ReplyDelete"If we can't trust the judgement of people within the field, and we can't trust the judgement of people outside the field (because it can be said they don't understand it), where does that leave us?"
You are confusing two different things. People outside the field can (and do) judge whether the academic system is configured so that they can trust the judgement of people inside the field about the value of hypotheses. For this you don't need to understand those hypotheses (or their tests), you merely need to understand how the system works.
Sabine, we are both cheering for the same team. Science needs to clean up its act. The issue is what has got it so screwed up.
ReplyDeleteLet me try again. There are hierarchies of evaluative techniques. Useful evaluation is not a flat space. Part of the method for making a useful evaluation is to select the correct level from which to make it. Many of those levels involve feedback loops that work at a distance from the object being evaluated. Others involve a simultaneous examination of the appropriateness of the evaluative technique (including its level) at the same time a primary evaluation is being made. That whole selection process is a somewhat arcane and largely unrecognized skill in itself. But when it works, it works pretty well. Lately the current “system” has stopped working, for the most part.
You suggest that scientists can wear more than one hat, but you also express rightful doubt about the trustworthiness of their conclusions when making such evaluations. The problems that arise with many evaluations are due to the fact that evaluation is a skill, every bit as difficult to learn as math or physics. A scientist who has not spent the time and energy needed to develop the skill into a reliable tool will always incorporate a bias, whether the person making an evaluation is aware of it or not. A large part of developing that skill is learning how to spot and then overcome one’s bias in a process that is similar to learning how to use logic. I have never heard of a requirement for a background in logic to become an administrator.
The “system” as you describe it in your book is not working and it is not formal nor is it rigorous, also well described in your book. One of the great struggles of science, and especially so in the hard sciences, is to find and eliminate factors that confound the study of the subject. That struggle can never end because it involves people and their prejudices. Messy and as non-formal as they are, people are systems too, both individually and collectively. People "systems" are inextricably involved in all endeavors.
The people you mention “outside” of a field are just as subject to bias as anyone inside it. And the biases supplied by the people who evaluate scientific efforts show in the lack of meaningful scientific results in a variety of fields, which you bemoan.
When you wrote “… you merely need to understand how the system works” it forces me to ask which system you are referring to. The “judgments” arrived at by academic systems are clearly failing, as shown by many of the experiments selected for testing (funding). That is, there is a “system” for coming up with ideas to test, and there is another “system” for selecting some of them. And because they both depend on the people "system", aren’t they nearly the same system and aren’t they about equally unreliable? Thus I am asking you what it is I am confused about. Which systems works?
Lying,
ReplyDeleteThe system I am referring to is the academic system, that includes both hypotheses-development and test. No, I do not say that scientists can step outside the field, I say that people can be scientists and non-scientists, not necessarily at the same time.
Of course people outside the field are also subject to bias. Everyone is. As I write in my book, we'll never get rid of bias entirely (of if we do, we wouldn't be able to prove it, see earlier remark), and the struggle will never and, etc etc. But that's not a reason to not try and eliminate known biases.
Bob Whalen wrote:
ReplyDeleteDear Dr. Hossendelder, I just now read your review and thank you. One small query: Can there actually be "randomly selected" ideas in a superdeterministic universe? :)