Friday, February 21, 2014

The eternal tug of war between science journalists and scientists. A graphical story.

I am always disappointed by the media coverage on my research area. It forever seems to misrepresent this and forgets to mention that and raises a wrong impression about something. Ask the science journalist and they'll tell you they have to make concessions in accuracy to match the knowledge level of the average reader. The scientist will argue that if the accuracy is too low there's no knowledge to be transferred at all, and that a little knowledge is worse than no knowledge at all. Then the journalist will talk about the need to sell and point to capitalism executed by their editor. In the end everybody is unhappy: The scientists because they're being misrepresented, the journalist because they feel misunderstood, and the editor because they are being blamed for everything.

We can summarize the problem in this graph:



The black curve is the readership as a function of accuracy. Total knowledge transfer is roughly the amount of readers times the information conveyed. An article with very little information might have a large target group, but not much educational value. An article with very much information will be read by few people. The sweet spot, the maximum of the total knowledge transfer as a function of accuracy, lies somewhere in the middle. Problem is that scientists and journalists tend to disagree about where the sweet spot lies.

Scientists are on the average more pessimistic about the total amount of information that can be conveyed to begin with because they do not only believe but know that you cannot really understand their research without getting into the details, yet the details require background knowledge to appreciate. I sometimes hear that scientists wish for more accuracy because they are afraid of the criticism of their colleagues, but I think this is nonsense. Their colleagues will assume that the journalist is responsible for lack of accuracy, not the scientist. No, I think they want more accuracy because they correctly know it is important and because if one is familiar with a topic one tends to lose perspective on how difficult it once was to understand. They want, in short, an article they themselves would find interesting to read.

So it seems this tug of war is unavoidable, but let us have a look at the underlying assumptions.

To begin with I've assumed that science writers and scientists likewise want to maximize information transfer and not simply readership, which would push the sweet spot towards the end of no information at all. That's a rosy world-view disregarding the power of clicks, but in my impression it's what most science journalists actually wish for.

One big assumption is that most readers have very little knowledge about the topic, which is why the readership curve peaks towards the low accuracy end. This is not the case for other topics. Think for example of the sports section. It usually just assumes that the readers know the basic rules and moves of the games and journalists do not hesitate to comment extensively on these moves. For somebody like me, whose complete knowledge about basketball is that a ball has to go into a basket, the sports pages aren't only uninteresting but impenetrable vocabulary. However, most people seem to bring more knowledge than that and thus the journalists don't hesitate assuming it.

If we break down the readership by knowledge level, for scientific topics it will look somewhat like shown in the figure below. The higher the knowledge, the more details the reader can digest, but the fewer readers there are.


Another assumption is that this background level is basically fixed and readers can't learn. This is my great frustration with science journalism, that the readership is rarely if ever exposed to the real science and thus the background knowledge never increases. Readers don't ever hear the technical terms, don't see the equations, and aren't explained the figures. I think that popular science reporting just shouldn't aim at meeting people in their comfort zone, at the sweet spot, because the long-term impact is nil. But that again hits the wall of must-sell.

The assumption that I want to focus on here is that the accuracy of an article is a variable independent of the reader themself. This is mostly true for print media because the content is essentially static and not customizable. However, for online content it is possible to offer different levels of detail according to the reader's background. If I read popular science articles in fields I do not work in myself, I find it very annoying if they are so dumbed down that I can't make a match to the scientific literature, because technical terms and references are missing.  It's not that I do not appreciate the explanation at a low technical level, because without it I wouldn't have been interested to begin with. But if I am interested in a topic, I'd like to have a guide to find out more.

So then let us look at the readership as a function of knowledge and accuracy. This makes a three-dimensional graph roughly like the one below.


If you have a fixed accuracy, the readership you get is the integral over the knowledge-axis in the direction of the white arrow. This gives you back the black curve in the first graph. However, if accuracy is adjustable to meet the knowledge level, readers can pick their sweet spot themselves, which is along the dotted line in the graph. If this match is made, then the readership is no longer dependent on the accuracy, but just depends on the number of people at any different knowledge background. The total readership you get is the sum of all those.


How much larger this total readership is than the readership in the sweet spot of fixed accuracy depends on many variables. To begin with it depends on the readers' flexibility of accepting accuracy that is either too low or too high for them. It also depends on how much they like the customization and how well that works etc. But I'm a theoretician, so let me not try to be too realistic. Instead, I want to ask how that might be possible to do.

A continuous level of accuracy will most likely remain impossible, but a system with a few layers - call them beginner, advanced, pro - would already make a big difference. One simple way towards this would be to allow the frustrated scientist whose details got scraped to add explanations and references in a way that readers can access them when they wish. This would also have the benefit of not putting more load on the journalist.

So I am cautiously hopeful: Maybe technology will one day end the eternal tug of war between scientist and science writers.


32 comments:

Physics Today said...

What you're missing is time. You also have to factor in that to do a highly detailed piece which more closely matches what the scientists want and what the journalist wants (which is usually for readers to read more than the first paragraph) requires a lot of revisions, suggestions and reviews. To do all that is expensive and in a lot of occasions most places (such as newspapers) can't afford to do it. (I'm fortunate enough to work for an employer that does, but we're one of the exceptions along with the news dept at Nature and Science magazine).

Plato Hagel said...

Hi Bee,

I have an analogy in mind but that could be counterproductive....maybe?

I think when this communication technique is advanced toward internet for the general public there should be no noise to degrade the signal? Blogging can become much more direct.

Best,

Plato Hagel said...

Just a follow up.

"After finding financial backing, Outernet plans to ask NASA's permission to test its datacasting technology on the International Space Station. If that goes well, the company's cubesats could start launching as soon as June 2015. See: Ambitious 'Outernet' could bring unfettered Internet access worldwide via mini satellites"

Something has to happen outside the world governments(WO), to shake up the control on data caps. Can you own a plot of land on the moon? There is a treaty.

Just saying. :)

coraifeartaigh said...

Hi Bee. I'm doing quite a lot of interviews these days and one interesting thing I notice is that I find it easier in many ways to deal with journalists who have no science background at all!
I write out a list of questions and rough answers in advance of interviews and they pick out the ones they like..the less they vary from the script the better because it allows me to ad lib in the answers

Sabine Hossenfelder said...

Paul,

Yes, sure there's time. But look, if I talk to a journalist I have the details at hand and do my best to get them across. They just don't normally make it into the story. So I don't think it would take the scientist much time, which is why I'm saying why not give the scientist the opportunity to add some clarifications for those who are interested to read it. If it's online, it would be okay also to add it later.

See, those scientists who write blogs often do this on their blog. But a) that misses a big part of the readership of the article and b) not everybody whose research is in the news writes a blog.

Yes, I know change always takes money. But I do think that this might actually increase the readership after a transition period by increasing the target group. Physics today is already aimed at a specialized audience, it's not really what I had in mind. Best,

B.

Sabine Hossenfelder said...

coraifeartaigh,

Is the result a good one from your perspective? I actually think that science journalists have an important task, that of adding objectivity to a reporting that the scientist can't really claim to have. That, I'd think, requires at least some background. Best,

B.

Paul Guinnessy said...

Physics Today has a specialized audience I agree, in that 85% of its readership has a masters or higher in the Physical Sciences, the problem (from our perspective) is that they are all in different fields, so you have to be careful in not making assumptions that a particle physicists understands biology and vice versa. I don't think we do too bad a job at it, which is probably why a lot of our articles are used to teach physics in high schools and undergraduate university departments.

Our most popular article of recent years was "Chaos at Fifty" which is a good example of that (and the care we take in trying to make sure we meet the balance).

I've also met too many journalists without a background in science getting confused and putting the wrong spin on work, such as "oldest star in the universe found" for example.

Paul

hush said...

Shredder Online Chess levels are easy, average and hard.

Do you want maximizes readership?

Always, literally, always have three versions of the research online from which the reader can chose.

The real challenge comes original sources - the free access to other grand masters players o, in this case, the original papers.

Uncle Al said...

Your illustrations answer the question. The curve peaks not slumps at constant scale at high temperatures. "Quantum Foams Reel in String Theory!" LuboŇ° will go ballistic. Publicly argue in high scholarship and poor taste. Loathsome social media will swim in it. Details are irrelevant, hence affordable when accurate. Our White House Obamanation farts rainbows while ignoring concomitant solids deposition.

"Quantum physicist revolutionizes gravity while nursing her twins" LuboŇ° powers are formidable, but he cannot compete within that gauge transformation. (Test assumed vacuum geometry against enantiomorphic atomic mass distribution geometries re chiral EP violation.)

George Musser said...

Our present system DOES have levels, shading from Physics Today and Nature N&V to the daily papers. In universities, we have everything from graduate seminars to physics for poets. Do you not think this system is working? If not, what about it is failing?

I'm struck that you keep mentioning terminology as important. I think this is definitely a difference between science writers and researchers. Researchers tend to see education as professional preparation, in which case you do need jargon and technique. But journalists are catering to an audience of dabblers.

I find that it is often the SCIENTISTS, not the journalists, who push the knowledge curve toward the left. They preemptively dumb down the discussion, sometimes because they think the public can't handle it, and sometimes because they themselves haven't done the hard interpretive work of their own theories. They say things like, "It is as it is because the equations say so", rather than attempt to spell out the physical principles. Or they say, "This work has implications for X", and when I ask what the implications are, they can't tell me.

Michael said...

The conflict between science journalists and scientists is much smaller than it appears in my point of view.

I see two big trends, both have been heavily supported by the WWW during the last 20 years. One trend is the increasing childishness, which is satisfied and increased by parts of the press, private radio and tv, and especially the internet as cheap medium for mass entertainment.

The other trend is that educated people can reach high quality information without the journalist as filter and mediator at no cost in principle. Every internet user can access almost the same sources like a journalist (press releases, outreach material) that also point to the scientific sources. Even those could be accessed with a bit engagement in libraries at low cost (in principle only the ride there).

Though journalists (and editors) would have to add noticeable value to the emitted material (SciAm and SdW did this some 40..20 years before in an excellent way), or they have to orient themselves to readers that do not feel an overwhelming need to work hard for information and knowledge rather expect them to be served.

Summarized, I see that readers of every level will find their world to dive in into science, and scientists have nowadays all possibilities to complement where they see an imperfect landscape. The world is in fact better than in the past.

L. Edgar Otto said...

There is a big difference between creative writing and creative accounting. Science and the world is so much richer when you can understand both as a few of you do. Communication grounded in theoretical inquiry is indistinguishable from teaching in a world where inquiry is possible. As far as this poetic scientist or whatever I am or do, this blog and commenters, links and even detractors of questionable assumptions or foresights, is the center and current primary source of science news. We are poised well to get past nothing more to say for now so perhaps find the way to new concepts and ideas.

Giotis said...

In this case I’m not sure who are worse, the scientists or the journalists. I blame scientists as the inventors of the stupid misleading metaphors, journalists for the eye catcher misleading titles of their articles and for not pushing the scientists they interview hard enough for a better understanding of the actual science.

More than often scientists are arrogant and treat readers (and journalists) as morons. They want to attract attention to their work but they are lazy, not willing to spend time explaining things properly. In their mind this is a waste of time.

That being said this is a difficult subject anyway. Especially for theoretical physics it is very hard to convey it to the general public; readers, journalists and scientists need to do their homework.

Nemo said...

The worst thing I see too often happen, in particular in popular reports about theoretical/fundamental physics is that the science writers and/or editors try to maximise the size of the audience at the expense of reporting true facts and information about too fundamental topics such as quantum gravity for example.

Sometimes, the media outright promote so-called controversial articles, blindly cite (sometimes even dishonest) wannabe experts with an agenda, confused outsiders to the topic thei want to report about such as philosophers, etc instead of talking to and representing what real experts have to say about the topic. The emerging flame wars and too often dominating trolls in the comments below such "spiced up" popular reports are simply abominable.

This is why I have completely stopped reading articles about fundamental physics in the popular media, unless an acceptable report is pointed out by for example a blogging expert I trust.

I often think, no popular reports about fundamental physics would be better for science and the scientists than what is currently done in newspapers, popular magazines (including Nature and Scientific American !). If they are not able to do it right and/or more interested in instigating controversies to increase the audience, they should simply not report about such topics!


Robert L. Oldershaw said...

Here is a real example of a dumbed down and over-generalized news story that badly misled many people.

Short version: an Australian group used a new technique to test for cosmological fractality and found none of a specific type over a limited scale range. Many pop-sci venues picked up on this single paper and announced that "the Universe is not fractal!"

The more truthful version is quite different.
Here is the main conclusion given in the abstract of the Australian paper, which some misguided individuals have used as evidence for the incorrect conclusion that fractal geometry is not a fundamental property of the Universe.

"We can exclude a fractal distribution with fractal dimension below D_2=2.97 on scales from ~80 Mpc/h up to the largest scales probed by our measurement, ~300 Mpc/h, at 99.99% confidence. "

1. The methods they use to arrive at this conclusion have a fair amount of uncertainty and need to be carefully checked and repeated by other astrophysicists.

2. More importantly, they explicitly say that they only question a very limited fractal model with (A) D_2 less than 2.97, and (B) only for the puny scale range of 80-300 Mpc, and (C) only for CONTINUOUS fractals.

This paper cannot be used to make the very general claim that all fractal cosmological models are ruled out. That would be a false and unscientifically misleading conclusion, which the authors DO NOT MAKE. If science journalists and bloggers draw such an erroneous conclusion, then it is "their bad". It's sensationalism - not science!

One discrete fractal cosmological paradigm that I know quite a bit about (A) has D-2 = 3.174, (B) predicts inhomogeneity way beyond 80-300 Mpc, as was found by Clowes et al (1240 Mpc) and Horvath et al (2000-3000 Mpc), and (C) is a DISCRETE fractal model, not a continuous fractal model.

What the authors of the Australian paper were concluding was vastly less general and important than what science journalists and pop-sci bloggers turned it into.

I think this is unacceptably misleading, and the trend is very common. There is no real excuse for this type of disinformation.

Michael said...

And here is one more example where a scientist creates inaccuracy in the popular science presentation of his field:

"The redshift phenomenon [of distant galaxies] is a manifestation of the Doppler effect - the faster the motion, the larger the shift of the frequency. Therefore, the larger the redshift, the greater the distance to the observed galaxy."

:-o Every undergraduate student in a cosmology course has to learn that this redshift is an effect of the expansion of space. This also could be written in a popular science article, couldn't it?

http://theconversation.com/the-measure-of-the-universe-through-doppler-lensing-23228

L. Edgar Otto said...

Michael,
This is not obvious to me and there are papers recently suggesting space is not actually expanding.
If what every undergrad knows is how you say how would that be news and not just a review of what is taught?
And where are comments on Sabine's graphic method here? The masses crave the content perhaps as distractions and some do benefit as I have done following new scientist for decades. It helps us think objectively. I find little joy in just working out an obvious cross word puzzle passing time but a great deal of joy seeing one offered those few times that were impossible - and respect when the next issue admits this in a retraction. But as always as Edison knew, "Rumors of gold are more valuable than gold itself " then went on to invent the Teletype.

Sabine Hossenfelder said...

George,

No, I don't think the system is working. Yes, there are levels, I agree. What is missing is to help those who are interested making a transition between the levels. That's why I keep talking about introducing readers to terminology.

"I find that it is often the SCIENTISTS, not the journalists, who push the knowledge curve toward the left. They preemptively dumb down the discussion, sometimes because they think the public can't handle it, and sometimes because they themselves haven't done the hard interpretive work of their own theories. They say things like, "It is as it is because the equations say so", rather than attempt to spell out the physical principles. Or they say, "This work has implications for X", and when I ask what the implications are, they can't tell me."

A big problem in the communication between scientists and journalists is that the journalists never tell what their own background is. Unless I know better (as eg in your case) I assume they don't know nothing about nothing. Have you tried to start with telling them what degree you have and roughly what you already know? This might avoid your impression that they're dumbing it down too much.

About some scientists not being able to answer questions, I don't think that's a communication problem, they probably just never thought about it before and quite possibly they're not even interested in the answer. I can see that this must be annoying... Best,

B.

Sabine Hossenfelder said...

Michael,

Same remark to you as to George. Yes, readers can find their level, but it is very hard to get from one to the other. There is presently a huge gap between popular science and academic literature. At least in physics, in this gap there sit the membership magazines of the physical societies, but most people don't know them or it doesn't occur to them they exist or they're subscription only etc. I also suspect (hard to say) that even the membership magazines are hard to read without a degree.

See, the average level of knowledge about how modern science works will only change if people can climb up the levels. And that's what presently works very badly. Many people get stuck on the pop sci level just because they have nowhere to go from there. That's what my blogpost is about. Best,

B.

Sabine Hossenfelder said...

Giotis,

Yes, I agree with you, there's problems on both sides. Though I want to add it's not all bad. There's lots of great writing out there and some scientists are very good communicators. Best,

B.

Sabine Hossenfelder said...

Nemo,

I agree with you on the problem, but I don't think the solution can be no popular reports on fundamental physics at all. I think it is part of our profession to share our knowledge, and to share it not only with our colleagues but with the societies that we are part of and that support us.

I think what will happen (and you can already notice this happening) is that scientist have better access to funding of public outreach and training for how to make it work exactly because it is in the interest of the taxpayer. This will, in the long run, make science communication a CV skill with its own qualification scheme. Best,

B.

Sabine Hossenfelder said...

Michael,

Your redshift example just drives home my point that simplifying terminology can cause much confusion. As you probably know (it sounds like you do) the expression for the cosmological redshift (or that in the gravitation field of the Earth) is very similar to that of the usual Doppler effect and I can see that a scientist, not being able to just write down the damned equation, refers to it with a terminology that people know in different context from school.

George above complains that scientists sometimes just refer to equations that 'say so'. Of course they do. Because they have learned that equations are the only accurate way to do it. The downside is if you get used to this then translating it back into words is difficult.

See, the system with layered levels of accuracy that I suggest in my blogpost would essentially eradicate this problem. Best,

B.

t h ray said...

17Hi Bee,

Congratulations on reaching your milestone 8th year!

As a former journalist a long time ago (the 70s)and now almost set to retire from my third career as a technical writer-editor, my sympathies are with George Musser.

For me, reporting results from a major research university (only part of my "beat") for commercial mass media was usually a matter of communicating with the public relations department -- same for news from government scientists. For nonscience media, this is usually sufficient, on the premise that public knowledge of the subject will be rare, and the story is not much more than a headline.

For science media, I think George is dead on that one cannot assume the same audience for Scientific American or New Scientist as for Science or Nature, though there is bound to be overlap. One is expected to have a background in the subject when writing for a professional journal -- yet I think you are being a bit unfair when questioning the backgrounds of journalists in the commercial press. Their background should be assumed to be journalism, a field with its own professional journals (e.g., Columbia Review of Journalism) and standards. Yes, I know the field can stand a lot of improvement, or I would still be doing it.

I think you identified the solution that works across the board for curious individuals (which almost all scientists are) who read both commercial and science media: "I find it very annoying if they are so dumbed down that I can't make a match to the scientific literature, because technical terms and references are missing."

Me too!

Best,

Tom

Michael said...

Sabine,

you mentioned the missing bridges between the popular and scientific levels and the support for moving between them in two comments. A remark about that follows here.

Initially I also expected such input from products of science journalists or blogs. However, this has been brought down to earth very soon.

Newspapers and magazines reduced for years the amount of text in favour of more and larger pictures. As science did not become easier in the mean time, and language will be needed also furthermore in order to explain it, I do not see much help from there.

Blogs. There are excellent blogs that discuss current as well as fundamental topics. They are exempels of serious occupation with science and provide inspiration. In fact they are excellent windows into science. But for potential level-changers, I see the typically non-systematic style of blogs as a kind of shortcoming.

What remains for this goup of people? You can probably start from really good popular science books and the "easier research journals" that you mentioned. But depending on how far out one wants to crawl, one has to work to get a sound basis (esp. mathematics) and to accept that the scientific literature is the means to an end. So, it becomes again books, pen, and paper.

Hugh Matlock said...

Sabine,

A science journalist often reports on a result that has just been published in a journal. A simple solution in this case is for the journalist to provide a link to the journal article. The reader can then easily follow up for details (especially if the journal is Open Source).

I do not know why this is not yet standard practice... but perhaps publishers think any non-monetized link taking you away from their site is a bad thing.

Hugh

Sabine Hossenfelder said...

Hugh,

Yes, that should be done and I find it very annoying if it isn't the case. But my point is that most readers won't be able to understand the paper anyway. Best,

B.

George Musser said...

OK, so the problem, in your view, isn't the lack of levels per se, but the difficulty of jumping from one level to the next. Sci Am does do this to a limited degree by introducing common terminology and providing a readings list. Years ago I had an idea (but sadly I couldn't convince my colleagues) that the Sci Am website should have a slider for difficulty level, so readers could choose their desired level. You can achieve something similar with "tracks" (as MTW and other textbooks do) but it'd be nice to suppress the undesired levels dynamically. The scheme would have involved XML-tagging each paragraph for difficulty. Writers would actually like this a lot, because perhaps the most difficulty part of writing is knowing what to put in and what to leave out, and with tags we could suppress the text for some readers while revealing it to others.

One point to remember is that science and learning about science should be social activities. A single article can't possibly reach all levels, but a group of people (like a journal club or Cafe Scientifique) can choose a variety of readings and work through them together.

Sabine Hossenfelder said...

George,

Yes! That's exactly what I mean :) In fact it seems you've had pretty much the same idea years ago, plus you've thought about the technological necessities.

I disagree though that science and learning about science 'should be' social activities... I agree that this seems to work very well for many people, but I know many scientists and people interested in science who don't like anything that has the word 'social' attached to it. Best,

B.

Chris Clarke said...

A correction to a statement in a previous comment: to a first approximation, journalists do not write the titles that are attached to their articles. Some do, but it's not safe to assume. My editor at KCET runs headlines and titles past me to make sure he's not misrepresenting things, but he is a rarity.

Peter Byrne said...

If scientists could all write as well as Sabine, there would be no need for science writers. Alas, with some notable exceptions, most scientists are semi-literate and write only for their jargonated peers, and then they typically only express their own agenda-laden point of view. We journalists look at the broader picture and hold scientists accountable. In the best of all possible worlds, journalists are the Third Eye ;-)

Robert L. Oldershaw said...

Another example of inadvertent misinformation in a science venue - this time from sciencedaily.com.

The headline is: "Most Red Dwarf Stars Have At Least One Planet"

Actually, planets are reasonably common for red dwarf stars with masses =/> 0.4 solar mass.

BUT, planets are unusually and unexpectedly rare for red dwarf stars with masses between 0.1 and 0.25 solar mass. See Bonfils et al, Astronomy and Astrophysics, 2012, http://arxiv.org/abs/1111.5019 .

Only one theory has ever definitively predicted this anomaly. Can you guess which theory?

Robert L. Oldershaw said...

Dr. Tuomi and I have discussed this disinformation problem and he agrees that it is regrettable but that there is little a research group can do when the science and pseudo-science media get hold of a research paper and then misinterpret its conclusions by making sensational and false claims.

Let the reader beware. Even a supposedly scientific magazine like Astronomy is claiming that "All Red Dwarf Stars Have At Least One Planet"!

Does increasing readership justify publishing sensationalist disinformation?