Thursday, October 31, 2013

Science Marketing needs Consumer Feedback

It’s been a while since I read Marc Kuchner’s book “Marketing for Scientists”. I hated the book as I’ve rarely hated a book. I did not write a review then because it was a gift from an, undoubtedly well-meaning, friend who reads my blog. As time passed though, I changed my mind. Let me explain.

Product advertising and marketing is the oil on the gears of our economies. Its original purpose is to inform the customers about products and help them to decide whether they fit their needs. But marketing today isn’t only about selling a product, it’s also about selling a self-image. What we decide to spend money on tells others what we consider important and which groups we identify with.

In the quest to attract customers, advertisements often don’t contain a lot of information, and sometimes they bluntly lie. And so we have laws protecting us from these lies, though their efficiency differs greatly from one country to the next as Microsoft learned the hard way.

Everybody knows adverts make a product appear better than it is in reality, that microwave dinners never look like they do on the images, lotions won’t remove these eye bags, and ergonomic underwear will not make you run any faster. The point is not that advertisements bend reality but as that advertisements work regardless, just by drawing attention and by leaving brand names in our heads – names that we’ll recognize later. The more money a company can invest into good advertisement, the more likely they are to sell.

It isn’t so surprising that capitalistic thought is increasingly applied not only to the economy but also to academic research. Today, tax-funded scientists, far from being able to dig into the wonders of nature unbiased and led by nothing but their interests, are required to formulate 5 year plans and demonstrate a quantifiable impact of their work. And so scientists are now also expected to market themselves, their research and their institution.

Scientific knowledge however isn’t a product like a candy bar. A candy bar isn’t right or wrong, it’s right for you or wrong for you, and whether it’s right or wrong for you depends as much on you as on the candy bar. But the whole scientific process works towards the end of objective judgment, towards finding out whether a research finding should be kept or tossed. Scientific knowledge is eventually either right or wrong and academic research should be organized to make this judgment as efficiently as possible.

Marketing science is not helpful to this end for several reasons:
  • It puts at advantage those who are either skilled at marketing or who can afford help. This doesn’t necessarily say anything about the quality of their research. It’s not a useful selection criterion if what you are looking for is good science. Those who shout the loudest don’t necessarily sell the best fish.
  • Marketing of science advertises the product (research results), while what people actually want to sell is the process (the scientist’s ability to do good research). It draws attention towards the wrong criteria.
  • It has a positive feedback loop that gradually worsens the problem. The more people advertise their work, the more others will feel the need to also advertise their work as well. This leads, as with advertisement of goods, to a decrease of objectivity and honesty until it eventually nears blunt lies.
  • It takes time away from research, thus reducing efficiency.
In quantum gravity phenomenology, you will frequently see claims that something has been derived when in fact it wasn’t derived, or that something is a result, when in fact it is an ad-hoc assumption. I am aware of course, such exaggerations are advertisements, made to convince the reader of the relevance of a research study. But they’re not helpful to the process of science and even worse for science communication.

That’s why I hated Kuchner’s book. Not because his marketing advice is bad advice, but because he didn’t consider the consequences. If all researchers had Marc Kuchner’s “sell yourself” attitude, we’d end up with a community full of good advertisers, not full of good scientists. It’s the inverse of the collective action problem: A situation in which we would all benefit from not doing something (advertising), but each individual would put themselves at a disadvantage when behaving differently (not advertising), and so we all continue to do it.

Here’s why I changed my mind.

Researchers market and advertise because they have to, owing to the very real pressure of the collective action problem. There are too many people and not enough funding. Marketing might not be a good factor to select for, but standing out for whatever reason puts you at an advantage. The more people know your name, the more likely they’ll read your paper or your CV, and that’s not a sufficient, but certainly a necessary condition for survival in academia. And then there’s people, like Kuchner, who make money with that survival pressure. Sad but true.

Yes, this is a bad development, but collective action problems are thorny. Complaining about it, I’ve come to conclude, will not solve the problem. But what we can do is work towards balance. What we need then is the equivalent of customer reviews and independent product tests –  what we need is a culture that encourages feedback and criticism.

Unfortunately presently feedback and criticism on other people’s work is not appreciated by the community. Criticism is typically voiced only on very popular topics, when even criticism on other’s work is advertisement of one’s own knowledge, think climate change, arsenic life, string theory. But it’s a very small fraction of researchers who spend time on this, and it’s only on a small fraction of topics. It’s insufficient.

A recent nature editorial notes that “Online discussion is an essential aspect of the post-publication review of findings” but
In recent years, authors and readers have been able to post online comments about Nature papers on our site. Few bother. At the Public Library of Science, where the commenting system is more successful, only 10% of papers have comments, and most of those have only one.”
This really isn’t surprising. Few bother because in terms of career development it’s a waste of time.

In contrast to the futile attempt of preventing researchers from advertising themselves and their work however, the balance can be improved by appreciating the work of those who provide constructive criticism. By noting the community benefit that comes from researchers who publicly comment on other’s publications, by inviting scientists to speak not only for their own original work, but for their criticism of other people’s work, and by not thinking of somebody as negative who points out flaws. Because that consumer feedback is the oil on the gears that we need to keep science running.

19 comments:

Thomas Larsson said...

"Independent product tests" = experiments.

Rule out supersymmetry almost certainly, and all other kinds of BSM physics much stronger than that.

Companies that score poorly in independent product tests tend to obscure that fact. Not surprisingly, theorical physicists tend to behave in the same way.

Uncle Al said...

http://luxdarkmatter.org/papers/LUX_First_Results_2013.pdf
arxiv:1310.8214; 1306.5534, 1306.3983

Fire them for ruining expansively indeterminate theory. Rescue dark matter with Yukawa potential alpha-lambda fraud. Meanwhile, 26.8 mass-% of the CMB universe vanished on 30 October 2013. The Tully-Fisher relation lost its Tully-Fisherinos. Do a geometric Eötvös experiment to source MoND's Milgrom acceleration through Noetherian leakage of vacuum isotropy. If physics cannot understand it, chemistry does. Do it.

Mark Kuchner would have us all be streetwalkers showing a lot of thigh with promises of more to come. The labor end of that contract gets screwed. Flashy mediocrity converts glitter into a tyranny of immersive falsehoods. Management, rewarded for enforcing process not creating product, creates cultures of failure (other than for sales commissions).

Zephir said...

/* scientific knowledge is eventually either right or wrong */

In AWT it just depends on the observational perspective. For example, for observer inside of gravity field of black hole the space-time is curved and the speed of light is invariant, but for observer outside of it the space-time appears flat and it's the speed and the path of light, which is variable there. Both insights are perfectly relevant for their perspectives even without resorting to multiverses. The same situation exists with many other allegedly invariant "scientific truths".

Joseph Bramante said...

I think the problem is deeper and even less extricable than this blog post lets on. If comments on the arxiv were enabled tomorrow, and we could post short comments on new papers, I doubt anyone interested in staying in physics theory would post anything. Especially if a critical comment were correct, I would expect the authors to be embarrassed -- unnecessarily in a perfect scientific community, but that is not what we have.

We would have to incentivize not only review and discussion, but also the publication of that discussion even at the expense of embarrassing very good scientists, who appear to be correct 99% of the time by only publicly presenting material they are certain about and have checked thoroughly. Their collaborators know otherwise, but...

MarkusM said...
This comment has been removed by the author.
Sabine Hossenfelder said...

Thomas,

No, you misunderstand me. What I mean with "independent" is performed by people who have no stakes in the outcome. You cannot test everything by experiment, because most often the whole point of a theoretical paper is to convince somebody to invest money into doing the experiment in the first place. Best,

B.

Sabine Hossenfelder said...

Joseph,

I agree with you. The core of the problem I think is again that we conflate the process with the product. On the one hand, as I mentioned in my post, this has the consequence that people are getting hired not for how good they do science but for how interesting they topics that they work on currently are. Of course that is correlated to some extent, which is why the system works at all, but it's not what we actually need. On the other hand this identification of scientists with the topics they work on has the consequence that they take (have to take) criticism very personal, with the effect that criticism generally has come to be seen as a bad thing (not kind to your colleagues and career-wise a waste of time). That, I think, is very unfortunate for science.

I think you're right in that pretty much nobody would comment on papers on the arXiv would it become possible tomorrow. With the exception of papers by very well-known people of course, which would make the ground even less even than it already is. This is why I actually think it's good there is no comment option on the arxiv, it would promote a rich-get-richer trend. Though they do list blog trackbacks :) I would appreciate however if they would list 'comment on' and 'reply' submissions in a more organized way than just dumping them into the daily queue. Best,

B.

Unknown said...

The main point in marketing is not making the customer to believe that the product is good, but making him believe he needs the product. In "research marketing" I think that the goal is to get funds against equally good research, trying to convince the guys with the money that this particular research is somehow better for them. In this sense I think thats not so different from a candy bar.

Sabine Hossenfelder said...

Unknown,

You missed my point. I'm not saying that science *is* not marketed like a candy bar, I'm saying it *shouldn't* be marketed like a candy bar because it defies the purpose. Science is supposed to deliver an objective assessment on whether we "need" some particular study or research in the sense of how promising it is. Marketing skews this assessment. That's okay if it's a candy bar because we all know that people *really* like things more just because they are more expensive or have a fancy wrapper. That's why I say there's no right or wrong in this, people like what they like and it's their decision. In science it's not about liking, it's about working. You're supposed to evaluate whether something will work or not, and not convince people they need something you're secretly convinced will not work just because your employment/grant/fate of your students depends on it. Best,

B.

Unknown said...

Bee,

what I was trying to say is that after you have selected only things that work, using your words, you still need to do further selection, because resources are limited. This further selection is necessarily subjective, as the selection of a candy bar. We'll put money in a new space telescope or in a super linear collider? Assuming that these two ventures are both good science this is when marketing comes in: to convince people that, lets say, a linear collider is better, because have a fancier wrapper than the telescope. In a smaller scale I guess that's how research grants work: after discarding things that does not work you have still to make further selection.

Keith Woodward said...

Scientists should always strive to speak "truth to power" but they should not hesitate to speak, when they don't (public) perception drifts off course, we end up with climate denial, unfounded fear of radiation, etc.

Sabine Hossenfelder said...

Unknown,

Selecting first for scientific quality and then selecting second for marketing skills to narrow down isn't what's happening in reality. You notice people or products by marketing first, and then second you look at the quality, or how well they fit your needs. My point is that this means you'll a) entirely miss those people who lack marketing irrespective of how good they are and b) your quality judgement will at this point already by influenced by the marketing and is no longer objective. Best,

B.

Robert L. Oldershaw said...

Speaking of science marketing, the WIMP salespeople have done an amazing job.

They have sold stock in the pot of WIMPs at the end of the rainbow to nearly everyone.

Meanwhile 100 billion to 1,000 trillion unbound planetary-mass nomad objects roaming around the Galaxy only rate a "ho hum".

Way to go!

Phillip Helbig said...

Meanwhile 100 billion to 1,000 trillion unbound planetary-mass nomad objects roaming around the Galaxy only rate a "ho hum".

That estimate spans an awesome 4 orders of magnitude. If you don't know how many there are any better than that, what do you know?

Nevertheless, the question at hand is what objects have a total mass (that is what matters, not the total number) which could explain the dark/missing/non-baryonic matter and still be compatible with observational constraints.

Robert L. Oldershaw said...


Does Mr. Helbig have any idea how many orders of magnitude are involved in guesses about WIMP masses?

Are new ways of thinking anathema to Mr. Helbig's belief system?

Is an astrophysical answer to the dark matter problem unacceptable?

Is particle dark matter the only game in town?

Robert L. Oldershaw said...
This comment has been removed by the author.
Uncle Al said...

@Robert L. Oldershaw,

http://news.sciencemag.org/physics/2013/11/live-chat-does-dark-matter-consist-weird-particles-called-axions

Science has become a Craig's List escort service. Three orthogonal votes of no confidence against Tully-Fisherinos: arXiv:1310.8214, 1306.5534, 1306.3983. Put Wesley Crusher in charge of Particle Particulars.

http://www.nature.com/news/who-is-the-best-scientist-of-them-all-1.14108

That is despicable in so many ways. Indices of past quality assassinate the future. When something terrifically fails, we need different not more.

Robert L. Oldershaw said...


Jim Baggott's new book, Farewell To Reality, says it all in a polite and a well-informed manner.

We are clearly in the pseudo-science era.

Let's hope it does not last as long as the Ptolemaic paradigm did.

Phillip Helbig said...

"Does Mr. Helbig have any idea how many orders of magnitude are involved in guesses about WIMP masses?"

Huge difference: You are claiming that something which has been detected (your free-floating planets) are the dark matter but can't back it up with any numbers. Sure, people are looking for WIMPs, but also looking for other things which could be the dark matter. Obviously, since none of these have been detected yet, one doesn't know how many there are. Someone hypothesizes that something might exist, someone looks for it: that's science.