Peer review II

Last week, I received a manuscript that I was asked to referee. Since already 3 other manuscripts are lying on my desk (somewhere) waiting to be refereed, and the topic was pretty outside my field, I decided to let the editors know that I am not suitable as a referee. Being blessed with one of these online services, I clicked on the wrong link in the email and got thanked for having agreed to referee the paper.

Now I actually have to read this thing. Though at second sight it turned out to be more interesting than I initially thought.

Anyway, I had an engaged discussion on the weekend on the publishing issue that I would like to share with you.

We all know that the pressure to publish in peer reviewed journals is not improving quality in research. On the contrary, it leads people to favor publishing more of less quality. The reasons why some papers get published, whereas others don't, are sometimes just mysterious, sometimes clearly due to the authors names. Peer review, it seems, does not work as it should. And the number of publications, and their citations are - at least in my opinion - not necessarily a way to single out good research. However, the question is, what can there be done about it.

Some points that we came up with.

1. The referee should have some advantage from refereeing a manuscript. In such a way that (s)he is motivated to think about the content and make reasonable suggestions. I am mostly thinking in terms of credibility. I suspect that most journals probably have an internal ranking for referees anyway, but what does the referee ever get out of writing good reports? One might also consider giving some kind of bonus for writing reports in time. I would happily pay $ 50 for actually receiving a report within 2 weeks!
2. Being mentioned in acknowledgements should be rated higher. People who are frequently mentioned in acknowledgements show that they are engaged in discussions, are able to understand and criticize theories, and are active part of research. This is the more important, as those who don't want to be part of fashion waves often end up with less publications. As to my papers, the quality improves significantly with every person I can discuss its content. (Restrictions apply).
3. The number of citations should be normalized to the number of active workers on the field. At least approximately.
4. I would find it enormously helpful if the arxiv would allow reviews on the papers, maybe similar to those at amazon. You might argue that a good physicist should be able to judge on the quality on a paper by himself. Though that is in principle true, it is absolutely inapplicable if you are new in a field and try to get into it. Some kind of quality index, or references to basic papers on the field will help newcomers to get to the central questions much faster - and with less wasted toner. In addition, the possibility of having reviews on the arxiv would make it unnecessary to have follow up papers titled 'A note on gr-qc/...' and 'A remark on a note on ...' etc.

Another point that I have argued against is the idea of double blind refereeing process. First, it does not work when the papers are already on the pre-print archive before submitted to the journal. You then would have to make sure to only accept manuscripts not on the pre-print server, which would make the pre-print idea completely absurd. Second, it would only lead authors to write their papers such that for everyone in the field it's clear who the author is. Third, I actually do think that the credibility of the author is an input the referee might want to consider.

If you have further suggestions, let me know!

Best,

B.