- Research grants: Conform and be funded
Joshua M. Nicholson, John P. A. Ioannidis
Nature 492, 34–36 (06 December 2012) doi:10.1038/492034a
Nicholson and Ioannidis analyzed papers published between 2001 and 2012 in the life and health sciences, catalogued by the Scopus database. They looked those who had received more than 1,000 citations by April 2012 and an author affiliation in the United States. They found 700 papers and 1,172 authors matching this query.
The NIH invites PIs of funded projects to become members of study sections. The purpose of NIH study section is to evaluate scientific merit. Nicholson and Ioannidis found that from the 1,172 top-cited authors only 72 were currently members of study groups, and most of these 72 (as expected) currently received NIH funding. However, these 72 top-cited scientists are merely 0.8% of all section members. Maybe more insightful is that they further randomly selected 200 of the top-cited papers and excluded those with authors in a study group. From the remaining top-cited authors, only 40% are currently receiving NIH funding.
In a nutshell, this is to say that the majority of authors of research articles in the life and health sciences that were top-cited within the last decade do not currently receive NIH funding.
That's as far as the facts are concerned. Now let's see how Nicholson and Ioannidis interpret this finding and what they conclude. In the beginning of the article, they are careful to point out that scientific success is difficult to measure and the citation count should be regarded with caution:
- "The influence of scientific work is difficult to measure, and one might have to wait a long time to understand it. One proxy measurement is the number of citations that scientific publications receive. Using citation metrics to appraise scientists and their work has many pitfalls... However, one uncontestable fact is that highly cited papers (and thus their authors) have had a major influence, for whatever reason, on the evolution of scientific debate and on the practice of science."
- "The mission of the NIH is to support the best scientists, regardless of whether they are young, old or in industry... Such innovative thinkers should not have so much trouble obtaining funding as principal investigators. One cannot assume that investigators who have authored highly cited papers will continue to do equally influential work in the future. However, a record of excellence may be the best predictor of future quality, and it would seem appropriate to give these scientists the opportunity of funding their projects."
- "Funding all scientists who are key authors of unrefuted papers that have 1,000 or more citations would be a negligible amount in the big picture of the NIH budget, simply because there are very few such people. This could foster further important discoveries that would otherwise remain unfunded in the current system."
Now I know nothing about funding problems in the life sciences. Maybe they have a good reason to hold a grudge against NIH peer review practice. Be that as it may, the facts do simply not support their arguments. I am tempted to say it actually speaks in favor of the NIH that they do not pay so much attention to the citation count because, as the authors write themselves, it's a questionable measure: It measures not only innovative thinking, but also fashions and just usefulness (reviews and illustrative diagrams tend to gather lots of citations), it moreover picks up social dynamics, popularity of the authors, or the amount of secondary work that is created, irrespective of whether that work is particularly insightful.
Many top-cited works are created because somebody has been fast enough to jump onto a topic about to take off. Is that a sign for not being "conform", as the title of the article suggests? I am trying to imagine that somebody would argue that all top-cited physicists should get their projects funded without peer review. And would try to publish this as an essay in Nature.