E-Letter responses to:

n-week: Elizabeth Finkel PEER REVIEW: Australia's Proposed U.K.-Style Merit Ranking Stirs Debate Science 2006; 312: 176b [Summary] [Full text] [PDF]

E-Letters: Submit a response to this article

Published E-Letter responses:

Peer Review and Fair Evaluation

Clive R. McMahon, Rory P. Wilson   (12 October 2006)

Peer Review and Fair Evaluation 12 October 2006
Clive R. McMahon, Ecologist Antarctic Wildlife Research Unit, School of Zoology, University of Tasmania, Hobart, Australia, Rory P. Wilson Respond to this E-Letter: Re: Peer Review and Fair Evaluation	Individual scientists, research groups and entire research establishments are under increasing pressure to demonstrate that they produce many high quality publications (1). The interest in numerate expressions of this, such as ISI journal impact factor, the h-index (2), or the y-factor (3), reflects an understandable desire to stifle subjectivity. The difficulty is, however, to assess quality in such a way that it reflects general scientific opinion rather than the whims of a few individuals. We suggest, however, that a simple modification of theh-index as proposed by Jorge Hirsch would accomplish this. The h-index is an integer that reflects the number of publications that an individual has had cited that number of times at the time it is calculated. Higher values therefore demonstrate consistent production of publications of interest to the scientific community. Implicit in its calculation, however, is that an individual’s h-index will tend to increase over time. Thus, the h-index must be evaluated with respect to the number of years in active research to give a true measure of worth. We suggest that the h-index of any individual could be represented by plotting its value graphically over time since, e.g., completion of the Ph.D. The trajectory of the h-index should be particularly telling. Indeed, such trajectories can be compared to instantaneous plots of h-indices of all individuals corrected for time within the research establishment, and these, in turn, can be compared to instantaneous h-indices from previous years and in relation to those of other establishments. Most (96%) researchers who were questioned after the 2001 UK Research Assessment Exercise (RAE) as to how they wanted to be evaluated concluded that peer review was the preferred method of assessment (2). With researchers in the UK anticipating an imminent RAE, as are scientists in Australia with the announcement of Research Quality Framework (RQF) (4), perhaps it is time to leave decisions regarding the quality of research units to the opinion of the multi-faceted many rather than to the unavoidable biases of the well-meaning few. References 1. J. Giles, Nature 439, 770 (2006). 2. P. Ball, Nature 436, 900 (2005). 3. P. Ball, Nature 439, 770 (2006). 4. E. Finkel, Science 312, 176b (2006).