| Description |
¨Peer Review¨ is supposed to be the ¨gold standard¨ of quality control for research projects and academic studies, yet evidence of its many deficiencies has been building up for over 20 years. That knowledge has been confined to the academic world until recently but now peer review’s tarnished image is being revealed to a wider audience. Rightly so, because this is no academic argument – when it comes to medicine, inadequate peer review of proposed new drugs and other treatments can be literally a matter of life and death. This programme talks to the key players in the UK and the US, reveals the problems and looks at some of the possible solutions for a flawed process that remains the least-bad way of assessing quality in research.
‘There seems to be no hypothesis too trivial, no literature too biased, no design too warped, no presentation of results too inaccurate, obscure and contradictory, no analysis too self-serving and no conclusions too unjustified for a paper to end up in print.’ That was Professor Drummond Rennie, then the deputy editor of the Journal of the American Medical Association, kick-starting the long international investigation into peer review in 1986. Twenty-two years later, he tells us it’s as true as ever.
‘Peer review’ means getting a paper scrutinised by independent experts in the same field, to assess the quality of its conception, execution and analysis. Yet trials by top medical journals have shown that distinguished peer reviewers fail to spot multiple obvious errors deliberately introduced into test articles.
That’s just one of a series of weaknesses – other trials have shown that peer reviewers are seriously biased by the status of a report’s lead author (often a ‘guest’ who actually had nothing to do with the research) the prominence of the authors’ university, their country of origin and a host of other factors unrelated to the quality of the work. And once a report has passed peer review and gone into print it stays in the record – there are many examples of discredited articles that continue to be cited by other authors for years or even decades afterwards.
Now the American legal profession is seeing how it can exploit the weaknesses of peer review. Since 1993 publication in a peer-reviewed journal has been the main qualification for an expert witness in a US court. Recently however a group of major companies dragged five peer reviewers into pre-trial cross-examination, hoping to fatally undermine the expert witnesses whose work they had approved. – and that’s likely to become a common tactic.
|
