Skip to main content

Wellcome peer review report

Copyright Wellcome Trust CC BY 2.0


Copyright Research Information Network
A Wellcome Trust commissioned report centred around the issue of peer review was published last month. The report, conducted by the Research Information Network, sets out a detailed analysis of peer review, the critiques, and the new alternative systems for peer review that have appeared in recent years.

The report defines traditional peer review as 'the process of subjecting an author’s scholarly manuscript to the scrutiny of others who are experts in the same field’ (p.6) These experts must assess whether the research is of sufficient quality to be included in the journal. Many journals also require reviewers to assess the originality and significance of research, however it is worth noting that some open access mega-journals such as PLOS ONE and Collabra do not require this.

The report details many of the criticisms commonly made of peer review. One criticism is that peer review is not effective at weeding out unsound research. The report makes mention of the fact that retraction rates have increased in recent years, although the volume of retractions is a very small proportion of the 2 million papers published each year globally. Another criticism mentioned in the report is the burden faced by reviewers, the vast majority of whom are unpaid. Research suggests that around 3 million papers are submitted for review each year, and many are submitted and reviewed more than once. This represents a significant burden on reviewers, half of whom spend more than 6 hours on each review (see our recent blog post on the new journal Collabra that compensates reviewers by paying them a proportion of APC income). Other criticisms include potential for bias, expense, delays, and the potential for subversive behaviour.

Given the criticisms and the potential new avenues afforded by new digital technologies there have been many experiments with alternative forms of peer review. To take one example, open review sees both author's and reviewer's names disclosed from the outset (for example BMJ and PeerJ) and is designed to encourage constructive comments and avoid overly harsh reviews. Journals such as Frontiers (see previous blog post here) have expanded on this idea and introduced an interactive communication element to their open review process.

After investigating the various forms of peer-review the report concludes that peer review "remains a bedrock of the scholarly communications system", but reflects that there is likely to be increasing pressure on traditional peer review systems as the rate and volume of scholarly communication increases. It is this continuing pressure that will ensure experimentation with new forms of peer review will continue with cooperation between research and publishing communities:
'[W]e detect a sense in which while publishers will continue to explore new approaches in the ways we have described, they would welcome more guidance from key sections of the research community on the kinds of peer review services they want from publishers, and on the purposes that they should seek to fulfil. Unless the purposes are defined with greater clarity than they are at present, at least some of the current experimentation may prove to be of little point.' Scholarly Communication and Peer Review: The Current Landscape and Future Trends, p.30. CC BY

The full report, Scholarly Communication and Peer Review: The Current Landscape and Future Trends, can be found here.

Comments

Popular posts from this blog

We've now reached 8000 items in our repository!

Last week the St Andrews Research Repository reached a new milestone: 8000 items!

The last major content milestone we celebrated was when we reached 5000 items, this was back in February 2015. The blog post we wrote in recognition of this mentioned that the upsurge in activity was largely down to research funders and HEFCE (the folks behind the Research Excellence Framework) requiring authors to self-archive their publications. 17 months on and this trend is continuing.

In April 2016 the Research Excellence Framework open access policy came into effect (to find out more read our previous blog post). This means that to ensure compliance with the policy authors must deposit their accepted manuscripts for journal articles and conference proceedings into the University's research information system (Pure). To ensure all St Andrews researchers are aware of the policy we have been working hard to deliver the message: 'Act on acceptance: deposit in Pure'. This slogan, which is em…

Tickell report positive on the future of Open Access in the UK

Following the Burgess Review of Research Councils UK Open Access Policy and RCUK’s response, this report considers the wider scope of UK Open Access generally and how scholarly publishing markets and the policy landscape are developing including Open Data.

The advice was provided by Professor Adam Tickell, the respected Provost and Vice-Principal, University of Birmingham and Chair of Universities UK (UUK) Open Access Coordination Group.

Its main conclusion was positive on progress to date:
Open Access to research continues to be a public benefit and the UK remains a world leader. Research Councils UK should continue to support Gold Open Access charges. 
However, some changes are suggested in the recommendations.

Some key recommendations:
Universities should be encouraged to sign up to the San Francisco Declaration on Research Assessment (DORA) UUK Open Access Coordination Group to support the development of agreed service standards around Gold UK Open Access policy should offer gr…

Open Access is here! Make sure you are ready

Open Access is now an essential feature of scholarly communications. As well as maximising visibility of the University’s research outputs, Open Access is now a requirement of many funders. It is also critical for ensuring eligibility for submission of journal articles and conference proceedings to the next Research Excellence Framework (REF).

The Policy for open access in the post-2014 Research Excellence Framework is in force from April 1 2016, and states "to be eligible for submission to the next REF, authors’ final peer-reviewed manuscripts must have been deposited in an institutional or subject repository". For St Andrews, this means that all researchers must deposit the accepted version of journal articles and conference proceedings in Pure as soon after acceptance as possible. In common with other institutions, the Library has been promoting the message ‘Act on acceptance: deposit in Pure’. This applies not just for REF and all authors should deposit their manuscripts…