Reviewing Manuscripts for the American Journal of Neuroradiology: Responsibilities, Challenges, and Rewards

“Peer review is the critical assessment of manuscripts submitted to journals by experts who are not part of the editorial staff.”

—International Committee of Medical Journal Editors¹

The American Journal of Neuroradiology (AJNR) uses peer review for all Original Research articles, Technical Notes, and Case Reports. Only Review Articles, Letters to the Editor, Society News, and Book Reviews do not undergo a formal double-blind peer review. Ab initio rejections are rare but occasionally may be rendered if an Editor finds a manuscript to be inappropriate for AJNR. Our peer review system is double-blind; that is, the reviewers do not know the identity of the authors and vice versa (in a single-blind review system, the reviewers know the name(s) of the author(s), but the authors do not know those of the reviewers). I believe this assures equitable and honest reviews devoid of personal differences and contributes to a perception of fairness (which is as important as fairness itself!). We also mask any reference to the institution where a study originates, any identifying information, and all acknowledgments.

Despite these precautions, peer review is not perfect. Because handling the double-blind review process is not devoid of problems, some journals permit their reviewers to “see” the names of the authors. One of their reasons is as follows: previous publications may be checked to avoid duplication and self-plagiarism. AJNR requests that authors sign a conflict of interest/disclosure form stating that their work is original and has not appeared, partially or completely, in other journals.

How about the identity of the reviewers? AJNR also maintains this privately to assure that reviewers may express their thoughts honestly, without fear of repercussions. Conversely, some editors propose that identifying the reviewers’ names on articles leads to greater responsibility and civility, while giving them credit. Because academic credit is what most reviewers get from their work, this makes sense from theoretic standpoint. These editors also believe that blinded reviews result in many gratuitous and poor quality assessments (not my experience). A recently published survey shows that more than 50% of authors express concerns about reviewer competence and bias.² Younger reviewers may be afraid of retribution, and keeping their names blinded protects them. AJNR never reveals our reviewers’ names, and we go as far as publishing anonymous Book Reviews to assure that persons writing them can express an honest point of view. Conversely, many editors feel that revealing the names of those involved in the peer review process does not affect quality (http://www.wame.org/wame-listserve-discussions/blinding-reviewers-to-authors2019-identity-detecting-duplicate-publication-and-blacklisting-authors). I have heard double-blind peer review started approximately 30 years ago because some editors were more concerned about reducing “character assassination” than reducing reviewer bias.

Interest in paying reviewers has been expressed by some editors, but it is not a widespread practice because most scientific journals do not have this type of funding. Some editors believe that unpaid reviews are often cursorily done and delayed (again, this is not my experience). Some journals (www.jmir.org accessed on November 5, 2008) pay only those individuals involved in their “fast track” publications and only if these reviews are rapidly completed. If we paid for manuscript assessments, AJNR would have to pay 700 reviewers for more than 3000 reviews every year. If we paid our reviewers only $20 per review, this would account for a significant part of our budget, and I suspect, rather than pleasing most, this relatively small amount of money would insult some. Only our statistician receives a modest honorarium for manuscripts she is asked to evaluate, generally as a third reviewer. In my opinion, reviewers would be happy only if the money was worth more than the time they employ, and currently most societies are not in a position to satisfy this. Some individuals believe that paying or getting paid for reviewing (or authoring) corrupts the “purity” of science, and the transfer of monies would put us in the realm of “vanity” publishing. Paying reviewers would begin a chain reaction, a battle among journals to obtain reviewers, and lead to the bankruptcy of smaller journals unable to opt out.

Not all reviewers, such as those employed by the US government, are permitted to accept payments. In lieu of payment, many journals award “prizes” to their best reviewers in the form of personalized thank-you letters (sometimes copied to department chairpersons) and/or publish their names in end-of-year “best of” lists. Other journals give reviewers CME credits or offer discounted fees for society memberships, conventions, and journal subscriptions. Having a “thank-you” meal at meetings is also used and achieves mixed results as only approximately one third of invitees attend. I believe that the ultimate reward for doing a good job as a reviewer is being asked to be a part of our Editorial Board. I also believe many of us review scientific manuscripts because we feel an intellectual curiosity and an ethical obligation to do it. Fortunately, most of our reviewers are also our most prolific authors and understand the importance of the activity.

What are the responsibilities of our manuscript reviewers? Whereas reviewers make recommendations, our Editors make decisions. I would like to summarize the reviewer responsibilities as follows:

1. Critically and constructively evaluate submitted manuscripts. Ensure results are not speculative or too preliminary. A short guide to reviewing manuscripts is found on AJNR's submission site, on the Welcome page (“User Tutorials” under “Resources”).

2. Help authors improve their work. This is particularly important because our reviewers tend to be experienced investigators/clinicians and many authors are younger or from countries with less formal research infrastructures.

3. Make recommendations regarding suitability for publication to the Editors. Select articles appropriate for AJNR's audience. Once a decision is reached, reviewers have the right to receive feedback on this matter.

4. Treat the manuscript confidentially and with respect. This implies being on time with the evaluation. Avoid derogatory comments and do not use any part of the unpublished work for your own benefit.

5. Immediately communicate any suspicions of misconduct. Remember genuine errors may occur; do not spread unsubstantiated allegations.

6. Declare any potential conflicts of interest and, if present, decline the invitation to review. Both positive and negative biases affect reviewers’ judgment.

How do we select our reviewers? First, all reviewers are asked to join the peer review effort on a voluntary basis. All are experts in the scientific topic addressed by the manuscripts they evaluate. When a lesser experienced reviewer is used—such as when a fellow is employed—he or she serves as a third reviewer. I encourage all fellows to review articles and I am pleased with the quality of the reviews of those who now participate.

How are our reviewers assessed? After immediately being notified that a completed review is in the system, the Editors read and grade its contents and timeliness. If the review is below par, the Editor has the option to rescind it. Editors may choose to communicate directly with reviewers to express gratitude for excellent reviews or explain problems. Every 2 years, I will review the number of accepted and completed reviews as well as the reviewers’ scores and select the individuals with the highest rankings for our Editorial Board. Completion times are very important because they affect the overall performance of the journal (most journals currently allow 2–3 weeks for completion of reviews). All reviewer scores are confidential.

Some reviewers, particularly those who work outside of the United States and younger reviewers, believe that a structured electronic review form would be helpful to them. Although I prefer our more “loose” structure of reviews because they preserve the personality of the reviewer, assessments must follow the order stated in our Reviewer Quick Start Guide. Following this structure greatly facilitates the Editor's work and the author's revisions. For more information on reviewing, I suggest the following book: Peer Review and Manuscript Management in Scientific Journals by Irene Hames (2007; Blackwell).

This editorial would not be complete if I neglected to mention, albeit briefly, alternative types of peer review (because they are already implemented by some scientific journals). These include the following:

1. Presubmission peer review. I believe this already occurs in all multi-author articles.

2. Postpublication review or commentary. BioMed Central and the British Medical Journal permit these types of comments from the scientific community in general after registration on their Websites. Some others permit postpublication review only by defined bodies of specialists.

3. Hybrid systems. www.atmos-chem-phys.org (accessed November 5, 2008) is an open access journal with a 2-tier review process. The first review is a traditional one, and the second involves comments from scientific communities, other authors, and referees. The second review period lasts 8 weeks, and the editor serves as a referee between posted comments and responses from the authors of the original submission. In www.biology-direct.com (accessed November 5, 2008), authors are responsible for finding 3 reviewers from the editorial board, and if they are unable to do so, their submission is rejected. www.etai.org (accessed November 5, 2008) is an artificial intelligence open access journal that allows open comments to which the authors may respond. After the author makes revisions, their submission is sent out for formal peer review. For the journal Nature (www.nature.com/nature/peerreview/index.html, accessed November 5, 2008), on-line open and traditional peer reviewing are done simultaneously. www.plosone.org (accessed November 5, 2008) allows for prepublication and postpublication peer reviews.

Regardless of which of these methods is used, individuals who review articles are essential to maintain high-quality scientific publications. AJNR counts on many of these dedicated individuals and we are proud of our short—24 days on average—review period.

• Copyright © American Society of Neuroradiology