Strategies for Conducting Strong, Efficient Peer Reviews of Papers

Image credit: lexiconimages | stock.adobe.com

When doing a peer review of a paper for a journal, it can be valuable to identify a strategy that provides the time necessary to conduct a thorough, timely review, explained Ana L. Hincapie, PhD, MS, an associate professor at the University of Cincinnati James L. Winkle College of Pharmacy in Ohio. During her presentation at the American Pharmacists Association 2024 Annual Meeting & Exposition in Orlando, Florida, Hincapie said that after getting a request to review a paper, she first blocks off time in her calendar for both reviewing and writing the final report in order to complete the review in the allotted time frame requested by the editor of the journal.

“I read the paper 3 times. I know that sounds like a lot, but each read has a different purpose,” Hincapie said during her presentation. “You want to complete that review in the 2 or 3 weeks that you’re requested for. Look at your schedule and make sure you have the time to write the final report, and make sure you have that before the deadline.”

For the first read, Hincapie explained that she quickly reads the abstract and the paper to familiarize herself with the topic. Specifically, Hincapie noted that it is valuable to identify the type of paper it is, such as an observational study or an interventional retrospective study.

“I just want to know what’s in it. The reason for that is because, especially when I have less expertise, I want to make sure that I will follow all the key points that were needed to review the paper,” Hincapie said. “If the study design is survey research, I go and find the [EQUATOR (Enhancing the Quality and Transparency of Health Research)] checklists for publishing survey research. If it’s an observational study, I find the guidelines for an observational study so I have that checklist for myself.”

Hincapie explained that using checklists, such as those available through EQUATOR, can help to ensure that all the key points that are expected of a good paper of that type are addressed. Additionally, at this point in the review process, Hincapie begins to review the literature recently published on that topic, which allows her to confirm whether the most recent literature was cited.

“That will help me later on to see [whether] the papers the authors have cited are the most recent literature or whether there is something that needs to be included; that’s something I should let the authors know when I [write the report],” Hincapie said. “This also helps [me] get more familiar with the topic, but I don’t spend tons of time [on that step].”

Next in her process, Hincapie reads the paper again but spends more time on this second read. However, Hincapie noted that she skips the abstract at the start of the second read and instead focuses on each section of the paper, specifically on the introduction, methods, and results.

“I use the checklist that I’ve already identified based on the study design, then go point by point. When I do that, I’m doing my annotations at the same time for each point on each section of the review,” Hincapie said. “I do review the tables and figures. I’ve talked to some who have told me that they review the tables and figures before they read everything else…to see [whether] they can tell what the paper is about and whether the tables and figures are properly labeled and describe the data.”

According to Hincapie, readers should be able to identify the topic and scope of the paper based on the included tables and figures, so there can be some benefit to starting by reviewing those elements first during the second read. Another approach, Hincapie explained, is to start with methods and results, and then go back to the introduction to confirm whether it properly prepares the readers for the rest of the paper.

“It really depends on what works for you,” Hincapie said. “But don’t forget…to start doing your annotations [at the same time]. Once I have all my annotations in the paper, I review them and then take time to sit back and reflect on all those things I just wrote. What are the major issues? What are the minor issues? For this, I put a plus or minus in my annotations.”

Hincapie explained that, generally speaking, being able to identify major vs minor issues in a paper can help the authors make decisions regarding what needs to be revised extensively and what needs smaller edits, which can help them with their planning. “Imagine yourself as the writer, and you get 2 pages of comments—you kind of freak out with how many you see,” Hincapie said. “But if it’s organized, it helps you to plan your revisions...so take the time to classify all your annotations into major and minor issues.”

For major issues, Hincapie explained that it’s generally valuable to include study design problems, issues with the analysis, improper citations, wrong interpretation of results, or limitations that are not addressed by the authors. In the minor category, issues might include improvement of language or clarity, formatting, errors in tables or figures, a need for better labeling, or perhaps some minor analysis problems.

“One thing that’s important to remember is that when you’re doing the revision and making the comments, make sure you know what the scope of the paper is, as you cannot ask the authors to do additional analysis or change things that are outside the scope just because you think it would be great [to include],” Hincapie said. “Yes, that would be great [to include], and maybe you can point it out as potentially further research for the authors, but keep your review, comments, and requests for additional analysis within the scope of the paper.”

Notably, Hincapie explained one such comment that would be outside the scope of the paper would be if the reviewer recommends changing the study design of the paper. In Hincapie’s view, this is not an addressable recommendation for the authors.

“They cannot change the study design, so you cannot request that. If it’s something that is measured, you can point that out to the editor, but you cannot ask the authors to change the study design, as the study is done,” Hincapie said. “Keep that in mind with your comments.”

Another recommendation from Hincapie is to not get too caught up in the language, grammar, and typos. Instead, it is more valuable to focus on whether the paper has a logical flow, that all the sections are organized, and that they’re consistent with what is being described, such as the outcomes being defined in the same way as discussed in the methods section.

“You are not there to correct the language; that is not your job,” Hincapie said. “You can say, ‘I think this paragraph needs some revision on the grammar,’ but you don’t have to fix it…. Just make sure the writing is legible and clear.”

Additionally, Hincapie explained that for blinded reviews, reviewers may not know the origins of the authors, so it can be valuable to keep in mind that English may not be the authors’ first language. For this reason, avoiding judgment of the language quality can be important, as that judgment could lead to biases that may impact the overall objectivity of the review, as well as recommendations to publish the paper.

“You may start having biases and make assumptions of the work,” Hincapie said. “Avoid that as much as possible and just be impartial and provide constructive feedback.”

Impartial, constructive peer feedback can substantially elevate the quality of a paper, according to copresenter Laura A. Rhodes, PharmD, BCACP. For Rhodes, who is an assistant professor and assistant director of the PGY-1 Community-Based Pharmacy Residency Program at the University of North Carolina at Chapel Hill Eshelman School of Pharmacy, this is invaluable to the authors of that paper.

“I can tell you that every time I’ve published a manuscript, the peer review process has strengthened the content of my manuscript,” Rhodes said during the presentation. “It increases clarity, it’s made it more concise or more direct, or expanded upon things that I thought were clear, but I was too close to the subject or content to realize [it] may not be clear to somebody who’s not as close to the content. That feedback and improvements are meant to make the paper as good and as relevant as it can be for the audience that it’s going to while also maintaining integrity in the scientific process.”

Reference

Hincapie AL, Rhodes LA. From author to peer reviewer: practical considerations. Presented at: American Pharmacists Association 2024 Annual Meeting & Exposition; March 22-25, 2024; Orlando, FL.