Factors influencing acceptance or rejection by Iranian medical researchers of invitations to peer review

Background: Peer review is a necessary but costly and time-consuming process to identify good-quality and methodologically sound articles and improve them before publication. Finding good peer reviewers is often difficult. Objective: To identify the incentives that make Iranian biomedical researchers accept invitations to be a peer reviewer and factors that affect these incentives. Methods: Twelve reviewers selected at random from the reviewers pool of each of 26 biomedical journals published from Fars province, Iran, were surveyed using a questionnaire that we had developed and tested in a pilot study of 30 reviewers (Cronbach’s alpha of 0.779). The data included the reviewers’ demographics, history of their reviews, and choice of 11 reasons each for accepting or declining the invitation to review. Results: A total of 233 reviewers completed the questionnaire. The most important reasons for accepting the invitation to review were the journal’s practice to publish the names of the reviewers alongside the article they had reviewed, acknowledgement by the journals by publishing the names of reviewers once a year, free access to journals’ content, and lower publication charges as authors. The most common reasons to decline the invitation were lack of time, busy schedules, and lack of sufficient incentive to review. Conclusion: Acknowledgement by the journal, offering to publish the names of reviewers alongside the articles they had reviewed, and monetary rewards will be effective incentives for biomedical researchers in Iran to serve as peer reviewers.


Introduction
Peer review is an essential part of publishing articles and improving their quality. 1,2 Editors, as key figures in the peer review process, face several problems including low efficiency, high costs, delays, and biases, due mainly to the large number of submissions they receive and, consequently, the large number of invitations to peer review sent out by journals. 3,4 Slow peer reviews delay publication. Reviewers generally decline such invitations because of their busy schedule and insufficient motivation; 5 when they do accept, many take too long or send back reviews that are inaccurate or of poor quality. 6 Reasons for accepting the invitation include personal incentives such as commitment to science, opportunities to improve their writing and reviewing skills, and direct and indirect financial rewards from the journal. 2,7,8 Journals employ different strategies to motivate reviewers, such as trying to ensure that reviewers are invited to review articles in their particular field of interest, publishing their names as a form of appreciation on the journal's website or in the printed edition of the journal, waiving publication fees, providing free or discounted access to the journal's content or to different databases, research platforms, and digital libraries, paying a honorarium or a fee for reviewing, issuing certificates, and offering membership to the journal's editorial board. 2,7,8 In a study on 551 reviewers of five biomedical journals, the most important factors that led reviewers to accept the invitation to review were the contribution of the manuscript under review to the subject area, relevance of the topic to their own work or interests, and the opportunity to learn something new from the manuscript, whereas the most important factors that led them to decline the invitation were conflict with other commitments, having too many reviews to complete for other journals, and tight deadlines. 5 Similar findings were reported from another survey by the Publishing Research Consortium. 9 In the study of biomedical journals, financial or personal (for example, membership to the journal's editorial board) incentives had no significant effect on the decision to accept the invitations. 5 Another study involving 2982 respondents showed that the main reasons to accept the invitations were the desire to be an active participant in the research community, to gain professional credit, and to build relationships with particular journals and editors. 10 Reviewers tended to cooperate more readily with high-ranking journals than with low-ranking ones, and most reviewers declared that they would spend more time on reviewing if it was recognized as a measurable research activity by their institution. 10 In a study of a single journal, of 680 people who were invited to be reviewers over a 5-month period, 184 declined the invitation for three main reasons: not being available, manuscript irrelevant to their field of expertise, and conflict of interest. 11 Iranian journals and researchers have published several articles on different types of peer review, the best method of peer review, the time taken for reviewing and delays in peer review, and reviewer training. [12][13][14][15][16] Many of these articles emphasized that editors find it difficult to locate suitable reviewers, that reviews are often delayed because reviewers have too many manuscripts to review because of too many submissions, that in many fields only a few reviewers are available, and that many researchers are simply not motivated enough to undertake reviewing.
However, when we searched PubMed, Google Scholar, Scopus, and Magiran in English and Persian up to February 30, 2020, and found no published study from Iran that investigated the reasons behind acceptance or rejection of the invitation to review. In an effort to bridge this gap in knowledge, we sought to examine those reasons as they relate to biomedical journals published in Fars province in Iran and to explore the reviewers' motivations in accepting the invitation to review. The findings of the study will help editors to motivate reviewers, to reduce delays in peer reviews, and to improve the quality of reviews.

Methods
We focused on biomedical journals published by three universities of medical sciences in Fars province, Iran, namely Shiraz (23 journals), Jahrom (1 journal), and Fasaa (2 journals). All these journals (listed in Appendix 1) are published in English except the one journal published by Jahrom, which is in Persian but carries abstracts in English. We asked the journals for the names of reviewers in their pool, and most of the journals obliged; to those that did not, we asked for the number of reviewers in the pool and then requested them for the names of at least 20, selected at random.
The Ethics Committee of Shiraz University of Medical Sciences approved this study (ethics code: IR.sums.rec.1398.856).

Participants
We requested all the Iranian reviewers whose names we had collected to participate in this survey: the reviewers who we considered were not ethnic Iranians were excluded.
The questionnaire and the form to obtain informed consent were emailed to all during November 2019 to May 2020. For some reviewers who worked in Shiraz but did not respond to the emailed request to participate, the documents were delivered to their offices by hand. If we did not receive a response through email or by telephone and could not reach the selected reviewer in person, we selected an alternative participant at random from the list sent by the journal in question.

Questionnaire development
We designed the questionnaire based on earlier studies 5,10 as well as our research hypotheses. The questionnaire was in Persian and covered demographic information including age, gender, highest academic degree, field of study (basic or clinical sciences), university rank (1, 2, or 3, based on the criteria used by the Iranian Ministry of Health, with 1 denoting the highest rank), work experience, number of invitations to review (for domestic and international journals) accepted or declined in the last two years, and the number of articles actually reviewed in the last two years for journals indexed by the Web of Science or PubMed. The questionnaire also listed 11 reasons for accepting or declining the invitations, each reason to be scored on a 5-point Likert scale (0 = not important to 4 = very important). To supplement that list of 11 reasons and to ensure that we had not overlooked any other reason that may prove helpful and also to determine whether the reviewers were familiar with standard reviewing methods, we asked the participants, though some open-ended questions, to suggest any other means -other than those featured in the list -of motivating potential reviewers and what the participants considered the best method of peer reviewing. We used the graded response model as a special type of item-response theory 17 to assess whether the scores assigned to each of the reasons for accepting or declining the invitation to peer review measured the intended construct.

Statistical analysis
The minimum sample size to assess the construct validity of a questionnaire should be 5-10 times the number of items in the questionnaire. 18 Because the questionnaire included two sets of items, 11 reasons for accepting and 11 reasons for declining the invitation to peer review, at least 220 reviewers were needed for the study. To allow for missing data, we fixed the sample size at 300, which meant selecting 12 reviewers from the pool of reviewers of each of the 26 journals. We calculated the ratio of reviewers from basic sciences to those from clinical sciences for each journal and ensured that the ratio was maintained in the reviewers selected for each journal. The reviewers were selected randomly using List Randomizer, a tool available at https://www.random.org/lists/.
As mentioned earlier, we used the graded response model as a special type of item-response theory 17 with a i representing item discrimination and b i representing the thresholds. For each item in both the sets (reasons to accept and reasons to decline the invitation), a P value less than 0.05 indicated that the item did not belong to that subscale and did not measure the construct of the intended dimension. The item-response theory maintains that the discrimination parameter should always be positive: if negative, the item should be removed from the questionnaire because a negative value indicates that it does not provide any information on the intended dimension or is inversely related to it.
The mean score for each set was calculated by adding up the score assigned to each of the 11 items' scores and dividing the total by 11. Because the maximum score for any item was 4, the highest mean score could only be 4, and the minimum score could be zero.
The validity of the questionnaire construct was assessed using Spearman correlation. A correlation coefficient greater than 0.4 between an item and its own subscale was taken to mean that the construct was valid.
One-way ANOVA, an independent t-test, and Pearson's correlation coefficient were used to relate the mean score of the reasons for accepting or declining the invitation to such variables as the highest academic degree, age, gender, work experience, field of study (clinical sciences or basic sciences), university or institute's rank, and the indexing of the journal by the major databases (Web of Science or PubMed).
Statistical analyses were performed using SPSS ver. 18 and P <0.05 was considered statistically significant.

Results
In our pilot study, Cronbach's alpha was 0.70 for both sets, that is for the reasons for accepting or for the reasons for declining the invitation to peer review.
A total of 1400 questionnaires were sent out and after vigorous follow-up, 233 (16.6%) were returned and considered for analysis.
The respondents were predominantly men with a postgraduate degree (either a doctorate or a master's) and working at a high-ranking university (Table 1). Because faculty members from universities in Fars province were included by almost all the journals, most of the participants were from the same three universities, particularly the Shiraz University of Medical Sciences. Of the total, 159 respondents were affiliated to medical universities in Fars province and the rest were affiliated to other universities across the country.

Validity and reliability of the questionnaire
From the first set (reasons for accepting), we removed three items, namely 'sense of professional duty' , because its discrimination parameter was negative (a (SE) = −0.06(0.13)) and 'reputation of the authors of the manuscript' (P = 0.006) and 'academic rewards' (P = 0.038) because their P values indicated that these items did not belong to the intended construct (Table 2). This left us with eight items, of which the one with the highest correlation coefficient was 'publishing reviewers' names (if desired) with the reviewed article' . From the second set (reasons for declining), we removed two items, namely 'topic is not relevant to my field of expertise' (P = 0.040) and 'poor scientific quality' (P = 0.007) because their P values indicated that these items too did not belong (Table 3). This left us with nine items, of which the one with the highest correlation was 'tight deadline for completing the review' .
Testing for validity of the construct found that among the reasons for accepting the invitations, the highest correlation coefficient was for 'publishing reviewers' names' (0.738), followed, in descending order, by 'annual appreciation of reviewers' names' (0.720), 'free access to the journal content or the possibility of free publication' (0.699), 'receive appropriate reviewing fees' (0.610), and 'academic rewards' (0.589) ( Table  2). Among the reasons for declining the invitations, the highest correlation coefficient was for 'tight deadline for completing the review' (0.695), followed, in descending order, by 'having to review too many manuscripts for this journal' (0.666), 'length of the manuscript' (0.665), 'insufficient incentives from the journal' (0.607), and 'comments not taken into account in the past reviewing experience' (0.558) ( Table 3).
Overall, Cronbach's alpha was within acceptable limits, 0.76 for the reasons for accepting and 0.78 for reasons for declining the invitation.
The coefficient of correlation between the tendency to accept and the tendency to decline the invitation was 0.42 (P <0.001). We found no significant relationship between the tendency to accept or decline and gender, university or institute's rank, field of expertise (basic sciences or clinical sciences), highest academic degree (Table 4), or age (r = −0.185 and −0.101 for the reasons for accepting or declining, respectively) ( Table 5), work experience (r = −0.175 and −0.087, respectively) ( Table  5), the number of articles for which the requests were accepted or declined for Iranian and international journals (r = −0.139 and −0.175, respectively), and the number of these journals indexed in PubMed or Web of Science (P >0.05) ( Table 5). These analyses were performed on both the sets for all the 11 items, although the results were the same even after removing the three items from the first set (reasons for agreeing) and the two items from the second set (reasons for declining) .
Regarding the open-ended questions, 193 respondents offered additional suggestions on how to motivate reviewers. The incentives mentioned most often were relevance of the topic to the reviewer's own work or interests (59 respondents) and updating the knowledge on current research (17 respondents). Some other interesting suggestions were offering fast-track peer review when the reviewers submit their manuscripts as authors (12 respondents) and raising their profile as peer reviewers and thus their chances of being invited to review by international journals (3 respondents).
Regarding the best peer-review method, 110 of the 233 either did not answer the question or said they did not understand it. Of those who answered, 50 mentioned double-blind review; 10 each mentioned open review or single-blind review; and one respondent mentioned interactive open reviews.

Discussion
In this study of Iranian peer reviewers for Iranian biomedical journals, the most important factors affecting whether the invitation to review was accepted or declined were publishing reviewers' names (if desired) with the reviewed article, annual acknowledgement on the journal's website, free access to journal content or free or discounted publication in the journal, appropriate reviewing fees based on the time spent and the quality of the review, and academic rewards (career enhancement, for example) ( Table 2). The most important factors affecting whether the invitation to review was declined were a tight deadline, having too many manuscripts to review, length of the manuscript, insufficient incentives from the journal, and comments on reviewed manuscripts being ignored by the journal in the past (Table 3). We found no significant relationship between the reviewers' gender or the rank of their university or institute and their acceptance or rejection of the invitation, a finding consistent with that reported in earlier studies. 19,20 However, our finding no significant relationship between the journals' ranking in PubMed or Web of Science and the reviewers' tendency to accept or decline the invitation was in contrast to the findings of a study by a publisher, namely Wiley. 10 This difference might be due to the differences in the incentives offered in different regions and also due to some reviewers' tendency to build a relationship with journal editors (especially editors of highranking journals). The current study was limited to Iranian reviewers, whereas the study by Wiley included many countries from different regions of the world (Australia, Brazil, Chile, China, Germany, India, Japan, Malaysia, Mexico, Nigeria, South Africa, South Korea, Turkey, UK, and USA). Although the age range of the participants was almost similar in both the studies, with the majority being 50 years old or younger.
In our study, financial incentives, including free subscription or lower or zero charges for publishing an article, as well as receiving review fees, were among the most important factors influencing acceptance (Table 2). However, Tite et al, 5 in a study of international reviewers from five reputable journals (Archives of Disease in Childhood, BMJ, Emergency Medicine Journal, Gut, and the Journal of Epidemiology & Community Health), found that financial incentives were among the least important factors, although a significant number of respondents considered free subscription, as well as annual appreciation to be very important-which is what we found in the present study. The most important reasons for declining the invitation reported by Tite et al. were time constraints due to the large number of invitations (either from the same journal or from other journals) and personal issues, which were again consistent with the results of the present study. In both the studies, the respondents were men who were 50 years old or younger.
Ware conducted an international systematic review of 19 studies involving a total of 3040 academics. 20 Overall, the most effective factors motivating reviewers to accept the invitation to review, from the editor's perspective, were free subscription, appreciation from the journal, and zero or discounted publication fees. 9 Reviewers mentioned seven reasons for agreeing to review, of which the most important was a sense of commitment to academic community. We included this reason in our study, but few considered it an adequate In a study by Breuning et al, 3 the most important reasons given by reviewers for declining the invitation were a busy schedule, too many invitations to review, and being away on vacation or on a sabbatical. The first two reasons were common in our survey also, but the last one seldom featured in our study.
Warne found that feedback provided by the journal after the review and information about the journal's final decision on the submission were the most important factors motivating reviewers to continue accepting the invitations: this reason was not included in our questionnaire but was mentioned in the free-text responses of our study. In Warne's study, issuing review certificates, acknowledging the reviewers by the journals, and free access to the journal's content were important factors. 10 These factors were also significant in our study. Warne reported that the reasons for declining the invitations were lack of time, irrelevance of the topic to the field of expertise, and conflict of interest, 10 all of which also featured in the responses to our survey.
The more interesting suggestions from participants about how to motivate peer reviewers were offering a fast-track peer review to reviewers when they submit a manuscript to the journal and raising their visibility as peer reviewers; these two reasons were not considered in the earlier studies but could be helpful.
Regarding the best method of peer review, the most frequent response (50 of 71) was the double-blind review. Other responses were either not among the standard methods or were missing altogether, which indicates that the reviewers were probably unfamiliar with different types of peer review. The large number of respondents who either did not respond or failed to understand the question suggests that training courses for reviewers and university faculty on different types of peer review would be helpful.

Limitations
The study was confined to biomedical journals from Iran. Most of the journals included in the study were affiliated to Shiraz University of Medical Sciences, and most of the respondents were among the faculty of the same university. We excluded non-Iranian reviewers because of lack of resources but also because the Iranian journals in our study approached far fewer international reviewers compared to the number approached by other international journals. Of about 1400 questionnaires we sent out, only 233 were returned, further limiting the generalizability of the study. The reviewers were mainly prominent university professors who had busy schedules. For the reviewers who did not live in Fars province, follow-up was problematic because most journals did not divulge the reviewers' telephone numbers. Therefore, because of time limitations, limited access (especially to those living in other provinces), and the large number of reviewers from Fars province, it was not possible to ensure a balanced representation of different provinces. Our study did not investigate whether the institutions in which the respondents worked consider peer review during staff appraisals or promotions or whether such reviews should be recognized as part of continuing medical education and rewarded accordingly in the form of credits. Another limitation of the study is that we did not check the unidimensionality assumption; therefore the results of the item-response theory model regarding the questionnaire's validity should be interpreted with caution.

Recommendations
Journal offices, as well as universities, should take measures to increase the motivation of reviewers for improving the efficiency and possibly the quality of peer review. Journals should adopt strategies to expand their reviewers' pool; for example, journals could ask authors to suggest at least one reviewer at the time of submission. Journals should also invite young and motivated reviewers in different areas to reduce the number of review invitations sent out to one person. After reviewing the résumés and confirming the qualifications of potential reviewers, editors can add them to the journal's pool of reviewers.
To encourage researchers to become better peer reviewers, journals' websites could provide links to freely available online training modules such as Publons.
Our results suggest that an easy way to increase the motivation among peer reviewers is to publish the reviewers' names next to the published article (if desired) as well as publish the names of all those who served as reviewers in a given year as an acknowledgement of their contribution. However, in smaller journals, where it would be easy for authors to work out who their reviewers had been, such an acknowledgement Table 3. The construct validity of the reasons for declining peer review invitations based on item-response theory *b ik is the category boundary or threshold for category k of item i; SE = Standard error. N = 233; Cronbach alpha = 0.779 might not be a suitable option. Reviewers should also be offered fast-track review for their own submissions.
Financial incentives can be helpful. For example, journals that charge publication fees should consider zero or discounted publication fees for reviewers when they become authors (based on the number and the quality of their reviews) or paying an honorarium for reviewing, based on the number of articles reviewed and the time spent on the task. This will need a budget and may be tricky at times, because the practice may attract some reviewers but discourage others.
Universities or institutes could grant annual academic credits and certificates based on the number of completed peer reviews and the hours spent on reviewing. However, these measures are already implemented in the universities to which our participants were affiliated, which probably means that the reviewers are dissatisfied with the extent of credits and believe that they are not commensurate with the time and the energy spent on peer review. To validate the findings of this study, it should be replicated with a larger sample and through the offices of various journals across Iran over a longer time. Future research could also investigate whether the responses are affected by the journal's publication model (such as being free or open access or subscription-based). Another area of study would be to collaborate with reputable international journals to compare the responses of international reviewers with those of domestic reviewers.

Conclusion
Our findings suggest various ways in which journals could increase the reviewers' motivation to accept the invitation to review. Other measures require collaboration with universities or institutions with which the reviewers are affiliated. Because peer review is an important part of the overall academic publishing, journals should work with universities to offer greater incentives to reviewers and to recognize their contributions more explicitly. Table 4. The relationship between demographic and academic factors and the score of accepting or declining peer review invitations (n = 233)