Errata and retractions associated with research papers published by authors with Hungarian affiliations

Background: To examine the errata and retractions in total published output of Hungarian research and academia relative to that in 34 other European countries. Objective: To analyse the number of errata and retractions related to papers published by authors with Hungarian affiliations compared to those by authors with affiliations in the 34 other countries. Methods: Errata and retractions retrieved from three databases, namely Retraction Watch, Web of Science (WoS), and Scopus, were counted and sorted by country. Results: Scopus featured 7 retractions linked to Hungarian affiliations and WoS featured 10. Retraction Watch featured 26 such retractions, placing Hungary in 23rd position among the 35 countries arranged in descending order of the number of retractions. Of the 26 retractions from Hungary, 5 were in Elsevier journals and another 5 in Springer Nature; also, 8 of the 26 were associated with the University of Debrecen. When ranked for the number of errata notices for every 1000 published papers, Hungary was ranked 29th in WoS (2.54 notices per 1000 papers) and 26th in Scopus (2.3 notices per 1000 papers). Conclusions: The low numbers of Hungarian affiliations suggest that either research ethics are more stringently observed in Hungary or that publications from Hungarian research institutes, including papers in Hungarian – many Hungarian journals are indexed neither in WoS nor in Scopus – have not been scrutinized adequately through post-publication peer review.


Introduction
The integrity of academic literature can be assessed through the prism of amendments to it, either as corrigenda or retractions, and when evaluated at the level of a country or region, such data can provide unique insights. However, corrections and retractions cannot be equated: whereas retractions are usually published in the wake of evidence of violation of publishing ethics, such as data fabrication, falsification, and plagiarism, or that of even more serious malpractices such as research misconduct, corrections are typically published to set the record straight when any errors come to light. Thus, although both are forms of amending scientific literature and monitoring research quality after publication, corrections, unlike retractions, are usually issued to correct mistakes and to remove any erroneous or misleading information. 1 Ethics and the integrity of research and publishing lie at the core of research and publishing in the European Union (EU). 2 The present paper focuses on Hungary, which is geographically unique and occupies the centre of the region comprising seven other European countries but also has distinct linguistic, cultural and political ideologies, as do several other European countries. However, the academic challenges facing Hungarian scholars are essentially no different than those that confront scholars in the EU as a whole, or elsewhere in the world, and the objectives and the methods outlined in this paper are also applicable to similar studies of other countries.
Over the past few years, research ethics have become more stringent, partly as a result of more rigorous peer review after publication.
He (2013) noticed very few retractions associated with papers from Hungary between 2001 and 2010 compared to those associated with papers from economically stronger EU partners such as the UK, France, or Germany. 3 Hungary has one of the lowest publication counts per capita in the EU. 4 These indicators are important as part of Hungary's efforts to claim some of the funding, estimated at €100 billion, to be made available through Horizon Europe between 2021 and 2027. [5][6][7] Citations of Hungarian academics peak when the mean age of researchers is 41.53 years (compared to 41.75 years for grantees of the European Research Council), although the 'golden age' in research depends on the field of study. 8 It was against this background that the present study sought to compare the output of Hungarian research and academia with that in 34 other European countries as reflected in the errata and retractions related to the output by authors with Hungarian affiliations.

Database
The current research was conducted using a bibliometric approach. Data on corrections, including errata and retractions, were obtained from the following three sources: (1) the Retraction Watch database (RWD) 9 , (2) Clarivate Analytics' Web of Science (WoS), and (3) Elsevier's Scopus. Expressions of concern were also assessed from the RWD, but not from WoS or Scopus, which do not index this category of corrections.

Search strategy
For the RWD, a search was conducted by specifying Hungary in the country field, and the results were extracted manually. To extract data from WoS, Hungary was specified as the country or region; with Scopus, Hungary was specified as the 'affiliation country' . In the field 'document type' , the search was limited to 'corrections' , 'corrections additions' , and 'retracted publications' in WoS and to 'erratum' and 'retracted' in Scopus. In terms of time, no starting date was specified, whereas the end date was the date on which the databases were searched (13 October 2020 for RWD and 20 October 2020 for WoS and Scopus). Results from WoS were limited to those documents indexed in Science Citation Index Expanded (SCIE), Social Sciences Citation Index (SSCI), Art and Humanities Citation Index (A&HCI), and Emerging Sources Citation Index (ESCI). The data were extracted to Microsoft Excel files for further analysis. The parameters that were assessed and analysed are described in detail in the Supplementary file. Similar searches were conducted for 26 other EU member countries and 8 non-member European countries (Table 1) to ascertain the total number of publications and the number of corrections in WoS and Scopus. We included the UK in the 26 EU states because it was also part of the EU before Brexit, and we included those 8 non-member European countries based on their importance as representative players in academic output as well as their geographic and cultural relevance.

Correction and retraction indexes
Instead of comparing countries based on the absolute number of corrections, a correction index (CI) was calculated for each country, defined as the number of errata multiplied by 1000, as suggested by Fang and Casadevall, 10 and then divided by the total number of publications from that country as seen in the respective database. This measure shows the proportion of corrections for every 1000 publications for a given country. Using the same principle, we established a Retraction Index (RI) for each country, defined as the number of retractions multiplied by 1000 and then divided by the total number of publications from that country in the database. In addition to CI and RI, the following characteristics of the corrected documents from Hungary were also analysed based on the data collected from WoS and Scopus: the year of publication, the institution, subject area, collaborating countries, access type, the journal in which the paper had been published, the language of the paper, and the number of times the papers had been cited.

Overview of errata and retractions in three databases
The search yielded 26 entries from RWD (2 errata, 23 retractions, and 1 expression of concern), 665 (655 errata and 10 retractions) from WoS, and 680 (673 errata and 7 retractions) from Scopus (Table 1). Figure 1 shows the distribution of errata from Hungary in WoS and Scopus and Figure 2, the number of retractions from the three databases. The first relevant correction in WoS was of a paper published in 1993, whereas that in Scopus dated back to 1926. The number of corrections from Hungary increased considerably over the last decade, from 24 in 2010 to 62 in 2020 in WoS (a 158% increase) and from 23 in 2010 to 82 in 2020 in Scopus (a 257% increase) (Figure 1).

Errata
The data from WoS put Hungary in the 21st position among the 35 countries when ranked for the absolute number of errata (N = 655) but in the 29th position when ranked on the basis of errata per 1000 publications (CI = 2.54) ( Table 1). The index for Hungary was lower than that for most other countries. As shown in Table 1, the top three countries with the highest number of errata were Germany (11,646), the UK (10,690), and France (7,589) whereas Malta had the highest proportion of errata per 1000 publications (5.39), followed by Luxembourg (5.22) and Cyprus (4.87). The lowest CI was recorded for Ukraine (1.69), followed by Bulgaria (1.95) and Romania (2.23).
The data from Scopus also placed Hungary in the 21st position when ranked on the basis of the absolute number of errata (N = 673) but in the 26th position in relative terms (CI = 2.30). The same three countries topped the list -Germany (12,503), the UK (12,466), and France (7,560) -and Malta, at 5.28, was again at the top, followed by Estonia (4.04) and Luxembourg (3.87). In lowest CI was once again recorded for Ukraine (1.27), followed by Russia (1.60) and Romania (1.69) ( Table 1).

Retractions
The data from WoS put Hungary in the 24th position when ranked for the absolute number of retractions (N = 10) but in the 31st position (along with Russia, Slovenia and Ukraine) in relative terms (RI = 0.04) ( Table 1). The top three countries in terms of the number of retractions were Germany (364), the UK (323), and Italy (266), whereas the top three in terms of RI were Serbia (0.27), followed by Belarus (0.17) and Greece and Turkey (both 0.16). The lowest RI was that of Lithuania (0.02), and Ukraine, Russia, Slovenia, and Hungary all scored 0.04.
The data from Scopus placed Hungary in the 23rd position when ranked for the absolute number of retractions (N = 7) but in the 29th position in relative terms, along with Austria, Bulgaria, and Slovenia (RI = 0.02) ( Table 1). As in the case of errata, the RI for Hungary was lower than that for most other countries. The top three countries with the highest number of retractions were the UK (212), Italy (168), and Germany (156), whereas Serbia had the highest RI (0.34), followed by Luxembourg (0.12) and Ireland and Greece (0.09), and Malta, Estonia, and Latvia showed no retractions at all.
The data from RWD showed that three of the retractions were cases of retraction and republication. Eight retractions were from the University of Debrecen, followed by Semmelweis University (3 retractions). The most common reasons for retraction were problems with data, which accounted for nearly 35% of all retractions (8 out of 23 retracted articles), followed by problems with results and images (22%). Other reasons for retraction included misconduct by the author (4 papers), problems with conclusions, duplicate publication, and miscommunication by the author (3 papers each), and problems with analyses or methods and errors by the journal or the publisher (2 papers each). Lastly, the following reasons were represented by one paper each: undisclosed conflict of interest, lack of approval from third parties, miscommunication by the journal or publisher, plagiarism, and copyright claims.

Bibliometric characteristics of errata in Web of Science and Scopus
As can be seen in Table 2, data from WoS showed that the Hungarian Academy of Sciences (164), University of Debrecen (99), and Semmelweis University (92) accounted for the highest number of errata. The top three research areas with the highest number of errata were physics (114), chemistry (77), and mathematics (63). With regard to collaborating countries, 178 of the papers had researchers from Germany as co-authors and 176 from the USA. Of the total, 418 (63.8%) of the papers were openly accessible whereas 237 (36.2%) were not. The journal with the highest number of papers with corrections was the Journal of High Energy Physics (18), followed by the European Physical Journal C (17), and Scientific Reports (14). Nearly all the errata (650, 99.2%) were published in English, four were in German, and one was in Hungarian. As of 20 October 2020, the day on which the databases were searched, papers from Hungary had been cited a total of 1450 times in WoS, averaging 2.2 citations per paper. Over 60% of the papers had been cited at least once. The most cited, 'Spin filtering in a magnetic-electric barrier structure' , (DOI: 10.1063/1.1415371), was published in Applied Physics Letters in 2001 and had been cited 101 times in the WoS core collection.
Data from Scopus showed that the Hungarian Academy of Sciences (158), Eötvös Loránd University (110), and Semmelweis University (93) had the highest number of errata. In terms of the field or domain, 167 were in physics and astronomy, 165 in medicine, and 133 in biochemistry, genetics, and molecular biology. In terms of the collaborating countries, 191 documents were published in collaboration with researchers affiliated to institutions in USA; 187, in Germany; and 173, in the UK. Almost two-thirds (438, or 65.1%) of the papers were open access, but the rest (235, or 34.9%) were not. The top three journals with the highest number of papers associated with corrections were Scientific Reports (25), Journal of High Energy Physics (20), and Journal of Chemical Physics (12). Nearly all (97.8%) the entries were in English, 1.2% were in Hungarian, and 1.0% were in German. All these papers had received 874 citations in total, or 1.3 citations per paper. Nearly 70% had never been cited. The paper with the highest number of citations in WoS was also the most cited in Scopus, with 98 citations. Table 3 shows some bibliometric characteristics of retracted publications from Hungary (data from WoS and Scopus). The institution with the highest number of retractions in WoS was the University of Debrecen (3), followed by Budapest University of Technology and Economics, University of Pécs, and Semmelweis University (2 each). The most frequent subject areas were biotechnology, applied microbiology, cardiovascular system and cardiology, cell biology, and life sciences and biomedicine, with two documents each. Four of the ten retracted publications from Hungary in WoS were co-authored with researchers from either USA or Germany or both, and the rest with those from Canada, France, Romania, or Serbia. Nine of the retracted papers were openly accessible; two were from the Journal of Molecular and Cellular Cardiology, and each of the other eight was published in a different journal; all were in English. The retracted publications had been cited 205 times in the WoS core collection, averaging 20.  Table 1).

Bibliometric characteristics of retractions in Web of Science and Scopus
Data from Scopus showed that University of Debrecen (3) and Hungarian Academy of Sciences (2) had the highest number of retracted documents (Table 3). Biochemistry, genetics, and molecular biology and materials science were the two subject categories with the highest number of retracted publications (three documents each), followed by engineering, medicine, and physics and astronomy (two each). Four of the retracted publications were co-authored with researchers from USA or Germany of both, and the rest with those from France, Iran, Pakistan, Portugal, Romania, and Turkey (one each). Only two of the seven retracted publications were openly accessible. Two  were classified as retracted in RWD, but not in WoS and Scopus. One document (DOI = 10.1097/MPG.0000000000001904) was identified as an expression of concern in the RWD, but was classified as an original paper in WoS and Scopus.

Discussion
The number of errata and retractions related to papers from 35 European countries were counted, as retrieved from three databases (RWD, WoS, and Scopus).
The numbers were represented both as absolute numbers and as relative numbers (that is, for every 1000 papers); the latter were referred to as CI, for corrections index, and RI, for retractions index. Fang and Casadevall were the first to suggest that CI should be used for comparing the frequency of corrections in scientific journals, 10 and have extended the use of both CI and RI to the assessment of other entities such as countries. For example, although the UK had the highest number of entries in RWD, and ranked second in WoS and Scopus in terms of the absolute number of errata, its rank dropped to 31 (based on WoS data) and to 27 (based on Scopus data) when the ranking was in relative terms. In both Scopus and WoS, Malta was at the top in terms of CI, suggesting some consistency between the two databases. This consistency was also observed for retractions. In terms of RI, Serbia topped the list. Our assessment also included expressions of concern, which are included only in RWD. Retraction and replacement allows for data sets in the retracted papers to be republished after corrections, 11 avoiding, to some extent, the stigma associated with retractions. 12 Both CI and RI showed that Hungary's share of the retractions and corrections was much smaller than that of the majority of the other 34 European countries. Publications in Hungarian are not widely indexed in major international citation indices, and only 10% and 5% of the total publications from Hungary in Scopus and WoS, respectively, were in Hungarian. This suggests that corrections, including errata and retractions, related to papers in Hungarian are under-represented compared to those to papers in English simply because a large part of academic literature published in Hungarian is indexed neither in Scopus nor in WoS. The problem of duplicate data or publications in English and Hungarian probably needs a separate analysis. 13 Duplications were the most common reason for retraction among papers published by Croatian authors between 1990 and 2017. 14 In our data set, the most common reasons for the retraction of papers with a Hungarian affiliation were, in descending order, 1) problems with data, 2) problems with results, 3) problems with images, and 4) other problems including those with conclusions, analysis, misconduct by authors, some issues originating from journals or publishers, duplicate publications, miscommunication by authors and journals, conflict of interest, plagiarism, and copyright claims. Greater transparency and more details about the background of a retraction in retraction notices would benefit readers. 15 Although RWD offered wider coverage of retractions than Scopus and WoS, it had considerably fewer errata-related entries. In addition, seven articles in RWD were not found in WoS. This suggests that rankings of countries, authors, institutes, or any other entity based exclusively on data from a single database may differ from those based on another database. For example, none of the three databases gives the entity that issued the retraction 16 or authors that may have objected to the retraction notice; however, these aspects can also influence author-or journal-based metrics. Therefore, we suggest that additional details about these and other aspects of retractions (and to a lesser extent of errata) be included in these databases to make them more complete. Schmidt, for example, found that retracted publications that were labelled as such in PubMed were not indicated as being retracted in WoS-a discrepancy that can potentially affect the accuracy of conclusions of bibliometric studies that rely only on WoS. 17 A retracted paper the retracted status of which is incorrectly indexed or insufficiently labelled as such in a retraction notice as occurs in Scopus may continue to be cited. 18 Given the small size of the data sets used in this paper, readers are cautioned about drawing any conclusions on Hungary or on any of the listed countries or institutions in those countries based on any one, or even all, of these databases.
The ranks or corrected ranks in Table 1 should not be used to equate correction (errata or retractions) with misconduct, or to draw any other wider interpretations beyond the values indicated, because corrections can also result from unintentional or accidental errors. 19 For the RI, the number of retractions may be too low, resulting in some countries sharing the same rank (Table 1), but the ranks will separate out as the number of retractions increases. However, as more errors in published literature are detected, and corrected, the ranking of Hungary may rise or go down. Moreover, factors such as the total size of academia, the number of publications per researcher, the proportion of academic population to the total population, and economic indicators such as the GDP and income per capita were not considered in the present study but would make for a more insightful and comprehensive study in the future.
Although Hungary is a member of ENERI, the European Network of Research Ethics and Research Integrity, and of several ethics-related committees, 20 it is not -but should be -a member of ENRIO, the European Network of Research Integrity Officers, 21 and should participate in the activities of this organization because it integrates national policies on research integrity.
Although our focus was on Hungary, we examined it in relation to all other member countries and some major nonmember European countries. As one example of a paper focusing on retractions in the EU, in the field of engineering, the majority (57%) of retractions assessed from WoS from 1945 to 2015 were from Europe, mainly the UK and the Netherlands. 22 Despite the limitations mentioned above, including those of the databases used in this and other studies on corrections and retractions, we believe such analyses to be valuable because they help to build a wider picture of the complex interplay between the incidence of errors in academic publications and the mechanisms by which such errors are corrected, whether adequately or inadequately.    names of institutions were transliterated to include Hungarian characters to represent their names as accurately as possible. 2 Scopus erroneously indexes the authors by their first names (given names) rather than their last names (family names).