In addition to the known barriers to sharing research work openly (see https://doi.org/10.57874/7d3t-kv62), there is evidence for pressures that affect the quality of the research too, such as questionable research practices (QRPs), biases in research selection and assessment, and anti-collaborative practices.
Factors leading to QRPs
Quantity oriented environment and pressure to publish
One of the biggest underlying factors related to Questionable Research Practices is the so-called ‘publish or perish’ culture. A survey by Schoot et. al. (2021) emphasised that the modern research culture is quantity rather than quality oriented - a point supported by arguments from several previous reviews (Ball, 2016; Ding et al., 2020; Nosek et al., 2012; Schoot et al., 2021; Vuong, 2019; Werner, 2021). When academics are rewarded for the number of publications it “will pressure all but the most ethical scientists, to overemphasise quantity at the expense of quality, create pressures to “cut corners” throughout the system, and select for scientists attracted to perverse incentives” (Edwards and Roy, 2017). The pressure to publish high quantities leads not only to a decrease in the quality of publications but also to the dubious practice of so-called “salami-slicing”: dividing results from one study into many fragments and using them for different purposes as separate publications.
Pressure for positive findings
Another factor leading to QRPs is pressure to get positive results or novel findings. Several surveys (Fong and Wilhite, 2017; Fraser et al., 2018; José Perezgonzalez et al., 2021; Moran et al., 2022; Wolff et al., 2018) have found that research culture is oriented towards ‘novel’ findings, thereby encouraging researchers to QRPs to obtain them. Reproducibility is valued less than novelty, and those who have “clean” data and “significant” results win the race for recognition. This conclusion has also been supported by reviews (Bergkvist, 2020; Open Science Collaboration, 2015), meta-studies (Fidler et al., 2017; Nissen et al., 2016), qualitative analysis and analysis of secondary data (Baldwin et al., 2022; Gibelman and Gelman, 2005) and other sources (Laitin et al., 2021; Verma and Detsky, 2020). It is inextricably related to pressure to publish, since “statistically significant findings that are visually and numerically clean are easier to publish” (Diong et al., 2018). Such conditions force researchers to fight for publication and recognition and incline them towards misusing data analysis (p-hacking) and selectively publishing results.
Dependence on funding
A systematic review and meta-analysis by Fanelli (2009) found that 33% of respondents admitted ‘changing the design, methodology or results of a study in response to pressures from “a funding source’’. Other studies have noted pressure from grant schemes (Edwards and Roy, 2017; Fanelli et al., 2015; Huistra and Paul, 2022) and demonstrated imperfections of the science system. According to some interviews conducted with scientists, competition for money and dependence on grant funding negatively affects research integrity. The expected results of grant schemes are the funding of research programs and promoting growth, however the actual results are lack of time for gathering and thinking about data and focusing on getting the positive results. (Edwards and Roy, 2017).
Hierarchical pressure from superiors/competitive environment
Another factor that can affect research integrity is hierarchical or environmental pressure. Some meta-analyses (Fanelli et al., 2015) and reviews (Rupp et al., 2019; Sharma and Verma, 2018) conclude that it strongly affects young researchers, who are trying to build a reputation and might be subject to criticism from colleagues. According to a mixed-method study by Gerrits et al., (2020) and some surveys, competitiveness and hierarchical pressure had a negative impact on research integrity (Gopalakrishna et al., 2022; Metcalfe et al., 2020). Competitive environments make scientists focus on the speed and statistical significance of their research in order to build their CVs, get funding or gain promotion. This affects research integrity and the quality of findings.
Lack of clear policies
Several reviews and commentaries (Bouter, 2015; Ding et al., 2020; Kiri et al., 2018) mention that scientists committed misconduct for two reasons. The first is confusion in definitions of “good” and “bad” practices. The second is existing gaps in policies and regulations that allow the use of QRPs with low risk of consequences.
Factors leading to publication bias
‘Positive’ versus ‘negative’ findings
Multiple studies have demonstrated that ‘positive’ research findings are favoured in the literature, with statistically nonsignificant results being less likely to be published (Fidler et al., 2017; Fong and Wilhite, 2017; Vuong, 2019). Several studies have also found that current research culture undervalues null or negative findings, which in turn causes researchers to make changes in statistics (for example, p-hacking, data dredging, selective reporting) and present statistically significant or positive results in order to get their work published (Brembs, 2018; Carbine et al., 2019; Nissen et al., 2016; Smaldino et al., 2019; Schweitzer and Schulz, 2018; Verma and Detsky, 2020: Stanley et al., 2022). A meta-research study by Duyx (2017) found that papers with ‘positive’ findings are cited twice as often as ‘negative’ ones. Pressure to publish and the desire and incentive to be cited can lead to decrease of quality and reliability of studies.
The factors which make it difficult to publish null or negative results are the same as those that lead to QRPs. Firstly, null findings are seen to be less valuable for publishing by both journals and scientists (Nissen et al., 2016; Ioannidis et al., 2014). Under the pressure to publish, authors may “cherry pick” the information and publish only positive results (Schweitzer and Schulz, 2018). According to Nissen et al. (2016), publication bias is so strong in the current research climate that a significant part of scientific literature does not present negative results at all. This encourages a self-fulfilling cycle. For example, several reviews (Brembs, 2018; Vuong, 2019) indicated pressure to publish and the trend of “not to publish negative results” as motivating factors for scientists to selectively publish positive findings. This, in turn, leads to biased knowledge (Schweitzer and Schulz, 2018).
Research in certain journals
A quantitative research study which analysed poorly and well -cited articles in orthopaedic journals (Kortlever et al., 2019) concluded that there is no difference between the proportion of poorly cited articles in subscription-based and open access (OA) journals. This study found that 36% of the total analysed articles were defined as ‘poorly cited’ five years after publication. Unfortunately, this study did not provide information on factors that led to this situation, but it suggested there is more polarisation in what research receives attention. Other research has shown that the platform of publishing impacts the number of citations, with articles from better-known platforms more likely to be cited. For example, according to (Wakeling et al., 2016), scientists are more likely to cite articles from PLOS ONE and Scientific Reports than other less well-known mega-journals.
Field of study
Quantitative bibliometric analysis by Larivière et al., (2015) has shown that there is a difference between publishing papers in natural and medical sciences (NMS) and social sciences and humanities (SSH). The results of the analysis revealed that during the transition to the digital environment, social science communities began to publish their work in giant commercial publishers (Reed-Elsevier, Wiley-Blackwell, Springer-Nature, and Taylor & Francis), while representatives of the medical sciences preferred to stay more independent and publish in smaller publishers. This happened because the social science communities were more dispersed and were likely to have fewer resources to adapt to the digital age. Therefore, for instance, social scientists were more likely to have agreements with commercial publishers. Consequently, 70% of papers from the top five publishers belong to the social sciences field.
Another problem that has been noted in literature is related to the funding of different types of research. Some studies mentioned a problem of unfair funding stratification among different types of research, for instance, education research and implementation research are underfunded (Duyx et al., 2017; Yarris et al., 2014). According to Yarris and colleagues (2014), there are a few available grants that are not enough for covering research projects’ needs.
Factors leading to bias in the assessment of research quality
Demographic biases
Gender bias in research assessment has been identified by many studies. Cruz- Castro and Sanz-Menendez (2021), Fox et al. (2015), Johnson et al. (2021), Morales et al. (2021), and Silberzahn et al. (2018) demonstrated that bias against female researchers exists in various disciplines. This finding also appeared in reviews and included such disciplines as medical science (Toews et al., 2017; Upthegrove et al., 2021), economics (Rousseau, 2021), ecology (Eisen et al., 2013; Sing et al., 2017), chemical sciences (Bennie and Koka, 2021), and biology (Resnik et al., 2008). The bias against women manifests in multiple ways. Firstly, an analysis of conference abstracts and whether they were eventually published in a journal found that the “last author's female gender was predictive of a lower likelihood of publication” (Johnson et al., 2021). Previous studies have mentioned that the assessment process for journal publications might be affected by subjectivity of editors. Lack of gender diversity of editorial boards leads to disparity of published papers. In journals, where editors were male, there was less proportion of female reviewers (Fox et al., 2015). Secondly, there is an imbalance in the review process which appears because of the tendency of editors to invite reviewers “like themselves”. Some reviews (Detweiler et al., 2016; Eisen et al., 2013; Upthegrove et al., 2021) and a survey (Morales et al., 2021) found that women are less likely to receive invitations for peer review and also that representation of women in positions of senior author is less than that of men. A survey by Gunthe and Gettu (2022) showed that the output and quality of research publications by some academics (especially women and early career researchers (ECRs) who moved between institutions, changed career paths or had a pause in research activities) are not fairly assessed.
Geographical and language bias
These biases affect researchers from low-middle income countries and those whose first language is not English. The current research culture prioritises English because all high-ranking journals are in English (Baltazar et al., 2019; Naik, 2017) and publication in English increases the likelihood of citation (Vinkenburg et al., 2021). Therefore non-English speaking researchers experience problems with readership of their works (Lawrence, 2007). The prioritisation of the English language in the academic literature creates a disadvantage for non-native English speaking authors and forces them to spend extra time and resources getting their work published (Hagan et al., 2020).
Another factor which may result in bias against researchers is under-representation of reviewers from low-middle income countries (LMIC). Reviewers mostly come from high-income countries, as academics from low-income countries do not have time and resources to do additional jobs. “Having reviewers mainly from high- income countries means that the interest of these scientists and populations are perpetuated, and those in low-resource settings are marginalised” (Cheah and Piasecki, 2022). Same tendency appeared in a survey by Publons (2018) that has evidence that reviewers from low-middle income countries are not invited to review academic papers. This means that researchers from low-middle income countries are not equally included in the evaluation process.
Bias for author and institutional prestige
Manuscripts are frequently assessed for publication based on status of authors or institutions or other subjective factors (Detweiler et al., 2016; Cazap et al., 2020; Eyre-Walker and Stoletzki, 2013), which makes reviews biassed (Nestor et al., 2020). The fact that reviewing and decision-making often does not happen openly (Bonn and Bouter, 2021; Siler et al., 2015; Wicherts et al., 2012) enables assessors, even inadvertently, to prioritise manuscripts based on their personal biases. For instance, assessors may favour articles which cite famous authors (Urlings et al., 2021; Gøtzsche, 2022), show “significant” findings (Cazap et al., 2020; Ekmekci, 2017; Jannot et al., 2013), or whose authors are based at institutions with more prestigious reputations.
Anti-collaborative practices
Division of labour
Work has been done on how best to foster collaborative research practices. Some authors distinguish different types of contributions during the research process. For example, a mixed-method study evaluating a technological collaboration tool (Julpisit and Esichaikul, 2019) analysed knowledge sharing practices of research teams and concluded that collaborative activities could be categorised into four types: identifying research goals, designing tasks, performing tasks, and writing reports. "A survey of research teams across a range of scientific disciplines (Lee et al., 2015) found that while the impact of research increased with team size, the novelty of research was boosted by a variety of team members with distinct knowledge bases". In relation to kinds of activities, some studies mention the importance of “collaborative supportiveness” (Liu et al., 2013; Woodzicka et al., 2015). It may increase productivity by providing a broader understanding of the research process, and improve research culture by supporting individual contributions to team activities. This encourages more collaborative ways of thinking among researchers and highlights research contributions of individual researchers. Other studies note that it is important to value all types of contributions as this affects the productivity of research collaboration and the research culture itself (Lariviere et al., 2021; Mauthner and Doucet, 2008; Wolfe and Alexander, 2005). Despite the fact that many studies have a similar structure (conceptualisation, operationalisation and written communication), other, more niche and narrow tasks cannot be ignored. When one type of work is perceived as more worthy than another, it leads to inequalities across disciplines and teams. Another study by Haeussler and Sauermann (2020) found that interdisciplinary teams have greater division of labour.
Generalist vs specialist roles
Researchers’ ways of working have been categorised into several types. Studies, both qualitative and quantitative, describe potential roles for researchers as: generalists (who are team players), specialists (who work alone) and versatiles (who do both). According to Lu et al. (2020), which used more than 100,000 articles from PLOS and extracted author contribution statements, generalists are the majority. (Note, however, that this method cannot separate whether most authors are actually generalists, or whether contribution statements might be inflated due to pressure to appear as generalists.) A qualitative study (Haeussler and Sauermann, 2020) analysed pre-defined contribution statements from PLOS and found that roughly 22% of authors perform 20% or less of all contributions (“specialists”), while 29% perform more than 60% of all contributions (“generalists”). As for the authorship of articles, versatiles are more likely to be first authors (Lu et al., 2022), confirming past studies (e.g. Lu et al., 2020) who found that versatiles are most often senior authors and are associated with funding and supervision.
All of these potential barriers come together to make it difficult for researchers wanting to carry out best research practice, and make their work freely available for others to build on, feel that they can do this and succeed in an academic career.