Through a <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.57874/1c7r-nc49">survey</a> (Hsing et al. 2023a) and <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.57874/tx7n-ff73">interviews</a> (Hsing et al. 2023b), we previously conducted a baseline evaluation of research culture across different disciplines, including obtaining researchers' views on what may motivate or hinder sharing research more openly.In this second stage, we will continue the evaluation, but focus on feedback about the Octopus platform itself. To obtain this feedback, we will ask participants in the study: <ol><li>How is the Octopus user interface/user experience (UI/UX) received by researchers? Specifically: <ol><li>How effectively is the website communicating the goals of Octopus to different types of researchers, and how it works?</li><li>How discoverable, understandable, and usable are the features of Octopus? What are the facilitators and barriers to their use?</li></ol></li><li>How well does Octopus fit into existing research culture and infrastructure? We would like the perspectives of: <ol><li>Individual researchers</li><li>Institutional research infrastructure managers</li></ol></li></ol><h1>Methods</h1>The methodology of this evaluation is primarily a focus group with researchers focused on the user experience of Octopus. In addition, we plan to hold semi-structured interviews with institutional research infrastructure managers which is described here. These infrastructures could be institutional data repositories, library services, or support and training that are provided to researchers. We will focus on whether and how Octopus would fit into their institutional structures for supporting research. We will conduct 60-minute online interviews using the Zoom video conferencing tool with at least five participants, with audio recordings transcribed into text for analysis. The recordings will be deleted once the transcripts are complete with personally-identifiable information removed.The methods described here have received ethical approval from the University of Bristol School of Psychological Science Research Ethics Committee with reference number 12335.<h1>Participants and recruitment</h1><h2>Participant characteristics</h2>We would like to obtain feedback from institutional research infrastructure managers. An example of such a manager could be those in charge of supporting researchers at the University of Bristol through the use of the <a target="_blank" rel="noopener noreferrer nofollow" href="https://data.bris.ac.uk/data/">data.bris</a> institutional data-sharing platform (this is an example from our institution, we will not interview this specific individual). We will strive to reach those in equivalent roles at other institutions. Depending on the institution, they could be librarians, research support staff, or heads of research. For these infrastructure managers, because they typically operate on the institution level, we believe their experiences and views would cut across many of the criteria we defined for our focus group participants. To be clear, this means we will not be targeting those maintaining the technical infrastructure, such as technicians.<h2>Participant recruitment</h2>Similar to the previous evaluation, we have a diverse and international network of professional contacts, and participant recruitment could begin with a snowball approach from a convenient sample from this network. In addition, we can recruit participants through professional organisations. Other potential ways to reach participants include:<ul><li>Contacts of Octopus within universities (mainly in Europe, including the United Kingdom) </li><li>Contacts in private companies</li><li>National laboratories/research institutions in the United Kingdom and United States.</li></ul>We anticipate that, time-wise, participant recruitment and scheduling the study will probably be the least predictable step. Each participant will be offered an incentive worth GBP 25 in the form of an online shopping voucher or donation to a non-profit organisation.<h1>Materials and measures</h1>The following is the interview guide we will use: <table><tbody><tr><th colspan="2" rowspan="1"><h2>Introduction (5 mins)</h2></th></tr><tr><th colspan="1" rowspan="1">Question</th><th colspan="1" rowspan="1">Prompts</th></tr><tr><th colspan="1" rowspan="1">Housekeeping</th><th colspan="1" rowspan="1">Welcome participant and thank for their timeCheck if there are any issues with consent formSound check (no need to do explicitly)Explain how long interview will take and that they can stop/take a break at any timeEstablish procedure in case of connectivity issues (i.e. try to rejoin ASAP and resume interview when ready)Any questions?</th></tr><tr><th colspan="1" rowspan="1">Ask consent to record</th><th colspan="1" rowspan="1"></th></tr><tr><th colspan="1" rowspan="1">**start recording**</th><th colspan="1" rowspan="1"></th></tr><tr><th colspan="1" rowspan="1">Introduce the aims of the study</th><th colspan="1" rowspan="1">To find out more about participant’s: a. 	Role in supporting research infrastructureb. 	Immediate reactions after being introduced to Octopusc. 	Views on if and how Octopus could be a part of their service offering to researchers in their institutionsd. Views on what their counterparts at other institutions might think</th></tr><tr><th colspan="2" rowspan="1"><h2>Part 1: Participant’s role (10 mins)</h2></th></tr><tr><th colspan="1" rowspan="1">To start off with, could you tell me a bit about the nature of your work in supporting researchers?</th><th colspan="1" rowspan="1">Try to keep it under 2 minutes - don’t let them get carried away!</th></tr><tr><th colspan="1" rowspan="1">For the research you support, what kinds of output do they produce?</th><th colspan="1" rowspan="1">Methods: Quantitative, qualitative, field, etc.What kinds of output do these forms of research produce?</th></tr><tr><th colspan="1" rowspan="1">I’d like to zoom out to your institution, what other ways does it support researchers to share research outputs?</th><th colspan="1" rowspan="1">Does it work well or poorly? In what ways?What, if anything, would you change about it?</th></tr><tr><th colspan="1" rowspan="1">If there is one thing you can change to help you support researchers in sharing more of their research, what might it be?</th><th colspan="1" rowspan="1">To identify key barriers to research sharing.</th></tr><tr><th colspan="2" rowspan="1"><h2>Part 2: Introduction &amp; reactions to Octopus (15-20 mins)</h2></th></tr><tr><th colspan="1" rowspan="1">Interviewer gives a 10 minute introduction to Octopus via shared screen</th><th colspan="1" rowspan="1">It will cover: <ul><li>Motivations for Octopus</li><li>The 8 publication types on Octopus</li><li>How they form the “version of record” - everything in full, in one place</li><li>Branching chains of research</li><li>Finer grained attribution for researchers</li><li>Can be used for pre-registration, like pre-printing or post-printing.</li><li>Differences with traditional journal articles: sits alongside - like having all the supplementary information in there.</li><li>If time permits, could share two example publications: <ul><li><a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.57874/gv8n-ap77">https://doi.org/10.57874/gv8n-ap77</a></li><li><a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.57874/xw90-dm13">https://doi.org/10.57874/xw90-dm13</a></li></ul></li></ul>Participants can interrupt and ask clarifying questions throughout this introduction. This means that this part might last longer than 10 minutes, and the interviewer will carefully watch the time, leaving enough for the next part.</th></tr><tr><th colspan="2" rowspan="1"><h2>Part 3: How might Octopus fit into research support infrastructure? (10 mins)</h2></th></tr><tr><th colspan="1" rowspan="1">What do you think of Octopus?</th><th colspan="1" rowspan="1">What are your initial reactions?</th></tr><tr><th colspan="1" rowspan="1">How might Octopus fit into research support infrastructure?</th><th colspan="1" rowspan="1"></th></tr><tr><th colspan="1" rowspan="1">What are some barriers that might prevent your institution from including Octopus as part of its offering of support for researchers?</th><th colspan="1" rowspan="1"></th></tr><tr><th colspan="1" rowspan="1">In contrast, what have you seen today that would help/facilitate your institution to include Octopus in its offering?</th><th colspan="1" rowspan="1"></th></tr><tr><th colspan="1" rowspan="1">How do you think those providing research support infrastructure at other institutions might think about Octopus?</th><th colspan="1" rowspan="1"></th></tr><tr><th colspan="2" rowspan="1"><h2>Closing (5 mins)</h2></th></tr><tr><th colspan="1" rowspan="1">Is there anything that we haven’t discussed that you’d like to talk about before the end of the interview?</th><th colspan="1" rowspan="1"></th></tr><tr><th colspan="1" rowspan="1">**stop recording**</th><th colspan="1" rowspan="1"></th></tr><tr><th colspan="1" rowspan="1"></th><th colspan="1" rowspan="1">Thank the participant for their time.Explain next steps (debrief, final consent etc)Explain data management procedures (i.e. when will interview recordings be deleted, how will transcripts be anonymised etc)Ask if they would like to be acknowledgedAny questions?Contact details for questions/concerns</th></tr></tbody></table>

Interviews with research infrastructure managers at various institutions, to understand if and how Octopus may fit into their offering for researchers.

Interviews with research infrastructure managers on their perceptions of Octopus

Through a <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.57874/1c7r-nc49">survey</a> (Hsing et al. 2023a) and <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.57874/tx7n-ff73">interviews</a> (Hsing et al. 2023b), we previously conducted a baseline evaluation of research culture across different disciplines, including obtaining researchers' views on what may motivate or hinder sharing research more openly.In this second stage, we will continue the evaluation, but focus on feedback about the Octopus platform itself. To obtain this feedback, we will ask participants in the study: <ol><li>How is the Octopus user interface/user experience (UI/UX) received by researchers? Specifically: <ol><li>How effectively is the website communicating the goals of Octopus to different types of researchers, and how it works?</li><li>How discoverable, understandable, and usable are the features of Octopus? What are the facilitators and barriers to their use?</li></ol></li><li>How well does Octopus fit into existing research culture and infrastructure? We would like the perspectives of: <ol><li>Individual researchers</li><li>Institutional research infrastructure managers</li></ol></li></ol><h1>Methods</h1>The methodology of this evaluation is primarily a focus group with researchers, focused on the user experience of Octopus. Participants would be emailed a link to Octopus before the session, and shown a brief demonstration at the beginning of the focus group.Then, the focus group would commence with some guiding questions on if and how Octopus would fit into a researcher’s workflow, and how they perceive the wider culture will respond to this platform. These open-ended questions will provide insight into what may facilitate or hinder the adoption of Octopus.We intend to run at least two focus groups with five participants each for a total of at least 10 researchers. Online focus groups will be conducted using the Zoom video conferencing tool. Ideally, at least one of the focus groups would be in-person, which may be more conducive to a stimulating discussion among the participants. For the in-person focus group, it will likely be with researchers at the University of Bristol (where we are physically based) due to logistical constraints.We aim for focus groups to last approximately 60 minutes, with audio recordings that are transcribed into text for analysis. The recordings will be deleted once the transcripts are complete with personally-identifiable information removed.The methods described here have received ethical approval from the University of Bristol School of Psychological Science Research Ethics Committee with reference number 12335.<h1>Participants and recruitment</h1><h2>Participant characteristics</h2>For individual researchers, we will strive to recruit those who represent diversity across the following characteristics: <ul><li>Basic dimensions of diversity such as gender</li><li>Geographical location, including outside of the Global North</li><li>Research discipline<ul><li>Natural sciences</li><li>Social sciences</li><li>Engineering</li><li>Arts and humanities</li></ul></li><li>Research methods employed<ul><li>Qualitative</li><li>Quantitative</li><li>Theoretical</li><li>Applied (e.g., engineering)</li></ul></li><li>Sector<ul><li>Academic</li><li>Industry (ideally not just the pharmaceutical industry)</li><li>Ideally some from none of the above</li></ul></li><li>Publish other research objects (i.e. not code or “data”) such as: <ul><li>Multimedia</li><li>Hardware designs</li><li>Digitised resources</li><li>Non-article-based publication types (e.g. monographs, art pieces)</li></ul></li></ul><h2>Participant recruitment</h2>Similar to the previous evaluation, we have a diverse and international network of professional contacts, and participant recruitment could begin with a snowball approach from a convenient sample from this network. In addition, we can recruit participants through professional organisations. Other potential ways to reach participants include:<ul><li>Early-career or postdoctoral researcher networks</li><li>Professional societies in various disciplines</li><li>Contacts of Octopus within universities (mainly in Europe, including the United Kingdom) </li><li>Contacts in private companies</li><li>National laboratories/research institutions in the United Kingdom and United States.</li></ul>We anticipate that, time-wise, participant recruitment and scheduling the study will probably be the least predictable step. Each participant will be offered an incentive worth GBP 25 in the form of an online shopping voucher or donation to a non-profit organisation.<h1>Materials and measures</h1><h2>Pre-focus group activity</h2>Approximately one week before the focus group, we will email and request participants to visit Octopus (<a target="_blank" rel="noopener noreferrer nofollow" href="https://www.octopus.ac/">https://www.octopus.ac/</a>) with guiding questions including: “In these 5 minutes, can you…”: <ol><li>Find and understand the key motivations behind Octopus?</li><li>Identify the types of research that could be published on Octopus?</li></ol><h2>Focus group plan (approximately 60 mins)</h2><ol><li>Quickly introduce yourselves (10 mins) - In two minutes, tell us about your career stage and the nature of your research, including methods and outputs.</li><li>Octopus demonstration - Where the facilitator goes through the Octopus.ac platform, giving an overview of its motivations and key features (15 mins)<ul><li>This could include using a demo account to publish something in real time.</li></ul></li><li>Barriers/facilitators to using Octopus (15 mins) - Would you want to use Octopus? What about Octopus would make you want or not want to use it?</li><li>Perception of how others might react (10 mins) - Do you think others in your field would want and will use Octopus to publish research? Why or why not?</li><li>Feedback form (5 mins) - Dedicated time for participants to provide additional feedback by answering these questions in writing (no identifying information will be collected, and participants will be asked to not provide it): <ul><li>Which research discipline(s) do you primarily identify with?</li><li>On a scale of 1 to 5 (with 5 being highest), how familiar are you with open research practices (broadly defined)?</li><li>Does the organisation of publications on Octopus fit with the research workflow of your discipline(s)? Please explain.</li><li>What would convince you and others in your discipline(s) to make use of Octopus to publish research earlier and more often?</li><li>Do you have any other critique or feedback for Octopus or today's session?</li></ul></li></ol>

Focus groups with researchers from various fields of research to understand their initial perceptions of Octopus.

Focus groups to understand researcher perceptions of Octopus

Pen-Yuan Hsing has a PhD in biology, with highly multidisciplinary experience ranging from ecology and conservation, engineering, citizen science, to meta-research on open research best practices. Having developed and published relevant training material, online courses, books, and policy documents on the international level, Pen is a strong advocate for the idea that good research is open research. As someone who belongs to an ethnic minority group during their research career, Pen is particularly sensitive to issues of diversity related to geographical origin and language. Alex Freeman has a DPhil in biology, which she followed with a career in factual television and the media. She has spent the last 6.5 years working in an interdisciplinary group in academia on evidence communication (funded by the David & Claudia Harding Foundation) and here came up with the concept of Octopus. She is the sole Director of Octopus CIC which is a UK-registered not-for-prot company, from which she derives no salary. She does unpaid work advocating for and developing Octopus in collaboration with Jisc, and is also a strong believer in Open Science practices and research transparency. Octopus is currently funded by Research England, and has previously had awards from Mozilla, the Royal Society and an anonymous philanthropist. Tim Fellows has a BSc in Business Economics, with experience as a product manager on various research management platforms. He currently works for Jisc as the product manager for Octopus, and is responsible for overseeing the growth and development of the platform alongside Alex Freeman. Marcus Munafò has a PhD in health psychology, and has worked across a range of disciplines in the biomedical sciences (public health, primary care, clinical pharmacology, psychiatry, epidemiology). He is a proponent of open research and scholarship, and co-founded the UK Reproducibility Network, which receives funding from several major funders, including a Research England Development Fund award to promote open research practices. He is also co-director of the Tobacco and Alcohol Research Group (TARG), which is based within the School of Psychological Science at the University of Bristol, and a Programme Lead within the MRC Integrative Epidemiology Unit at the University of Bristol. Jackie Thompson has a PhD in experimental psychology, followed by several years postdoctoral research experience in various sub-disciplines of psychology and meta-research. She has spent several years as an advocate for open research practices within psychology and academia more broadly, including working with the UK Reproducibility Network on several initiatives, mainly due to training researchers in open research practices.

This research suggests that open research publication platforms such as Octopus could be an important mechanism for achieving various reforms, but need to be supported by those who carry out assessments for funding or for career advancement.&nbsp;For example:&nbsp;<ul><li>Outreach and education about Octopus (and other similar platforms) would alleviate a substantial barrier to sharing, as respondents to the survey and interviews often do not know how and where to publish non-paper-based research outputs.</li><li>Funder and institutional policies need to be checked to ensure they reward and do not hinder good research sharing.</li><li>Overtly recognising the value of good ideas and methodological innovations as much as data or results would help address concerns about “scooping”, whereby researchers feel that they only get credit for work if it involves a “finding” (and a “positive” finding at that).</li><li>The eight Octopus publication types can both allow recognition for forms of research other than “results”, and bring recognition to specialists – such as data scientists or data collectors – especially if research assessment can recognise these contributions.</li><li>Breaking the link between “results” and other parts of the research process can, as in registered reports as well as modular publication platforms such as Octopus, mitigate the pressure for QRPs and publication bias in order to produce “positive findings”.</li><li>Peer review (and the proposed ratings system) within Octopus, focussed as it is on smaller publications than an entire “paper”, could help assess the intrinsic qualities of research, without such an assessment being influenced by the potential findings and implications of it, again focussing the incentives on research quality and minimising publication bias and the pressures for QRPs.</li><li>The Octopus open peer review mechanism could satisfy the strong desire for early feedback when developing studies, particularly if the risk of retribution can be mitigated.</li><li>The removal of first names and institutions from the top of publications within Octopus (which could potentially be extended to an entire replacement of names by <a target="_blank" rel="noopener noreferrer nofollow" href="https://orcid.org/">ORCID iD</a>s) helps remove some cues for unintentional bias (gender and institutional) by making a reader follow a link to find out more details of any individual author. This allows readers to read and assess the quality of a publication more on its own merits, rather than unintentionally noticing gender and institutional cues that might bias their assessment.</li><li>The use of automatic language translation (with care, to avoid mistranslation incidents) could help reduce the biases faced by non-native English speakers.</li></ul>During the interviews, participants struggled to break down research in their disciplines into an ordered, discrete set of steps. In addition, we received criticism during the survey that it was framed around the natural and applied sciences without consideration for other fields, such as the arts and humanities. This reflects the diversity in how research is done within and across disciplines, and where Octopus could be placed within the wider open research publishing ecosystem. For example, while Octopus presents itself as the “global primary research record”, its eight ordered publication types might not be the one size that fits all forms of research. Indeed, it is unlikely that the “research” Octopus currently claims to represent includes fields such as, but not limited to, the arts and humanities. This should be clarified on its website. In contrast, some platforms allow publishing individual components of research without an overarching structure. For instance, GitHub is commonly used to publish the code behind scientific software, though it was not originally conceived for that purpose.&nbsp;Some of the issues revealed by our study – such as the culture around chasing novelty and a “good story” – might need to be tackled through using a publishing platform like Octopus (which offers itself as a venue where there is no need for a story) as places to carry out research assessment, breaking the current perceived link between “good story” and “good research”.&nbsp; Such platforms can&nbsp;sit alongside outlets where a story is the driver of readership (and those story-driven articles could be commissioned by such outlets, and potentially even authored by, and with commensurate credit to, story-writing specialists such as science writers and journalists). This could be part of a broadening of what counts as a research “output” in assessments to include non-narrative publications. It could also improve recognition for specialist contributions – such as that from statisticians, methodological experts, or local “fixers” – and be part of a movement to include criteria for doing open research in assessments. These reforms should be sensitive to the fact that researchers, especially academics, are already overburdened, and for open research to be prioritised and become the norm, other dis- or mis-incentives should also be tackled so that what is being assessed is “different” and not “extra”.In summary, Octopus, while not sufficient in itself, could be seen as a necessary part of realising systematic reforms in research culture, especially with regards to research sharing and assessment. However, other players within the system (such as funders, institutions, and anyone carrying out research assessment) need to be aligned to create an environment that incentivises best practice and open sharing and collaboration.

The real world applications of findings about the current research culture

The two strands of research revealed a wide variety of barriers to more open sharing of, and unbiassed, best quality research. While some are related to perceived or experienced biases based on personal characteristics such as gender or inequitable access to support, most result from a research culture that primarily assesses achievement and quality through traditional, peer-reviewed papers. This focus, and the resulting competition, encourages researchers to hide their work at least until a traditional journal paper is published. In some situations, these pressures lead to questionable research practices (QRPs), such as data manipulation to achieve an “interesting” or statistically significant result more likely to appeal to a journal with higher impact metrics or perceived “impact”. In general, open research practices are viewed as not beneficial, or even detrimental, to job security and career advancement. This is especially true given competing demands and the need for academics to prioritise their time on outputs that count in assessments that they are subject to.Barriers and hierarchies that restrict research sharingLack of time was one of the most prominent barriers to research sharing, as noted in the literature review, interviews, and the survey. Data from the interviews suggested that this may arise from researchers being incentivised to prioritise other activities, such as publishing papers in high impact journals or networking to gain influence and power, which directly benefit their careers.The interviews and survey also revealed that the fear of scooping constitutes a major barrier to sharing research. This is related to a highly competitive culture which places great value on being first with “findings”. And since traditional journal articles are the only recognised way for receiving credit for research, other forms of sharing such as publishing methods or data are heavily discouraged. In addition, since institutions tend to reward flashy research – such as those with a “sound-bitey” story or can claim very high and dramatic “impact” – researchers are afraid of sharing outputs which may be immature (e.g., intermediate work that is still subject to revision) or incorrect.There is also inequitable access to resources – such as funding – to support research sharing. This can be seen through our interviews and survey, where respondents are mostly based in the Global North and typically did not express these problems. On the other hand, existing literature shows that, among other global inequities, researchers from less affluent backgrounds are being excluded from academic discourse simply because they cannot afford to publish their work in the same venues.Pressures that lead to questionable research practices (QRPs)Some interviewees discussed how the untenable stresses of academia force researchers into QRPs, such as data manipulation. Existing literature reveals several factors leading to this stressful environment, such as a focus on the quantity of papers rather than quality; or the pressure to obtain only positive or novel results. These factors are mirrored in our survey results, where the trendiness or novelty of research is viewed as at least as influential (and possibly more so) as methodological rigour.Factors that lead to certain topics and types of research being favouredWe did not find specific factors leading to certain research topics being favoured over others. Instead, our findings in this area showed that factors leading to QRPs are similar to those favouring the publication of certain types of research. This includes a research culture which values positive or flashy results. For example, one meta-study in our literature review found that papers with positive findings are cited twice as often as those without. Journal editors also consider the newsworthiness of an article which favours research with dramatic results.Bias in the assessment of publications and their qualityOur study, together with those of others, has revealed multiple causes for bias in research assessment. One is discrimination based on the personal characteristics of the researcher, which may be, but not limited to:&nbsp;<ul><li>Gender</li><li>Primary language</li><li>Geographical location</li><li>Personal and institutional prestige</li></ul>In addition, one of the largest sources of bias in assessment is a perceived over-emphasis on a researcher’s publication record. Metrics tend to measure quantity over quality, to the extent that competition to put one’s name on a paper matters more than its content. In this race, power and seniority often decide authorship, while leaving behind junior researchers or specialist contributors. The latter could include those in “plumbing” roles like statisticians, or those with crucial tacit knowledge such as local “fixers” in a social sciences study. And because of this bias, researchers feel pressured to expend substantial effort on networking to build up connections and prestige.According to our interviews, institutions – such as universities, academic journals, or funders – also tend to favour research with a “good story” with “impact”, and researchers are pressured to present their work in those terms. Similarly, existing literature suggests that “novel” findings are favoured. Our interviewees lamented the need to be “sound-bitey” where "...it almost feels like [...] I'm a novel writer instead of a researcher.” They fear that this bias disadvantages the mundane but important “grunt” work in research that does not have glamorous narratives.More openly collaborative ways of thinking and workingExisting literature focused on the value of collaboration in research, such as increased productivity, or the development of novel ideas via contributors with different backgrounds. Currently, collaboration occurs more often for higher-level tasks such as when defining the research question, interpreting analyses, or applying results to a different context.While those we interviewed described many problems with the research cultures they inhabit, they also suggested solutions:&nbsp;<ul><li>Focus on processes instead of outputs - Instead of measuring the number of papers that a researcher has published, focus on providing useful feedback on how their work is conducted. Interviewees consistently stressed that a key benefit to the more open sharing of research is receiving feedback, especially during its early stages to improve methodology.</li><li>Value null or negative results - Research culture should value null or negative results at least as much as positive ones with a flashy story. Similarly, researchers should be rewarded for being open about mistakes and what could be learned from them.</li><li>Recognise diverse forms of contribution - Authorship on traditional papers do not reflect the diverse forms and magnitude of contributions to research. There should be a more granular approach to giving credit that reflects contributions throughout the research lifecycle from ideation to execution. This can also partly alleviate concerns about scooping.</li><li>Provide space and recognition for specialists - Academics are required to be generalists, taking on roles outside of research such as teaching, administration, fundraising, mentoring, among others. Rather than being good at everything, many interviewees wish for space to focus on their strengths in research and be recognised for it with a secure career.</li><li>Assessments should be assessed - The way assessments for funding and career advancement are conducted should be subject to rigorous, scientific scrutiny and improvement just like any other research.</li><li>Provide opportunities for feedback outside traditional peer review - There is widespread sentiment among interviewees that critique of research is crucial, especially in its early stages. Currently, however, feedback is only formally given during peer review of traditional papers. Outside of that, researchers are hesitant to give unsolicited critique, sometimes with fear of retribution.</li></ul>Some of these solutions are congruent with the aims of Octopus and the problems it can solve.Figure 1 summarises the issues that affect researchers and the research culture, as identified by researchers themselves through our interviews or survey, and from the existing literature on the topic.<img src="https://science-octopus-publishing-images-prod.s3.eu-west-1.amazonaws.com/clo5rxcx7000fme08rfv68k7j" alt="A diagram of a diagram

Description automatically generated with medium confidence"> Figure 1. High level diagram of issues as identified from our evaluation. The pathway from assessment to recognition passes through the lenses of bias and incentives - the latter causing additional pressures towards q research practices - which tend to result in inappropriate recognition. The whole process takes place within a competitive environment, enhanced by the inappropriate recognition (especially the lack of reward for specialism) which also raises barriers to sharing and open practices.Essentially, assessment (either for funding or for career advancement) drives the incentive system for research and researchers, and it is focussed heavily on what researchers publish (since this is currently the tangible, public output of research that can be assessed). Assessment, of course, creates a competitive environment - which is inevitable when funding and career opportunities are limited.This competitive environment, then, surrounds the publishing system, as the main arena for researchers to try to gain recognition. Unfortunately, the path through the publication system to recognition is affected by the filters of biases (many unintentional but inevitable without mitigation, such as bias against non-native English speakers), and by a system of incentives that do not support best practice because they have not been designed with that in mind (such as the pressure for “novelty” or “a good story”).The poor incentives tend, instead, to cause pressure for QRPs, and those, alongside the action of the biases, can lead to inappropriate recognition rather than recognition on true merit.The whole system also inevitably drives a number of barriers to sharing research and its outputs, since these are not incentivised, and indeed the competitive environment sometimes actively acts to make it better not to share. Because of the lack of recognition for open sharing practices, many researchers do not have the time or financial resources to become trained in them, or carry them out.The high prevalence of inappropriate recognition itself then feedback back into a vicious circle, as it can lead to a feeling that specialism is not rewarded and that researchers need to be “good at everything” (especially since producing a whole paper requires a lot of different skills, and collaboration is not well incentivised), which makes researchers feel an increased sense of pressure and competition as well as lack of professional satisfaction.These findings chime well with the different aspects of the system that Octopus is designed to improve or mitigate.Octopus is an alternative publication system specifically designed with the aim of removing many of the biases and the perverse incentives that affect the pathway from assessment to recognition.The table below shows the main issues discovered during this evaluation and how Octopus’ design should help improve them:<table><tbody><tr><td colspan="1" rowspan="1">Biases</td><td colspan="1" rowspan="1">Octopus solution</td></tr><tr><td colspan="1" rowspan="1">Researchers with prestigious credentials get preferential publication and assessment of their work</td><td colspan="1" rowspan="1">Researchers’ institutional affiliations and credentials (including geographic location) are not displayed on publications (this is stored as meta-data for reporting purposes only)</td></tr><tr><td colspan="1" rowspan="1">Research that produces “positive” findings is more likely to be published and assessed highly</td><td colspan="1" rowspan="1">All research can be published - there are no barriers to publication.&nbsp;Publications are assessed (post-publication) on their intrinsic qualities, rather than their impacts (i.e. what is important is how well the work was done, not what it ended up showing)</td></tr><tr><td colspan="1" rowspan="1">Researchers with certain personal characteristics (e.g. sex, ethnicity, age) are treated differently during the publication and assessment processes</td><td colspan="1" rowspan="1">Researchers’ surnames and initial only are shown on publications to reduce opportunities for unintentional bias&nbsp;(first names and more details are available if clicked-through to allow disambiguation and further background on a research to be available)</td></tr><tr><td colspan="1" rowspan="1">Researchers who are not native or as-native speakers of English find it harder to publish or are assessed less highly</td><td colspan="1" rowspan="1">Publications are welcomed in any language. Automatic language translation is anticipated as a feature in the near future to make the platform truly language-agnostic.</td></tr><tr><td colspan="1" rowspan="1">Incentives</td><td colspan="1" rowspan="1">Octopus solution</td></tr><tr><td colspan="1" rowspan="1">There is an incentive to “be first” and have “novel” findings (over doing careful and high quality research)</td><td colspan="1" rowspan="1">Publications are assessed only on intrinsic qualities (i.e. how well it was done)</td></tr><tr><td colspan="1" rowspan="1">There is an incentive to produce a lot of publications (over doing high quality research)</td><td colspan="1" rowspan="1">Publications are assessed only on intrinsic qualities (i.e. how well it was done)</td></tr><tr><td colspan="1" rowspan="1">There is an incentive to publish in “prestigious” outlets and work on “prestigious” subjects (over intrinsically high quality work)</td><td colspan="1" rowspan="1">Publications are assessed only on intrinsic qualities (i.e. how well it was done)</td></tr><tr><td colspan="1" rowspan="1">There is an incentive to “tell a good/impactful story” in a publication</td><td colspan="1" rowspan="1">Publications are intentionally non-narrative. Each is a small piece of work, judged entirely on its own merits.The publications are linked together to improve discoverability, collaboration and emergent findings</td></tr></tbody></table>By tackling these biases and improving the incentives, Octopus aims to ensure that the pathway from assessment to recognition is no longer skewed.The breaking down of the assessment units from a traditional, peer-reviewed journal paper into smaller, more specialised units, also tackles the perceived issue of lack of recognition for specialism. This should also remove the feedback loop identified which increases the sense of a competitive and unfulfilling professional environment (as well as improving meritocratic recognition overall).The barriers to sharing research that have been identified also need to be addressed, in order to unlock the full potential of Octopus. These include a lack of knowledge about how and where to share, and policies that actively (unintentionally) make sharing difficult or impossible.

Interpretation of interviews and survey on research culture.

Interpretation of survey and interviews: might Octopus help researchers produce high quality, open research?

Pen-Yuan Hsing has a PhD in biology, with highly multidisciplinary experience ranging from ecology and conservation, engineering, citizen science, to meta-research on open research best practices. Having developed and published relevant training material, online courses, books, and policy documents on the international level, Pen is a strong advocate for the idea that good research is open research. As someone who belongs to an ethnic minority group during their research career, Pen is particularly sensitive to issues of diversity related to geographical origin and language.

Alex Freeman has a DPhil in biology, which she followed with a career in factual television and the media. She has spent the last 6.5 years working in an interdisciplinary group in academia on evidence communication (funded by the David & Claudia Harding Foundation) and here came up with the concept of Octopus. She is the sole Director of Octopus CIC which is a UK-registered not-for-profit company, from which she derives no salary. She does unpaid work advocating for and developing Octopus in collaboration with Jisc, and is also a strong believer in Open Science practices and research transparency. Octopus is currently funded by Research England, and has previously had awards from Mozilla, the Royal Society and an anonymous philanthropist.

Jackie Thompson has a PhD in experimental psychology, followed by several years postdoctoral research experience in various sub-disciplines of psychology and meta-research. She has spent several years as an advocate for open research practices within psychology and academia more broadly, including working with the UK Reproducibility Network on several initiatives, mainly due to training researchers in open research practices. 


In order to complement in-depth, but necessarily small sample size, interviews, an online survey allowed us to reach a broader population to gain some quantitative data on the current research culture and barriers to best practice, as well as whether the Octopus platform might help overcome them.The finalised survey was implemented on <a target="_blank" rel="noopener noreferrer nofollow" href="https://ec.europa.eu/eusurvey/">EUSurvey</a>. It is a fully open source online survey platform developed and administered by the European Commission, adhering to relevant privacy regulations (e.g. the <a target="_blank" rel="noopener noreferrer nofollow" href="https://eur-lex.europa.eu/eli/reg/2016/679/oj">General Data Protection Regulation</a> (GDPR)).Below is the structure and content of this survey, with the questions and items for multiple choice questions. For multiple choice questions (except the one about research tasks), the order of answers are randomised for each respondent.<ul><li>Participant information and consent</li><li>Information about you<ul><li>In which field do you primarily conduct your research?<ul><li>Agriculture</li><li>Anthropology</li><li>Archaeology</li><li>Astronomy</li><li>Biology</li><li>Business</li><li>Chemistry</li><li>Computer science</li><li>Data science</li><li>Earth science</li><li>Economics</li><li>Education</li><li>Engineering</li><li>Environmental science</li><li>Finance</li><li>Galleries, libraries, archives, and museums (GLAM)</li><li>Geography</li><li>Mathematics</li><li>Medicine and health</li><li>Physics</li><li>Political science</li><li>Psychology</li><li>Public policy</li><li>Sociology</li><li>Statistics</li><li>Other (please specify)</li></ul></li><li>How long would you say you have been conducting research in this field?<ul><li>Less than 1 year</li><li>1-3 years</li><li>3-5 years</li><li>5-10 years</li><li>More than 10 years</li></ul></li><li>Where in the world are you primarily based?<ul><li>Africa</li><li>Antarctica</li><li>Asia</li><li>Australia</li><li>Europe</li><li>North America</li><li>Oceania</li><li>South America</li></ul></li></ul></li><li>Feelings about research culture<ul><li>At what levels do you feel the following aspects are present in your current research job?<ul><li>Support from colleagues you work closely with</li><li>Support from the wider research community in your field</li><li>The worry about securing your next job or career advancement</li><li>Professional satisfaction you get from your research</li><li>Impact that your individual work has on your research field as a whole</li><li>Collaboration (e.g. between different groups) to tackle research challenges in your field</li><li>Confidence that you and your research are being judged fairly on the true quality of what you produce</li></ul></li></ul></li><li>Causes of publication bias<ul><li>Have you ever conducted any piece(s) of research in the past, large or small, but chose to NOT publish or share it/them publicly?<ul><li>Yes</li><li>No</li></ul></li><li>(for those who answered yes) Which of the following factors prevented you from sharing or publishing this research?<ul><li>It did not conform to established thinking (e.g. a "desired" result, etc.)</li><li>The findings did not make a clean “story”</li><li>I did not have confidence in the analysis/interpretation</li><li>It did not fit in a traditional peer-reviewed paper</li><li>There was not enough data</li><li>It did not have sufficient "impact"</li><li>It was not a "positive" or “desired” result</li><li>It would not help my career</li><li>It was rejected by peer reviewers and I did not/could not do the requested revisions</li><li>I did not have time/resources to write it up</li><li>I could not find an outlet that would publish work on this topic</li><li>I cannot think of an instance where I’ve done something I did not share publicly</li><li>Other (please specify)</li></ul></li></ul></li><li>Causes preventing sharing research faster and earlier<ul><li>Imagine you have a good research question and a plan to investigate it. Which of the following factors may prevent you from publishing or otherwise sharing your research question and plan at this stage (i.e. before there is any evidence to inform it)?<ul><li>I wouldn't know where to publish or share this type of output</li><li>Fear of being "scooped" or plagiarised because I would want to collect the evidence myself and someone else might do so and then claim credit for the idea</li><li>There is no benefit to me and my career to publish something like this</li><li>This is not the way things are done in my field</li><li>I wouldn't expect useful feedback from readers or reviewers at this stage</li><li>It would not have enough "impact", so is not worth sharing</li><li>When evidence is collected it might not support the theory/hypothesis and I would have been shown to be ‘wrong’</li><li>I wouldn’t know to write up a publication like this</li><li>Other (please specify)</li></ul></li></ul></li><li>Research specialisation<ul><li>The following is a list of potential tasks involved in the research lifecycle. Think about them for a typical project you’ve been involved with. For each task, think about whether you typically: do it mostly on your own; do it mostly as a team; don’t do it because someone else does it; or it is not applicable.<ul><li>Defining the research problem or question</li><li>Finding out what is already known about the problem (e.g. reviewing literature)</li><li>Developing the theoretical rationale or hypothesis that would help address the research problem</li><li>Developing the methodology</li><li>Collecting data (qualitative)</li><li>Collecting data (quantitative)</li><li>Curating and cleaning data</li><li>Analysing results/data</li><li>Interpreting the analysis</li><li>Applying the results in a different context/implementing the findings</li></ul></li></ul></li><li>Focus of research assessments<ul><li>When your research outputs (i.e. your whole body of work) are being assessed as part of consideration for funding or career advancement, how much do you think the following factors influence that assessment?<ul><li>Having "positive" instead of "negative" or "null" results</li><li>The "impact" of your research on the real world</li><li>The prestige of your institution</li><li>Your publication record (e.g. number of publications, citation metrics, etc.)</li><li>Personal connections that you have (e.g. knowing and collaborating with highly-regarded researchers)</li><li>Your personal characteristics (e.g. gender, ethnicity, language fluency, location, etc.)</li><li>The trendiness of your research topic(s)</li><li>The rigour of your research method(s)</li><li>How much you share your data, code, or other work publicly (i.e. open research practices)</li><li>Having research that confirms existing ideas rather than going against them or proposing new ones</li></ul></li></ul></li><li>Platforms for sharing research outputs<ul><li>Which of the following platforms are you aware of for publishing parts or all of your research?<ul><li>Octopus.ac</li><li>FigShare</li><li>Video hosting platforms (e.g. YouTube, PeerTube, Vimeo, etc.)</li><li>Wikipedia</li><li>Institutional repositories</li><li>Specialist data repositories (e.g. ProteomeXchange, OpenNeuro, PubChem, PANGAEA, Qualitative Data Repository, etc.)</li><li>Peer Community In (PCI)</li><li>Multimedia repositories (e.g. Flickr, Internet Archive, Wikimedia Commons, etc.)</li><li>Zenodo</li><li>Registered reports (i.e. journal articles with pre-study peer review)</li><li>ResearchEquals</li><li>Github, Gitlab, Wikifactory, or similar service</li><li>Dryad</li><li>Protocols.io</li><li>Open Science Framework (OSF)</li><li>Research Ideas and Outcomes (RIO)</li></ul></li></ul></li><li>Opt-in to acknowledgements and prize draw<ul><li>Please indicate if you would like to be acknowledged by name in our report. This is strictly optional and your name will not be associated with your answers to this survey in any published outputs from our study. The name you provide here will be listed in the acknowledgements section under survey respondents.<ul><li>Yes, I would like to be acknowledged by name</li><li>No, I do not want to be acknowledged by name</li></ul></li><li>If you answered yes, please provide the name with which you want to be acknowledged<ul><li>[text box for respondent’s name]</li></ul></li><li>Please let us know if you would like to enter a drawing for a prize worth GBP 25 as a token of our deep appreciation for your valuable time. This is strictly optional and your information will not be associated with your answers to this survey in any published outputs from our study.<ul><li>Yes, I would like to enter the prize draw</li><li>No, I do not want to enter the prize draw</li></ul></li><li>If you answered yes, please provide a reliable email address for contact<ul><li>[text box for email]</li></ul></li></ul></li></ul>

Survey to assess researchers' perception of the current research culture and barriers to open, high quality research

Survey to help identify whether Octopus might help researchers produce high quality, open research

Pen-Yuan Hsing has a PhD in biology, with highly multidisciplinary experience ranging from ecology and conservation, engineering, citizen science, to meta-research on open research best practices. Having developed and published relevant training material, online courses, books, and policy documents on the international level, Pen is a strong advocate for the idea that good research is open research. As someone who belongs to an ethnic minority group during their research career, Pen is particularly sensitive to issues of diversity related to geographical origin and language.

Alex Freeman has a DPhil in biology, which she followed with a career in factual television and the media. She has spent the last 6.5 years working in an interdisciplinary group in academia on evidence communication (funded by the David & Claudia Harding Foundation) and here came up with the concept of Octopus. She is the sole Director of Octopus CIC which is a UK-registered not-for-profit company, from which she derives no salary. She does unpaid work advocating for and developing Octopus in collaboration with Jisc, and is also a strong believer in Open Science practices and research transparency. Octopus is currently funded by Research England, and has previously had awards from Mozilla, the Royal Society and an anonymous philanthropist.

Jackie Thompson has a PhD in experimental psychology, followed by several years postdoctoral research experience in various sub-disciplines of psychology and meta-research. She has spent several years as an advocate for open research practices within psychology and academia more broadly, including working with the UK Reproducibility Network on several initiatives, mainly due to training researchers in open research practices.



In order to assess whether or not Octopus’s aims were in alignment with researchers’ perceived difficulties, we conducted a series of semi-structured qualitative interviews about various aspects of research culture. It was done with researchers in a range of research disciplines and institutions across the natural and social sciences. Participant recruitment was done through the global, professional connections of the authors of this study and their contacts. Interviewees were offered a small incentive for their participation worth GBP 25. We aimed to secure interviews with researchers across: <ul><li>Career stages </li><li>Academic and non-academic contexts, such as industry or non-profit organisations </li><li>Locations, including outside of the Global North </li><li>Natural and social sciences </li><li>Qualitative and quantitative research </li></ul>As this study includes no inferential statistics, our sample sizes for the interviews were based on reaching enough representatives of various criteria (demographics, research disciplines, etc.) and on logistical considerations of time. We scheduled 60-minute online calls with the participants, which were recorded and transcribed. The recordings were deleted once the transcriptions were complete with personally-identifying information removed. The interviews were structured with questions on the following topics on research culture: <ul><li>Division of labour </li><li>Credit/attribution </li><li>Critique </li><li>Assessments </li><li>Attitudes on open research practices (without referring to the Octopus platform) </li></ul>The interviews were conducted using the following interview guide: <table><tbody><tr><td colspan="1" rowspan="1">Introduction</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">Question</td><td colspan="1" rowspan="1">Prompts</td></tr><tr><td colspan="1" rowspan="1">Housekeeping</td><td colspan="1" rowspan="1">Welcome participant and thank for their timeSound check (no need to do explicitly)Explain how long interview will take and that they can stop/take a break at any timeEstablish procedure in case of connectivity issues (i.e. try to rejoin ASAP and resume interview when ready)Any questions?</td></tr><tr><td colspan="1" rowspan="1">Ask consent to record</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">**start recording**</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">Introduce the aims of the study</td><td colspan="1" rowspan="1">to find out more about participant’s views and experiences of:&nbsp;a. research environment and culture - esp related to publishing/sharingb. contributions to research projects – who does what, how are different parts valued and creditedc. how publication formats affect or interact with various aspects of research process</td></tr><tr><td colspan="1" rowspan="1">Part 1: Distribution of labour</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">To start off with, could you tell me a bit about the nature of your work?</td><td colspan="1" rowspan="1">What is your field, and your specific research focus?Other demographics - field, career stage, and geography“what kinds of data do you work with?” “modalities of your work, lab based, field based, existing datasets, etc.?”Try to keep it under 2 minutes - don’t let them get carried away!</td></tr><tr><td colspan="1" rowspan="1">Think about a typical project in your field. I’m interested to know about the division of labour.Let’s start by first thinking about the different components, tasks, and skills involved in a project? Who does these?What does the distribution of tasks or contributions look like for a research project in your field (your lab) in terms of who does what, and how responsibilities are shared?</td><td colspan="1" rowspan="1"> How many people might work on a project? How would they split the roles?What parts (if any) of the research process tend to be done by a specialist?<ol><li>Problem</li><li>Hypothesis / theoretical rationale</li><li>Protocol / method</li><li>Data / results</li><li>Analysis</li><li>Interpretation</li><li>Applications / impact</li></ol>(if needed) How does your experience compare to others in your field?</td></tr><tr><td colspan="1" rowspan="1">For the next question I want you to consider this picture you’ve sketched out for me in terms of the division of labour - what parts of the research tend to be done by specialists or generalists. With that in mind, how do you feel about this division of labour?</td><td colspan="1" rowspan="1">Does it work well or poorly? In what ways?What, if anything, would you change about it?</td></tr><tr><td colspan="1" rowspan="1">What parts of the research process do you think are most valued, in your field?</td><td colspan="1" rowspan="1">By ‘valued’, I mean considered by other people, organisations and systems to be important and worthy of time, effort, attention, prestige.How can you tell these parts are valued?How are the different parts perceived by outside systems? Open-ended first, allow ppt to come up with outside systems - then prompt for below if needed:<ol><li>Research assessment systems (REF, promotion, funder-specific)</li><li>The current publishing system</li></ol></td></tr><tr><td colspan="1" rowspan="1">We’re also interested to hear what YOU value within the research process. What parts do you think are most (or least) important to get right (in order to create ‘good’ research)?</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">Part 2: Sharing</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">Thinking about the various parts of the research process --&nbsp; I’m interested to know more about how, and when, and whether information about these various parts of the research process are shared, beyond the individual person or team working on it. I’m thinking of the term sharing very broadly, it doesn’t have to be something formal.&nbsp;What parts of the research process do researchers tend to share beyond their own research team?</td><td colspan="1" rowspan="1">Recap the parts ppt mentioned, or Octopus list if needed Careful with phrasing - don’t want to sound judgemental that people SHOULD be sharing.Are papers the main way people share various aspects of the research process?</td></tr><tr><td colspan="1" rowspan="1">How are these parts each shared?</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">With whom? (How widely?)</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">Why do you think people in your field share those things?</td><td colspan="1" rowspan="1">(what’s “right” vs “norm/required”)<ol><li>Dissemination - share knowledge</li><li>Credit - give creators credit/attribution</li><li>Transparency - gain credibility, show you have nothing to hide</li><li>Feedback - get input or insight from others</li><li>Storage&nbsp;</li></ol></td></tr><tr><td colspan="1" rowspan="1">When? (and why then?)</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">Which parts are most valuable to get feedback (constructive critique, input, scrutiny) on?</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">How well do you think this current process of sharing works?</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">Part 3: Critique</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">I’d like to talk about the culture surrounding critique of others’ work. In your experience, what are situations where you feel you could or couldn’t critique others’ work? What is this like for other researchers?</td><td colspan="1" rowspan="1">This could be, but is not limited to, peer review of papers.</td></tr><tr><td colspan="1" rowspan="1">Are there aspects of research work that you think are acceptable or not acceptable to critique? Why is this?</td><td colspan="1" rowspan="1">e.g. the method versus the data collection versus the research problem versus the analysis or interpretation versus the writing style versus the detail shared? (Does it feel more constructive to critique a proposed method than a 'finished paper', for example? Or to review and comment on a&nbsp; dataset, knowing that the authors can act constructively on that review?)</td></tr><tr><td colspan="1" rowspan="1">Part 4: Open research (Octopus) model</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">Imagine working in a very open way - making each part of your research work open/shared - to anyone in the world, at any point in the future. How do you feel about this?</td><td colspan="1" rowspan="1">What parts of that might you object to?How do you think other researchers would feel about doing this?</td></tr><tr><td colspan="1" rowspan="1">Part 5: Credit</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">In terms of giving people credit for the various contributions to the project, how does that tend to go?</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">How do you feel about the current state of it?</td><td colspan="1" rowspan="1">Probe how much people care about giving credit for work</td></tr><tr><td colspan="1" rowspan="1">Why?</td><td colspan="1" rowspan="1">Encourage them to be honest - how do you FEEL about it?</td></tr><tr><td colspan="1" rowspan="1">Part 6: Research Assessment</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">I would like to ask a bit about research assessment now - both formally, as in hiring and promotion, prizes/honours, the REF, assessment by funders, etc., but also informally (others judging quality of research in their own opinions).In an ideal world, if someone is trying to assess the quality of a piece of research in your field, how would they go about doing that? What aspects would they need to consider/need to know?</td><td colspan="1" rowspan="1">Try to get them to think NOT in terms of a paper/narrative. In an ideal world, how would one assess the quality of research? What aspects about the research are most important to consider? I.e., How could someone tell whether the work you did in your day job, on these various parts of the research, is ‘good’?</td></tr><tr><td colspan="1" rowspan="1">Now, could I have you think about the current state of how research actually is assessed?&nbsp;What have your experiences with research assessment been like?</td><td colspan="1" rowspan="1"> As in, when your own research has been assessed, or when you’ve assessed others.Both formally and informally - and not just in terms of papers!</td></tr><tr><td colspan="1" rowspan="1">What are your views on the current state of assessing research quality? What is good or bad about it?</td><td colspan="1" rowspan="1">Do any parts of the research process get more focus than others during assessments? Which ones? Why is that? (Is that good?)</td></tr><tr><td colspan="1" rowspan="1">Have you encountered (or seen) any biases in assessing research quality? What kinds, if so?&nbsp;</td><td colspan="1" rowspan="1">Does anyone get more or less recognition than they deserve?Prompt demographics, roles, fields, location, etc. if needed. Remember this is about THEIR experiences/views, not generalities.</td></tr><tr><td colspan="1" rowspan="1">Part 7: Participant’s role</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">What are the things you feel are necessary for career progression (in terms of research)?&nbsp;How do they match up with the things you feel are your strengths, or the things you want to be doing, as a researcher?</td><td colspan="1" rowspan="1">Focusing on the disparity between what you feel you COULD be bringing to science with your skills etc., and what you feel you have to do to progress.Sometimes other things come up like teaching, etc., if so and if running out of time, could refocus them on research aspects.</td></tr><tr><td colspan="1" rowspan="1">Closing</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">Is there anything that we haven’t discussed that you’d like to talk about before the end of the interview?</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1">**stop recording**</td><td colspan="1" rowspan="1"></td></tr><tr><td colspan="1" rowspan="1"></td><td colspan="1" rowspan="1">Thank the participant for their time.Explain next steps (debrief, final consent etc)Explain data management procedures (i.e. when will interview recordings be deleted, how will transcripts be anonymised etc)Any questions?Contact details for questions/concerns</td></tr></tbody></table>

Semi-structured interview plan for assessing researchers' feelings about the research culture and perception of barriers to open, high quality research.

Qualitative interviews to identify whether Octopus might help researchers produce high quality, open research

Pen-Yuan Hsing has a PhD in biology, with highly multidisciplinary experience ranging from ecology and conservation, engineering, citizen science, to meta-research on open research best practices. Having developed and published relevant training material, online courses, books, and policy documents on the international level, Pen is a strong advocate for the idea that good research is open research. As someone who belongs to an ethnic minority group during their research career, Pen is particularly sensitive to issues of diversity related to geographical origin and language.

Alex Freeman has a DPhil in biology, which she followed with a career in factual television and the media. She has spent the last 6.5 years working in an interdisciplinary group in academia on evidence communication (funded by the David & Claudia Harding Foundation) and here came up with the concept of Octopus. She is the sole Director of Octopus CIC which is a UK-registered not-for-profit company, from which she derives no salary. She does unpaid work advocating for and developing Octopus in collaboration with Jisc, and is also a strong believer in Open Science practices and research transparency. Octopus is currently funded by Research England, and has previously had awards from Mozilla, the Royal Society and an anonymous philanthropist.

Mariia Tukanova has a BSc in sociology and social policy with experience in research on social policy and participating in international student exchange programs. Mariia brings a perspective to this evaluation from an earlier stage in their research career than the other authors.

Jackie Thompson has a PhD in experimental psychology, followed by several years postdoctoral research experience in various sub-disciplines of psychology and meta-research. She has spent several years as an advocate for open research practices within psychology and academia more broadly, including working with the UK Reproducibility Network on several initiatives, mainly due to training researchers in open research practices. 

Marcus Munafò has a PhD in health psychology, and has worked across a range of disciplines in the biomedical sciences (public health, primary care, clinical pharmacology, psychiatry, epidemiology). He is a proponent of open research and scholarship, and co-founded the UK Reproducibility Network, which receives funding from several major funders, including a Research England Development Fund award to promote open research practices. He is also co-director of the Tobacco and Alcohol Research Group (TARG), which is based within the School of Psychological Science at the University of Bristol, and a Programme Lead within the MRC Integrative Epidemiology Unit at the University of Bristol.

Octopus (<a target="_blank" rel="noopener noreferrer nofollow" href="https://octopus.ac/">octopus.ac</a>) is a new research publishing platform with the goal of reforming how research is conducted, shared, and reviewed. It is different from other open research platforms by breaking down the reporting of research into eight distinct elements:<ul><li>Research problem</li><li>Rationale/Hypothesis</li><li>Methods</li><li>Results</li><li>Analysis</li><li>Interpretation</li><li>Real world application</li><li>Peer review of any of the above</li></ul>&nbsp;The platform encourages the publication of research in smaller increments, with a lower barrier to entry compared to the traditional journal article format. In addition to being free-of-charge for authors, Octopus is distinct from other platforms because it requires published items to be linked to one another. For example, a method proposed by one researcher could be linked to a hypothesis (or multiple hypotheses) published by others.&nbsp;Ultimately, Octopus aims to reform research culture and foster better research quality in five ways:<ol><li>Reducing barriers and hierarchies that restrict research sharing, leading to more meritocratic recognition (especially for early career researchers and specialists)</li><li>Reducing pressures that can lead to questionable research practices (QRPs)</li><li>Addressing factors that lead to certain topics and types of research being favoured</li><li>Reducing bias in the assessment of publications and their quality</li><li>Encouraging more openly collaborative ways of thinking and working – division of labour and open critique of all parts of the research process</li></ol>&nbsp;In order to know to what extent Octopus, or any other platforms or policies, can help achieve these aims, it’s important to understand the perceived barriers from the perspective of researchers from different disciplines, career stages and types of institution to see whether the design of Octopus is indeed likely to achieve its aims, and also what else could be done alongside Octopus to help.&nbsp;

The theory of how Octopus aims to help encourage open and high quality research.

Could Octopus help researchers produce high quality, open research?

Pen-Yuan Hsing has a PhD in biology, with highly multidisciplinary experience ranging from ecology and conservation, engineering, citizen science, to meta-research on open research best practices. Having developed and published relevant training material, online courses, books, and policy documents on the international level, Pen is a strong advocate for the idea that good research is open research. As someone who belongs to an ethnic minority group during their research career, Pen is particularly sensitive to issues of diversity related to geographical origin and language.

Mariia Tukanova has a BSc in sociology and social policy with experience in research on social policy and participating in international student exchange programs. Mariia brings a perspective to this evaluation from an earlier stage in their research career than the other authors.

Alex Freeman has a DPhil in biology, which she followed with a career in factual television and the media. She has spent the last 6.5 years working in an interdisciplinary group in academia on evidence communication (funded by the David & Claudia Harding Foundation) and here came up with the concept of Octopus. She is the sole Director of Octopus CIC which is a UK-registered not-for-prot company, from which she derives no salary. She does unpaid work advocating for and developing Octopus in collaboration with Jisc, and is also a strong believer in Open Science practices and research transparency. Octopus is currently funded by Research England, and has previously had awards from Mozilla, the Royal Society and an anonymous philanthropist.

Tim Fellows has a BSc in Business Economics, with experience as a product manager on various research management platforms. He currently works for Jisc as the product manager for Octopus, and is responsible for overseeing the growth and development of the platform alongside Alex Freeman.

Marcus Munafò has a PhD in health psychology, and has worked across a range of disciplines in the biomedical sciences (public health, primary care, clinical pharmacology, psychiatry, epidemiology). He is a proponent of open research and scholarship, and co-founded the UK Reproducibility Network, which receives funding from several major funders, including a Research England Development Fund award to promote open research practices. He is also co-director of the Tobacco and Alcohol Research Group (TARG), which is based within the School of Psychological Science at the University of Bristol, and a Programme Lead within the MRC Integrative Epidemiology Unit at the University of Bristol.

Jackie Thompson has a PhD in experimental psychology, followed by several years postdoctoral research experience in various sub-disciplines of psychology and meta-research. She has spent several years as an advocate for open research practices within psychology and academia more broadly, including working with the UK Reproducibility Network on several initiatives, mainly due to training researchers in open research practices.

In addition to the known barriers to sharing research work openly (see <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.57874/7d3t-kv62">https://doi.org/10.57874/7d3t-kv62)</a>, there is evidence for pressures that affect the quality of the research too, such as questionable research practices (QRPs), biases in research selection and assessment, and anti-collaborative practices.<h3>Factors leading to QRPs</h3>Quantity oriented environment and pressure to publishOne of the biggest underlying factors related to Questionable Research Practices is the so-called ‘publish or perish’ culture. A survey by Schoot et. al. (2021) emphasised that the modern research culture is quantity rather than quality oriented - a point supported by arguments from several previous reviews (Ball, 2016; Ding et al., 2020; Nosek et al., 2012; Schoot et al., 2021; Vuong, 2019; Werner, 2021). When academics are rewarded for the number of publications it “will pressure all but the most ethical scientists, to overemphasise quantity at the expense of quality, create pressures to “cut corners” throughout the system, and select for scientists attracted to perverse incentives” (Edwards and Roy, 2017). The pressure to publish high quantities leads not only to a decrease in the quality of publications but also to the dubious practice of so-called “salami-slicing”: dividing results from one study into many fragments and using them for different purposes as separate publications.Pressure for positive findingsAnother factor leading to QRPs is pressure to get positive results or novel findings. Several surveys (Fong and Wilhite, 2017; Fraser et al., 2018; José Perezgonzalez et al., 2021; Moran et al., 2022; Wolff et al., 2018) have found that research culture is oriented towards ‘novel’ findings, thereby encouraging researchers to QRPs to obtain them. Reproducibility is valued less than novelty, and those who have “clean” data and “significant” results win the race for recognition. This conclusion has also been supported by reviews (Bergkvist, 2020; Open Science Collaboration, 2015), meta-studies (Fidler et al., 2017; Nissen et al., 2016), qualitative analysis and analysis of secondary data (Baldwin et al., 2022; Gibelman and Gelman, 2005) and other sources (Laitin et al., 2021; Verma and Detsky, 2020). It is inextricably related to pressure to publish, since “statistically significant findings that are visually and numerically clean are easier to publish” (Diong et al., 2018). Such conditions force researchers to fight for publication and recognition and incline them towards misusing data analysis (p-hacking) and selectively publishing results.Dependence on fundingA systematic review and meta-analysis by Fanelli (2009) found that 33% of respondents admitted ‘changing the design, methodology or results of a study in response to pressures from “a funding source’’. Other studies have noted pressure from grant schemes (Edwards and Roy, 2017; Fanelli et al., 2015; Huistra and Paul, 2022) and demonstrated imperfections of the science system. According to some interviews conducted with scientists, competition for money and dependence on grant funding negatively affects research integrity. The expected results of grant schemes are the funding of research programs and promoting growth, however the actual results are lack of time for gathering and thinking about data and focusing on getting the positive results. (Edwards and Roy, 2017).Hierarchical pressure from superiors/competitive environmentAnother factor that can affect research integrity is hierarchical or environmental pressure. Some meta-analyses (Fanelli et al., 2015) and reviews (Rupp et al., 2019; Sharma and Verma, 2018) conclude that it strongly affects young researchers, who are trying to build a reputation and might be subject to criticism from colleagues. According to a mixed-method study by Gerrits et al., (2020) and some surveys, competitiveness and hierarchical pressure had a negative impact on research integrity (Gopalakrishna et al., 2022; Metcalfe et al., 2020). Competitive environments make scientists focus on the speed and statistical significance of their research in order to build their CVs, get funding or gain promotion. This affects research integrity and the quality of findings.Lack of clear policiesSeveral reviews and commentaries (Bouter, 2015; Ding et al., 2020; Kiri et al., 2018) mention that scientists committed misconduct for two reasons. The first is confusion in definitions of “good” and “bad” practices. The second is existing gaps in policies and regulations that allow the use of QRPs with low risk of consequences.<h3>Factors leading to publication bias</h3>‘Positive’ versus ‘negative’ findingsMultiple studies have demonstrated that ‘positive’ research findings are favoured in the literature, with statistically nonsignificant results being less likely to be published (Fidler et al., 2017; Fong and Wilhite, 2017; Vuong, 2019). Several studies have also found that current research culture undervalues null or negative findings, which in turn causes researchers to make changes in statistics (for example, p-hacking, data dredging, selective reporting) and present statistically significant or positive results in order to get their work published (Brembs, 2018; Carbine et al., 2019; Nissen et al., 2016; Smaldino et al., 2019; Schweitzer and Schulz, 2018; Verma and Detsky, 2020: Stanley et al., 2022). A meta-research study by Duyx (2017) found that papers with ‘positive’ findings are cited twice as often as ‘negative’ ones. Pressure to publish and the desire and incentive to be cited can lead to decrease of quality and reliability of studies.The factors which make it difficult to publish null or negative results are the same as those that lead to QRPs. Firstly, null findings are seen to be less valuable for publishing by both journals and scientists (Nissen et al., 2016; Ioannidis et al., 2014). Under the pressure to publish, authors may “cherry pick” the information and publish only positive results (Schweitzer and Schulz, 2018). According to Nissen et al. (2016), publication bias is so strong in the current research climate that a significant part of scientific literature does not present negative results at all. This encourages a self-fulfilling cycle. For example, several reviews (Brembs, 2018; Vuong, 2019) indicated pressure to publish and the trend of “not to publish negative results” as motivating factors for scientists to selectively publish positive findings. This, in turn, leads to biased knowledge (Schweitzer and Schulz, 2018).Research in certain journalsA quantitative research study which analysed poorly and well -cited articles in orthopaedic journals (Kortlever et al., 2019) concluded that there is no difference between the proportion of poorly cited articles in subscription-based and open access (OA) journals. This study found that 36% of the total analysed articles were defined as ‘poorly cited’ five years after publication. Unfortunately, this study did not provide information on factors that led to this situation, but it suggested there is more polarisation in what research receives attention. Other research has shown that the platform of publishing impacts the number of citations, with articles from better-known platforms more likely to be cited. For example, according to (Wakeling et al., 2016), scientists are more likely to cite articles from PLOS ONE and Scientific Reports than other less well-known mega-journals.Field of studyQuantitative bibliometric analysis by Larivière et al., (2015) has shown that there is a difference between publishing papers in natural and medical sciences (NMS) and social sciences and humanities (SSH). The results of the analysis revealed that during the transition to the digital environment, social science communities began to publish their work in giant commercial publishers (Reed-Elsevier, Wiley-Blackwell, Springer-Nature, and Taylor &amp; Francis), while representatives of the medical sciences preferred to stay more independent and publish in smaller publishers. This happened because the social science communities were more dispersed and were likely to have fewer resources to adapt to the digital age. Therefore, for instance, social scientists were more likely to have agreements with commercial publishers. Consequently, 70% of papers from the top five publishers belong to the social sciences field.Another problem that has been noted in literature is related to the funding of different types of research. Some studies mentioned a problem of unfair funding stratification among different types of research, for instance, education research and implementation research are underfunded (Duyx et al., 2017; Yarris et al., 2014). According to Yarris and colleagues (2014), there are a few available grants that are not enough for covering research projects’ needs.<h3>Factors leading to bias in the assessment of research quality</h3>Demographic biasesGender bias in research assessment has been identified by many studies. Cruz- Castro and Sanz-Menendez (2021), Fox et al. (2015), Johnson et al. (2021), Morales et al. (2021), and Silberzahn et al. (2018) demonstrated that bias against female researchers exists in various disciplines. This finding also appeared in reviews and included such disciplines as medical science (Toews et al., 2017; Upthegrove et al., 2021), economics (Rousseau, 2021), ecology (Eisen et al., 2013; Sing et al., 2017), chemical sciences (Bennie and Koka, 2021), and biology (Resnik et al., 2008). The bias against women manifests in multiple ways. Firstly, an analysis of conference abstracts and whether they were eventually published in a journal found that the “last author's female gender was predictive of a lower likelihood of publication” (Johnson et al., 2021). Previous studies have mentioned that the assessment process for journal publications might be affected by subjectivity of editors. Lack of gender diversity of editorial boards leads to disparity of published papers. In journals, where editors were male, there was less proportion of female reviewers (Fox et al., 2015). Secondly, there is an imbalance in the review process which appears because of the tendency of editors to invite reviewers “like themselves”. Some reviews (Detweiler et al., 2016; Eisen et al., 2013; Upthegrove et al., 2021) and a survey (Morales et al., 2021) found that women are less likely to receive invitations for peer review and also that representation of women in positions of senior author is less than that of men. A survey by Gunthe and Gettu (2022) showed that the output and quality of research publications by some academics (especially women and early career researchers (ECRs) who moved between institutions, changed career paths or had a pause in research activities) are not fairly assessed.Geographical and language biasThese biases affect researchers from low-middle income countries and those whose first language is not English. The current research culture prioritises English because all high-ranking journals are in English (Baltazar et al., 2019; Naik, 2017) and publication in English increases the likelihood of citation (Vinkenburg et al., 2021). Therefore non-English speaking researchers experience problems with readership of their works (Lawrence, 2007). The prioritisation of the English language in the academic literature creates a disadvantage for non-native English speaking authors and forces them to spend extra time and resources getting their work published (Hagan et al., 2020).Another factor which may result in bias against researchers is under-representation of reviewers from low-middle income countries (LMIC). Reviewers mostly come from high-income countries, as academics from low-income countries do not have time and resources to do additional jobs. “Having reviewers mainly from high- income countries means that the interest of these scientists and populations are perpetuated, and those in low-resource settings are marginalised” (Cheah and Piasecki, 2022). Same tendency appeared in a survey by Publons (2018) that has evidence that reviewers from low-middle income countries are not invited to review academic papers. This means that researchers from low-middle income countries are not equally included in the evaluation process.Bias for author and institutional prestigeManuscripts are frequently assessed for publication based on status of authors or institutions or other subjective factors (Detweiler et al., 2016; Cazap et al., 2020; Eyre-Walker and Stoletzki, 2013), which makes reviews biassed (Nestor et al., 2020). The fact that reviewing and decision-making often does not happen openly (Bonn and Bouter, 2021; Siler et al., 2015; Wicherts et al., 2012) enables assessors, even inadvertently, to prioritise manuscripts based on their personal biases. For instance, assessors may favour articles which cite famous authors (Urlings et al., 2021; Gøtzsche, 2022), show “significant” findings (Cazap et al., 2020; Ekmekci, 2017; Jannot et al., 2013), or whose authors are based at institutions with more prestigious reputations.<h3>Anti-collaborative practices</h3>Division of labourWork has been done on how best to foster collaborative research practices. Some authors distinguish different types of contributions during the research process. For example, a mixed-method study evaluating a technological collaboration tool (Julpisit and Esichaikul, 2019) analysed knowledge sharing practices of research teams and concluded that collaborative activities could be categorised into four types: identifying research goals, designing tasks, performing tasks, and writing reports. "A survey of research teams across a range of scientific disciplines (Lee et al., 2015) found that while the impact of research increased with team size, the novelty of research was boosted by a variety of team members with distinct knowledge bases". In relation to kinds of activities, some studies mention the importance of “collaborative supportiveness” (Liu et al., 2013; Woodzicka et al., 2015). It may increase productivity by providing a broader understanding of the research process, and improve research culture by supporting individual contributions to team activities. This encourages more collaborative ways of thinking among researchers and highlights research contributions of individual researchers. Other studies note that it is important to value all types of contributions as this affects the productivity of research collaboration and the research culture itself (Lariviere et al., 2021; Mauthner and Doucet, 2008; Wolfe and Alexander, 2005). Despite the fact that many studies have a similar structure (conceptualisation, operationalisation and written communication), other, more niche and narrow tasks cannot be ignored. When one type of work is perceived as more worthy than another, it leads to inequalities across disciplines and teams. Another study by Haeussler and Sauermann (2020) found that interdisciplinary teams have greater division of labour.Generalist vs specialist rolesResearchers’ ways of working have been categorised into several types. Studies, both qualitative and quantitative, describe potential roles for researchers as: generalists (who are team players), specialists (who work alone) and versatiles (who do both). According to Lu et al. (2020), which used more than 100,000 articles from PLOS and extracted author contribution statements, generalists are the majority. (Note, however, that this method cannot separate whether most authors are actually generalists, or whether contribution statements might be inflated due to pressure to appear as generalists.) A qualitative study (Haeussler and Sauermann, 2020) analysed pre-defined contribution statements from PLOS and found that roughly 22% of authors perform 20% or less of all contributions (“specialists”), while 29% perform more than 60% of all contributions (“generalists”). As for the authorship of articles, versatiles are more likely to be first authors (Lu et al., 2022), confirming past studies (e.g. Lu et al., 2020) who found that versatiles are most often senior authors and are associated with funding and supervision.All of these potential barriers come together to make it difficult for researchers wanting to carry out best research practice, and make their work freely available for others to build on, feel that they can do this and succeed in an academic career.

A summary of the known issues with incentives in scholarly publishing.

How can we overcome the barriers to researchers producing high quality, open research?

Pen-Yuan Hsing has a PhD in biology, with highly multidisciplinary experience ranging from ecology and conservation, engineering, citizen science, to meta-research on open research best practices. Having developed and published relevant training material, online courses, books, and policy documents on the international level, Pen is a strong advocate for the idea that good research is open research. As someone who belongs to an ethnic minority group during their research career, Pen is particularly sensitive to issues of diversity related to geographical origin and language.


Mariia Tukanova has a BSc in sociology and social policy with experience in research on social policy and participating in international student exchange programs. Mariia brings a perspective to this evaluation from an earlier stage in their research career than the other authors.



Alex Freeman has a DPhil in biology, which she followed with a career in factual television and the media. She has spent the last 6.5 years working in an interdisciplinary group in academia on evidence communication (funded by the David & Claudia Harding Foundation) and here came up with the concept of Octopus. She is the sole Director of Octopus CIC which is a UK-registered not-for-profit company, from which she derives no salary. She does unpaid work advocating for and developing Octopus in collaboration with Jisc, and is also a strong believer in Open Science practices and research transparency. Octopus is currently funded by Research England, and has previously had awards from Mozilla, the Royal Society and an anonymous philanthropist.



Tim Fellows has a BSc in Business Economics, with experience as a product manager on various research management platforms. He currently works for Jisc as the product manager for Octopus, and is responsible for overseeing the growth and development of the platform alongside Alex Freeman.


Marcus Munafò has a PhD in health psychology, and has worked across a range of disciplines in the biomedical sciences (public health, primary care, clinical pharmacology, psychiatry, epidemiology). He is a proponent of open research and scholarship, and co-founded the UK Reproducibility Network, which receives funding from several major funders, including a Research England Development Fund award to promote open research practices. He is also co-director of the Tobacco and Alcohol Research Group (TARG), which is based within the School of Psychological Science at the University of Bristol, and a Programme Lead within the MRC Integrative Epidemiology Unit at the University of Bristol.



Jackie Thompson has a PhD in experimental psychology, followed by several years postdoctoral research experience in various sub-disciplines of psychology and meta-research. She has spent several years as an advocate for open research practices within psychology and academia more broadly, including working with the UK Reproducibility Network on several initiatives, mainly due to training researchers in open research practices.

Open research, as defined by the <a target="_blank" rel="noopener noreferrer nofollow" href="https://opendefinition.org/od/2.1/en/">Open Knowledge Foundation</a>, is set of practices that results in open knowledge, where “Knowledge is open if anyone [has the freedoms] to access, use, modify, and share it — subject, at most, to measures that preserve provenance and openness.” It also involves considerations such as the <a target="_blank" rel="noopener noreferrer nofollow" href="https://doi.org/10.1038/sdata.2016.18">FAIR</a> principles – making it Findable, Accessible, Interoperable and Reusable – in order to make it as useful as possible for others.However, there are many barriers that have been identified that prevent researchers from carrying out open research.Environment and external factors Researchers have been shown to consider time pressure to be a powerful factor that impacted the publishing process, as well as research and data sharing (Chawinga and Zinn, 2019; Spanager et al., 2013; Stewart et al., 2020). According to some surveys (Al-Halabi et al., 2014; Duracinsky et al., 2017; Stewart et al., 2020) and interview studies (Yarris et al., 2014), lack of time might frustrate data sharing initiatives, cause work-life imbalance, impact research quality and demotivate publishing in general. It also affects early career researchers (ECRs), who mentioned that they had no official time for conducting research (Solaja et al., 2018) or adequately preparing manuscripts. There is also a lack of guidance. Some surveys reported that scientists, especially ECRs, were not given enough training and mentorship (Turk et al., 2018). This caused difficulties during the publication process. Another survey (D’Souza et al. 2018, Editage Insights 2018) showed that early career researchers (ECRs) needed more guidance on Open Access (OA) publishing. Respondents of another survey stated that they lacked data management and publication skills, and were uncertain about organising and preparing data for sharing (Houtkoop et al., 2018). The majority of those who shared such experiences, mentioned that they would like to have more training and education in this area. Another factor that affects research culture is limited support, resources and funding. Paying Open Access fees by individual researchers is considered problematic (Fuller et al., 2014; Nyamai et al., 2020; Schroter et al., 2005; Severin et al., 2018). A qualitative study by Watkinson et al. (2017) highlighted that the OA system creates barriers to publishing and inequity of access to funds for individual researchers; therefore, they can struggle to pay publishing fees. Some reviews (Day et al., 2020; Misra, 2016; Vervoort et al., 2021) supported this point, and also mention the problem of paying fees. Results of the review by Siler et al. (2018) show that authors from high-ranked or well-funded institutions are more likely to have the resources to allow them to choose publishing options. Therefore, “there is stratification in institutional representation between different types of publishing access, there is also inequality within access types” (Siler et al., 2018). As a part of a larger perceived problem, some respondents believed that time and resources of grant proposals were used in inappropriate ways, making academics do more administrative work instead of focusing on research itself (Herbert et al., 2013). The competitive environment has also been mentioned as a barrier. Researchers point out that an unsupportive and competitive environment makes them uncertain or unwilling to share research data (Stevens et al., 2021; Stewart et al., 2020). In publishing, there is a tendency to include the names of authors from high-income countries in the first and last positions, and not to list authors from low-income countries (Rees et al., 2017, 2019). A competitive environment also has a negative effect on ECRs. In an attempt to create a reputation and to be published, they might become victims of predatory journals (McCann and Polacsek, 2018). A lack of advantage or professional reward has also been mentioned as a barrier to sharing data and publishing open access (OA). A survey by Stieglitz et al. (2020) discovered that researchers do not want to share data due to fear that other researchers will benefit at their expense because findings are more valued than data themselves, so academics will not receive the expected recognition for their work. Some studies show that academics do not feel valued enough when publishing OA or do not see OA publishing as beneficial for their career (Mozersky et al., 2021; Turk et al., 2018; Yarris et al., 2014). Many researchers may simply not consider the benefits of publishing OA, and instead prioritise publishing in journals with a high reputation, as they consider this important for career progression (Köster et al., 2021; Kuballa et al., 2017; Severin et al., 2018). Additionally, there is evidence that authors may perceive open access publications as being less prestigious and lower quality than closed publications (O’Hanlon et al., 2020, O’Kelly et al., 2019). The reputation of OA publishing may also be tarnished by perceived similarity to predatory journals which also use a pay-to-publish model (Lam and Langer-Gould, 2021). Finally, another barrier is related to journal policies, regarding both open science (Editors and WHO November 2003 Group, 2004; Zečević et al., 2021; Moustafa, 2022; O’Kelly et al., 2019; Singh et al., 2021; Strømme et al., 2022). Several reviews pointed out that journals have inconsistent policies for regulation of open science, preventing authors from sharing their data (Bakker et al., 2017; Gentemann et al., 2022; Hrynaszkiewicz and Cockerill, 2012). Another quantitative study has shown that publishers’ policies play a crucial role in the willingness of academics to share their data, and weak journal policies or unfavourable policies may frustrate data sharing initiatives (Chawinga and Zinn, 2019). Inequalities between countries with different income levels Another major theme of the barriers to sharing data and research is the inequalities faced by researchers from countries with different income levels. A problem that has emerged in the literature is related to limited resources in middle- and low-income countries (LMIC) (Brant and Rassouli, 2018) and the dominance of publications from countries with high income (HIC) (Busse and August, 2020). Among the barriers faced by researchers from LMIC are: struggling to pay publication fees (Jain et al., 2021), lack of opportunities to participate in research projects (Cazap et al., 2020), inability to meet journals’ requirements due to limited access to resources (Turk et al., 2018), lack of guidance on the research process (Editors and WHO November 2003 Group, 2004), difficulty of writing in a foreign language (Brant and Rassouli, 2018), poor access to international publications or data (Matheka et al., 2014), and lack of education which leads to poor research quality (O’Hanlon et al., 2020). Also, a barrier of inconsistent internet access was mentioned (Brant and Rassouli, 2018; Matheka et al., 2014). Another problem that appears in this context is misconduct of researchers from HIC. Some studies show that scientists from HIC undervalue contributions from those from LMIC and may at times exploit them (Rees et al., 2019). Also, there is an unfair tendency not to indicate the authorship of researchers from low-income countries, but instead to include names of authors from high-income countries (Rees et al., 2017). For instance, according to this study, 40% of multi-country studies did not include authors from every LMIC involved. This was caused by power imbalance and “authorship parasitism” among researchers from HIC. Ideological barriers to sharing Another barrier that prevents scientists from sharing their studies or data online is concerns about open research practices. One concern researchers have is about data safety, especially in qualitative research. A survey by (Mozersky et al., 2021) has shown that only 4% of qualitative researchers have ever shared qualitative data in a repository. Their main concerns were related to the sensitivity of data, getting permission from participants, and breaching trust. They also cited a lack of finances to cover repository costs, lack of guidance on ethics and a lack of assistance with data anonymisation. Another concern regarding data security was related to the fine line between predatory journals and OA journals (McCann and Polacsek, 2018). According to this study, predatory publishing has created a negative image of OA. Some scientists fear sharing their data because predatory journals aim to make a profit rather than promote scientific activity. Predatory journals may trick authors by creating false websites, hijacking journals to make them believe that they are legitimate publishers. If researchers are to be supported in producing open knowledge, these barriers must be overcome.

A summary of the known barriers to open research practices in academic research.

Organisation	Role	Department	Start date	End date
University of Bristol	Not specified	Not specified
University of Oxford	UKRN Open Research Project Officer	Not specified	2023-02
University of Bristol	Senior Research Associate	Not specified	2019-02
University of Oxford	Postdoctoral researcher in social and cognitive psychology	Not specified	2015-02	2019-01

Organisation	Degree/title	Department	Start date	End date
University of Oxford	PhD (DPhil)	Not specified		2015
Yale University	BA	Not specified		2008

Title	DOI	Published date
A qualitative analysis of stakeholder experiences with Registered Reports Funding Partnerships [version 1; peer review: 1 approved]	10.12688/wellcomeopenres.17029.1	2021-09-14
Evaluating Registered Reports Funding Partnerships: a feasibility study [version 1; peer review: awaiting peer review]	10.12688/wellcomeopenres.17028.1	2021-09-14

Jacqueline Thompson

Employment

Education

Works

Octopus publications