Through a survey (Hsing et al. 2023a) and interviews (Hsing et al. 2023b), we previously conducted a baseline evaluation of research culture across different disciplines, including obtaining researchers' views on what may motivate or hinder sharing research more openly.

In this second stage, we will continue the evaluation, but focus on feedback about the Octopus platform itself. To obtain this feedback, we will ask participants in the study:

1. How is the Octopus user interface/user experience (UI/UX) received by researchers? Specifically:

   1. How effectively is the website communicating the goals of Octopus to different types of researchers, and how it works?

   2. How discoverable, understandable, and usable are the features of Octopus? What are the facilitators and barriers to their use?

2. How well does Octopus fit into existing research culture and infrastructure? We would like the perspectives of:

   1. Individual researchers

   2. Institutional research infrastructure managers

Methods

The methodology of this evaluation is primarily a focus group with researchers, focused on the user experience of Octopus. Participants would be emailed a link to Octopus before the session, and shown a brief demonstration at the beginning of the focus group.

Then, the focus group would commence with some guiding questions on if and how Octopus would fit into a researcher’s workflow, and how they perceive the wider culture will respond to this platform. These open-ended questions will provide insight into what may facilitate or hinder the adoption of Octopus.

We intend to run at least two focus groups with five participants each for a total of at least 10 researchers. Online focus groups will be conducted using the Zoom video conferencing tool. Ideally, at least one of the focus groups would be in-person, which may be more conducive to a stimulating discussion among the participants. For the in-person focus group, it will likely be with researchers at the University of Bristol (where we are physically based) due to logistical constraints.

We aim for focus groups to last approximately 60 minutes, with audio recordings that are transcribed into text for analysis. The recordings will be deleted once the transcripts are complete with personally-identifiable information removed.

The methods described here have received ethical approval from the University of Bristol School of Psychological Science Research Ethics Committee with reference number 12335.

Participants and recruitment

Participant characteristics

For individual researchers, we will strive to recruit those who represent diversity across the following characteristics:

• Basic dimensions of diversity such as gender

• Geographical location, including outside of the Global North

• Research discipline

  • Natural sciences

  • Social sciences

  • Engineering

  • Arts and humanities

• Research methods employed

  • Qualitative

  • Quantitative

  • Theoretical

  • Applied (e.g., engineering)

• Sector

  • Academic

  • Industry (ideally not just the pharmaceutical industry)

  • Ideally some from none of the above

• Publish other research objects (i.e. not code or “data”) such as:

  • Multimedia

  • Hardware designs

  • Digitised resources

  • Non-article-based publication types (e.g. monographs, art pieces)

Participant recruitment

Similar to the previous evaluation, we have a diverse and international network of professional contacts, and participant recruitment could begin with a snowball approach from a convenient sample from this network. In addition, we can recruit participants through professional organisations. Other potential ways to reach participants include:

• Early-career or postdoctoral researcher networks

• Professional societies in various disciplines

• Contacts of Octopus within universities (mainly in Europe, including the United Kingdom)

• Contacts in private companies

• National laboratories/research institutions in the United Kingdom and United States.

We anticipate that, time-wise, participant recruitment and scheduling the study will probably be the least predictable step. Each participant will be offered an incentive worth GBP 25 in the form of an online shopping voucher or donation to a non-profit organisation.

Materials and measures

Pre-focus group activity

Approximately one week before the focus group, we will email and request participants to visit Octopus (https://www.octopus.ac/) with guiding questions including: “In these 5 minutes, can you…”:

1. Find and understand the key motivations behind Octopus?

2. Identify the types of research that could be published on Octopus?

Focus group plan (approximately 60 mins)

1. Quickly introduce yourselves (10 mins) - In two minutes, tell us about your career stage and the nature of your research, including methods and outputs.

2. Octopus demonstration - Where the facilitator goes through the Octopus.ac platform, giving an overview of its motivations and key features (15 mins)

   • This could include using a demo account to publish something in real time.

3. Barriers/facilitators to using Octopus (15 mins) - Would you want to use Octopus? What about Octopus would make you want or not want to use it?

4. Perception of how others might react (10 mins) - Do you think others in your field would want and will use Octopus to publish research? Why or why not?

5. Feedback form (5 mins) - Dedicated time for participants to provide additional feedback by answering these questions in writing (no identifying information will be collected, and participants will be asked to not provide it):

   • Which research discipline(s) do you primarily identify with?

   • On a scale of 1 to 5 (with 5 being highest), how familiar are you with open research practices (broadly defined)?

   • Does the organisation of publications on Octopus fit with the research workflow of your discipline(s)? Please explain.

   • What would convince you and others in your discipline(s) to make use of Octopus to publish research earlier and more often?

   • Do you have any other critique or feedback for Octopus or today's session?