Participants had varying levels of familiarity with Octopus prior to the interviews and focus groups. After the Octopus platform was explained during the sessions, participants demonstrated in the ensuing discussion an understanding of key features such as: the motivations for creating Octopus; its eight publication types; how they form the “version of record” while facilitating branching chains of research; finer grained attribution for researchers; and possible use as a form of pre-registration.
Both researchers and research support staff identified ways in which Octopus could already be a useful part of the research lifecycle. However, the majority of their responses focused on cultural and systemic barriers to the widespread adoption of Octopus for research publishing. These issues, and specific feedback on the features of Octopus, are described below.
Feedback on Octopus features
Before being given a verbal explanation of the primary motivations and features of Octopus, it was unclear to several participants how it differs from other open research publishing platforms. One librarian visited the Octopus homepage prior to their interview and asked: "Why would you choose this over something else [such as the Open Science Framework]?" They noted that the primary differentiators of Octopus–such as its eight linked publication types, granular authorship, and branching chains of research–are not obvious at first glance. Another researcher remarked that, upon a cursory inspection, it was unclear if Octopus expects or allows authors to upload their dataset directly to a "Results" publication.
Once the key functionalities and workflows of Octopus have been explained, several researchers questioned whether this platform would be easier to use than those employed by academic journal publishers. According to one biomedical researcher: "...one thing that would make me choose [Octopus] instead of other [sic] is how easy it is to load the things on it... like all of the editing steps that are very often very annoying and different from every platform or every journal". That is, this researcher may be more willing to publish on Octopus if it is easy to use. As will be discussed below, researchers are time-limited, and they anticipate usage of Octopus will be low "if you're going to have to spend ages getting everything in the right format".
In particular, interviewees took note of the branching publication chains enabled by Octopus. Some observed that these branches are akin to "forks" of Git-based (https://git-scm.com/) version control code repositories. One librarian has experience supporting national-level research policy and sees these branches as a potentially effective way to facilitate multidisciplinary collaboration, a key goal of their policy. That said, some found it unclear how peer reviews published on Octopus would be visualised as part of a chain, while others were confused whether explicit links to existing items are required when publishing to Octopus.
A common concern was how Octopus implements quality control and moderation of published items, including for spam or other inappropriate content. It was also not clear whether permission from the author is needed before linking to an existing publication. For peer reviews, one asked: "Can anybody just show up and review? Does that sound a bit weird?" Specifically, one librarian was concerned about whether Octopus publications are tied into automated plagiarism detectors, and ambivalence on whether that will help or hinder publications. This is not just about screening submissions to Octopus. They were especially concerned that if a researcher publishes to Octopus first, their subsequent submissions to academic journals would be considered plagiarism.
The majority of comments about Octopus functionality related to interoperability on the level of publications, technical infrastructure, and peer review.
For publications, participants asked whether preprints such as those published on arXiv (https://arxiv.org/) or bioRxiv (https://www.biorxiv.org/) could be retroactively imported into Octopus. For librarians, they emphasised the importance of tying Octopus publications into existing tracking systems for publication metrics such as Altmetrics, and functionality to explore how they are cited in other works.
Technical interoperability is important to researchers in several ways. One is a desire for an Octopus application programming interface (API), where researchers can automate publishing to Octopus–or extracting data from Octopus publications–using scripts within their computational workflows. They believe having a programmable interface in parallel to the current web-based forms would ease collaboration between technical and non-technical researchers.
A related feature request is for Octopus to automate publishing through external triggers. Specifically, they cited the integration between Zotero (https://www.zotero.org/) and Figshare (https://www.figshare.com/) with GitHub (https://www.github.com/), a popular code hosting and version control platform. While the software development process in a GitHub repository is continuous, a user could "tag" a particular snapshot for release. The associated Zotero (or Figshare) item would be triggered to save a copy of that snapshot with a new digital object identifier (DOI). The bioinformatician from one focus group believed having such a feature for Octopus would allow researchers to try the platform without major changes to their existing computational workflow. Similarly, one librarian described it as "a good kind of easy selling point."
There is also a desire from participants for integrating Octopus into other open research infrastructures. One interviewee in charge of open science at a major non-profit funding body listed the systems they use including Datacite (https://datacite.org/), OA Works (https://oa.works/products/), OpenAIRE (https://www.openaire.eu/), and the Research Activity Identifier (RAID: https://raid.org/). In particular, they stressed that their grant management system is struggling to track the research outputs and associated metadata of funded projects. To get an overall sense of a project and its outputs, "you have to do a lot of [manual] curation to link those things."
An initial step, as one librarian suggested, is to embed more structured metadata for Octopus publications, namely the Contributor Roles Taxonomy for describing author roles (CRediT: https://credit.niso.org/). A further step would be integration with existing research discovery systems, such as Europe PMC (https://europepmc.org/). Ultimately, these integrations are viewed as benefiting researchers, to help them ask "how can I collaborate, who can I collaborate with, [and] who's working on this?"
The benefits of interoperability and integration with other systems is not limited to funding or conducting research. For example, if structured metadata about the nature of research labour is included with Octopus publications, then that could be integrated with job search and networking platforms like LinkedIn. For example, a statistician could showcase their skills on a job search website by linking to an Octopus publication that demonstrates them.
Even if Octopus is widely adopted, some are concerned of becoming over-reliant on one provider. One computational scientist observed that the underlying source code for Octopus is open source, and asked if that means individuals and institutions can host their own instances. If these self-hosted instances could then interoperate ("federate"), it will eliminate reliance on one organisation, thereby increasing the resilience of the overall network.
For peer review, participants wondered if Octopus helps organise peer review, because there is currently little incentive for researchers to do so. One focus group member suggested connecting to existing open peer review groups like Peer Community In (https://peercommunityin.org/) or the Review Commons (https://www.reviewcommons.org/). For one librarian, allowing researchers to publish peer reviews of works outside of Octopus would increase visibility and adoption of the platform, and facilitate the move towards post-publication peer review.
Lastly, one researcher stated that powerful search and discovery features would make Octopus useful for them, this includes checking "whether the question you're asking is already in there [as a research problem]". Without performant search functions, then "if I have to search for all the possible keywords, like sometimes it's really, really hard to find the results you're looking for..."
How is Octopus already useful?
Several participants identified Octopus functionality that already facilitates adoption, from granular recognition of research labour; focus on sharing more of the research process; multi-language support; to enabling meta analyses.
As discussed in our previous report on research culture, traditional peer-reviewed journal publications are the predominant way of sharing research. A widely-shared sentiment is that author lists on such publications do not communicate the diverse contributions to a research project. For one focus group participant, "it's not immediately obvious which [author made] a more valuable contribution to be like, first or second author say, so I like the idea of [...] being able to kind of split up that recognition [on Octopus]." Granular recognition of research labour also has a practical advantage, where: "...it's quite useful from a scientific approach to know who to email, to ask the silly questions, instead of asking the corresponding author who might have just put the whole publication together." Granular recognition is also appreciated by a librarian, who hoped they can one day receive authorship on an Octopus publication, as librarians are historically under-recognised in research outputs.
For a traditional paper-focused publication process, it often forces a researcher to retroactively develop a narrative for their research, and that's after a "several year long research process... [where] you may have changed your mind substantially over what it is that you've been working on since you started the project." In contrast, participants appreciate the ability to publish more diverse outputs from their work on Octopus, where, instead of the current publish or perish culture, there could be an "attitude shift in scholarly communication mechanisms [that reflects] different elements of research." What Octopus and, to some extent, pre-registration provide is "documenting that process, [providing] much more transparency and accountability and even clarity about your own thinking process and how that's changed over time".
Because of how Octopus publications resemble pre-registration, one librarian believes the platform will be easiest to adopt for researchers in medicine and the life sciences. Octopus could be presented as another way to pre-register research. For the interviewee who works for a research funder, Octopus is also attractive because they want their funded projects to publish results as they are, including "negative results". According to them: "...negative results are not really something people want to, you know, share [in a traditional paper]. But from a funder perspective that is open minded like us [...] we do want that information. We do want the reporting, we don't call it like negative results, we call it just [...] 'results'."
Several participants praised the support for multilingual publications on Octopus. For the mathematician in a focus group, citing very old, non-English papers is common practice: "...a paper from the 1700s is, it can be as relevant as anything else". For the librarian who supports national-level research policy in a Southeast Asian country, they described a government initiative to create more academic journals in their native language, and Octopus could be a part of that effort.
Finally, participants perceive Octopus publications to be useful for meta analyses. This could be systematic reviews of clinical guidance for medical research (such as a review of published "Method" publications as the dataset), or as a way to conduct network analyses on researcher roles and how they collaborate.
Barriers to adopting Octopus
Lack of time
While one objective of Octopus is to give granular recognition of labour, there is fear that it will not be accompanied by cultural change where the division of labour is more equitable. For example, when publishing a traditional paper, one researcher's experience was: "it's only me and my colleagues that make everything and detail everything", while other authors did very little. They fear the experience will be the same for Octopus even if the aim is to change authorship politics.
The primary barrier to adoption, however, is widespread perception of a lack of time to learn and use Octopus. As one university librarian described it: "We could certainly suggest [Octopus] to people. What the uptake would be on it, I don't know." Another remarked: "...everybody's trying to do more with less. So yeah, giving people a compelling reason to use their time on this particular platform is going to be the key thing."
This is almost universally supported from focus group feedback where participants–while praising the potential usefulness of Octopus–are hesitant to use it themselves, fearing it will not be worth their time. Many expressed feelings of being overwhelmed with academic work, including anticipating more time pressure later in their careers.
One engineer said: "...my boss is a professor, like he works until midnight [...] so I don't think it will change, it will never change..." When faced with a platform like Octopus, a physical scientist lamented: "...for the people I work with, there's no way they're gonna interact with [Octopus]. It's just not gonna do it... I love the idea, but [...] the workload... It's just like, no, I can't cope."
Another researcher compared Octopus to current preprint servers where there would be "50-60 preprints [relevant to my discipline] per day on arXiv". That is already overwhelming to keep up with, and they fear a similar predicament with Octopus publications. A differentiator between the two is that preprints are close to–and have a clear path towards publishing–a traditional paper, which justifies the time investment.
The lack of time is also accompanied by a perception that it will take substantial effort to learn a new tool such as Octopus. One life scientist described how their lab still worked in a "very analogue way" with physical notebooks, and a general cultural aversion to adopting digital tools.
Using Octopus is neither rewarding nor required
Overwhelmingly, the primary reason behind the perceived lack of time is the absence of career benefits for academic researchers. For Octopus to see higher adoption, "[it] requires an overhaul in the whole system of grant applications, job applications, funding and everything, because if we're devoting time to put it all here rather than writing a Nature paper, say, then that's got to be recognized." One participant observed that: "one thing that is probably similar, no matter what field you come from, is whether or not you're going to get a career benefit from doing the nitty gritty of breaking down all your work." This view is shared by a digital humanities researcher, who believes it is critical for the use of Octopus to have "a career related benefit [...] especially for early career researchers." One despondent librarian who promotes open research practices laments that "...we try and help but sometimes it feels like a sticking plaster rather than [...] actually addressing the underlying issue."
Critically, most participants believed the use of Octopus should be recognised by funding bodies, where "the more kind of funding bodies that recognize [Octopus], then the more that becomes important." And because UK Research and Innovation (UKRI)–through Research England–is one of the funders of Octopus, a common question during the focus groups was: "What does UKRI want to get out of this?" One researcher asked: "I was just wondering if UKRI has planned anything to incentivize people to use it?" Another was more explicit: ""If [Octopus] is funded by the [UK] funding councils and the funding councils want us to use it, then they have to use this rather than the first author or whatever [paper-based metrics] in their grant sifting processes and things."
In addition, some suggested that requiring the use of Octopus in research publishing–whether by institutions or funders–may engender a gradual cultural change where the use of such platforms eventually becomes normal and expected. A postdoctoral fellow studying abstract mathematics noted that "one of the things that made people start caring about [open access papers] is the fact that most grant awarding committees [started requiring] open access. And since then, all the mathematicians put everything open access." They stressed that, ideally, there should also be "international buy-in" for these requirements. That said, one librarian warned that if the use of Octopus is only required instead of rewarding (for career advancement), "[researchers] will probably still just see it as a box ticking thing, which is another arduous task that they have to do and may not see any purpose in it..." Similarly, the open science lead at a major medical research funder recalled a strong top-down push for pre-registration for their funded projects. However, "at the end of the day, it kind of fell flat, because the community was not ready, the culture, it just never went anywhere..."
As a compromise measure, one focus group participant noted that "if you want to have a permanent [academic] position, you have to publish in [a] very high impact factor journal. And if Octopus can provide some things, some connection with this kind of journal, maybe not Nature, but some well known journal, it can be really a plus." Such a connection could be, for example, an academic journal which uses Octopus as an officially sanctioned repository for supplementary materials that accompany papers. They stressed that "it's not a good idea for science, but [for] now it's the rule of the game."
In addition, both researchers and librarians suggested functionality for tying Octopus publications to existing metrics for research impact. This could be the UK's Research Excellence Framework (REF) categories, the United Nations Sustainable Development Goals (SDGs), or funder measures of public engagement. For the latter, an engineering researcher believes if Octopus would support interactive elements or videos that are friendly to non-experts, then its publications could be an avenue for public engagement that some funders assess. In any case, one interviewee emphasised that these integrations should consider national-level research assessment key performance indicators (KPIs) to incentivise wider adoption.
Incentives also need to exist for research institutions rather than only individual researchers. One librarian suggested that for universities, "...the people who are in charge of the money are the ones who are not as enlightened. So you have to have a financial or reputational advantage to adopting something and pushing something."
Epistemological diversity
The focus group participants revealed a diversity of ways in which research is conducted. Notably, a humanities researcher stressed that "the humanities often get squeezed out of these [open research] conversations [which] are geared towards what works for the sciences, and so I'm thrilled to be part of the conversation."
However, the publication types of Octopus may not be a good fit, where "the norms of [some] fields don't really support the sort of archiving of data, notes, that kind of thing that you might expect in chemistry or lab based work". This is a view shared by three of the research support staff interviewed, where one said: "...for fields where the workflow is less sort of, you know, idea, data gathering, testing, analysis kind of thing, [Octopus] will be I think more difficult to sell". For some disciplines, "it's sort of internal thinking and a discussion with colleagues and peers. But kind of the process of turning that thinking into something that is shareable is kind of done in quite informal ways." This form of informal or unstructured intellectual work may, for example, take place during a workshop where "you could obviously just record a workshop and put it online, but [...] where it's so freeform, it feels difficult to imagine an audience for that [...] It feels quite tricky... [to] pull out themes and all this kind of stuff. Yeah, that would be a whole other research project in and of itself."
One librarian has worked extensively with researchers in the humanities. According to them: "[On Octopus], you've got hypothesis or you've got rationale. It's not the way that arts and humanities people think, [which is] much more free flowing." This is confirmed by two focus group participants who are researchers in different fields of the humanities. One is a historian, who noted that–with the possible exception of some sub-fields such as epigraphy, papyrology, archaeology, or the digital humanities–"from the humanities perspective [...Octopus] doesn't really add much value". One reason is because single author monographs are their primary research output, so granular authorship considerations do not apply.
The historian further stated that "the intermediary steps aren't really things that can be identified and analysed in their own right". When asked about what those steps are, they described it as a process where research (primarily reading) is deeply intertwined with writing, and a published monograph is the culmination of that process. This research workflow is confirmed by a librarian, where "[Octopus] is almost like a structure which might seem alien to them because they haven't worked out what they're doing until they've written it all down, which is probably the exact opposite of the physical sciences... [some researchers]'s brains are just a flow of consciousness and they've then got to try and wrestle it into [...] an Octopus, which is a visual image I didn't really need."
Perhaps most critically, a sentiment shared by several participants is the lack of representation in the discourse on open research by social sciences and humanities researchers. One gave an example where, during a meeting about research culture, "I'm shocked by how there are 25 postdocs from the different sciences that I didn't even know existed, and I'm the only person from the humanities, and there's one person from the social sciences." If open research practices are decided democratically, then the "scientists are always going to [dominate]". The participants stressed that the Octopus publication model is based on that of the experimental sciences, and if subsequently pushed onto other researchers, "people will also ignore it and [be] very resentful".
Epistemological diversity and disciplinary differences are not limited to social sciences or humanities research. One bioinformatician said that their research workflow is entirely based on computational notebooks. While they expressed appreciation for how Octopus "focused on the quality of the work rather than necessarily the publishability of the work, which aren't terribly correlated", their open-ended workflow is not a good fit with Octopus because "[it will end] up being sort of confined into a what can we publish [on Octopus...] and so on".
For the researcher in abstract mathematics, "all the information about the proof or whatever would be contained within that one [paper]. And so there kind of really isn't anything extra you need to understand it or to support it. It's kind of an entirely self-contained thing..." while there is nothing substantial in terms of hypotheses or data collection.
The aerospace engineer in one focus group noted that one barrier to adopting Octopus is shared by other open research platforms, namely that industry partners on their projects tend to require research details to be kept secret. In one case, "90% of the work comes from industry. And then it would be like just [sic] impossible to use [Octopus]."
In any case, one social scientist observed that regardless of discipline, there could be researchers who are not academics. These researchers may not have ORCID iDs, which is required for registering an Octopus account. In research, they believe "recognition of non-academic or civil society [...], or even policymaker type contributions that could happen to shape research work [and] reports and things like that [are important]", and asked "how does that inclusion or exclusion [of eligibility] happen [on Octopus]?" They are concerned that because the technical infrastructure and logical structure of Octopus publications are geared for academic experimental scientists, other researchers will feel excluded.
Peer review
One aspect of disciplinary differences is researchers within them have diverging views on what the most pressing problems are. As reported above, the humanities researchers in the focus groups do not believe the Octopus publication types are epistemologically or methodologically compatible with their work. There is resentment towards open research initiatives because they are often informed only by the concerns of STEM research. For example, the push for open access publishing "came from the sciences", which is often well-funded compared to the humanities and can afford high article processing charges (APCs). In contrast, one historian in a focus group stated that within the humanities, effective peer review is widely considered to be a more urgent issue where "it's always really, really hard to find people that are willing to put effort to referee papers..."
In fact, several other researchers, including those in STEM subjects, shared the concern about peer review, noting "if people that behind Octopus could actually aim to change the way we peer review work, I think it would be great. Because as [the historian have] said, the peer review system is just wrong."
Some observed that having more, smaller pieces of open peer review will reduce the burden on individual reviewers who are currently expected to provide a comprehensive critique. One computational scientist used the analogy of open source software hosted on GitHub, where "on your work, you will get like 100 peer reviews [through discussion thread comments]. And then after hundreds, you know that your work is good because 100 people actually look [sic] at your work and just left a comment".
One suggested that if Octopus peer reviews could be integrated with open peer review communities such as Peer Community In or Review Commons, then it can become part of the wider peer review reform movement. This sentiment was shared between focus groups, where another participant believes "if Octopus could take over and maybe in 10 years time, actually completely revolutionise the way we do peer reviewing [that] would be great".
Notably, the librarian who has worked on national-level research policy recalled an effort to incentivise better peer reviews by funding small payments for reviewers, and by giving them "credits" that count towards research assessment. However, they observed that journals still struggled to find reviewers, perhaps as these rewards are too small to enable behaviour change in a publish-or-perish culture.
Fear of scooping
Several participants were concerned that publishing on Octopus–while showing "that you were the first person to have had this idea"–would expose them to scooping by competing researchers. This could mean that competitors get to publish a paper first, which is what currently counts in research assessments. For example: "...if you're working on a new compound [and something is not quite working], but another chemist might see that and go: 'Oh, actually maybe switching that functional group to another functional group is going to make it work'. So you're sort of helping them get there quicker because you've put your results out there, [then someone else would] go off and publish first [...] in a journal." In addition to papers, for research outputs that could be commercialised, there is also fear among some participants that their work would enable others to profit from it first, including by applying for patents.
On this topic, one researcher observed that one advantage of Registered Reports in the current academic system is that they come with assurance of a published paper, while Octopus does not. They suggest that if Octopus collaborates with journals so its publications can be treated as Registered Reports, it may drive adoption.
Fear of scooping also manifests during the competition for grant funding, where "if you're working on something that's quite sensitive and other people are working on something similar, then you might be a bit more reluctant to have all your thoughts out for the public who can then [...] apply for a grant..."
Network effects
In addition to listing barriers, a few participants suggested possible paths towards Octopus becoming more widely adopted. One interviewee supports the professional development of research staff at their university, and described how staff members would utilise social media platforms such as LinkedIn or Facebook for networking. The popularity of these platforms is partly due to the network effect, where their value proposition is proportional to the number of users. In other words: "And you almost felt left out if you weren't [using it]." This participant suggested that while the number of Octopus users is currently low, they can see early adopters seeding its growth so that eventually the network effect of Octopus would create social motivation for researchers to use it.
The focus group participants had suggestions on who those early adopters could be. For instance, one believed that it would take those in leadership roles, such as principal investigators (PIs), to catalyse behavioural and culture change within their projects and research groups. Though to motivate such researchers, "[it] probably depends on that how much visibility that would imply." This view is shared by a librarian, who noted it would take "champions"–including PIs–to drive initial adoption. They suggested that PIs who, for example, already publish preprints, could be more amenable to publishing Octopus.
A combination of raising awareness for Octopus among all researchers while working with early adopters in leadership roles, one participant believes "the next generation of researchers or one about five years down the line who grow up with this tool and collaborate internationally, I could see [Octopus] being amazing". It is also important for Octopus to be connected to wider institutional reforms where, according to one librarian, "...in future years, [Octopus] may become more useful as the groundswell of open research and different scientific practices is embedded [in research culture]".