The highly prestigious journal Nature this week announced that it is going to be publishing all peer reviews alongside the articles it publishes. This means that the reviews attached to each publication will be available for all to read, although reviewer identities will remain anonymous unless they opt to sign their review. Nature says this move will help to ‘increase transparency and (we hope) to build trust in the scientific process’ by opening it up for all to read.
To be sure, this is a big change. While Nature already publishes some peer reviews alongside articles where the reviewers agree, now reviewers will no longer have a chance to keep their comments private. While we won’t always see the reviewer identities, or the reviews for papers that were not published, it will be interesting to build a picture of how certain reviewer comments lead to published papers. Indeed, Michael Eisen (founder of PLOS and an outspoken open science advocate) is optimistic, claiming in an NBC interview that:
“I think seeing the sausage is good […]. There’s a lot of criticism that stems from lack of understanding. That lack of understanding, from my point of view, stems from lack of transparency from scientists and science over what the process is.”
The general response to this development on social media has been that sharing peer reviews is a good thing but that Nature does not go far enough by applying the change to all its portfolio of journals and still allowing reviewers to be anonymous. This is especially important for journals like Scientific Reports that have a much higher acceptance rate and where peer review may be more readily ‘compromised’ (as Dorothy Bishop writes on Bluesky). What’s more, we will still not see the contents of reviews of rejected papers, which means that what is being made available is only a limited form of ‘open’ review. As Mike Eisen claims, “seeing what questions came up in the reviews of accepted papers is one thing; seeing why papers were rejected by journals is another.”
For me, this decision illustrates a number of problems with the move towards open science, but particularly the idea implicit in many open science interventions that transparency is a substitute for governance. So much of open advocacy hinges on the idea that making materials openly available will improve the scientific process, although there is much less written on how specifically it will improve science and even less that directly shows the benefits of transparency. This means that commercial organisations may unilaterally decide to share a fairly small subsection of their peer reviews in accordance with their own commercial conditions, all in the name of ‘transparency’. Sharing peer reviews here adds to Nature’s reputation as a trustworthy and good actor, but also allows them to point to the external reviewers as the real people to blame when bad papers are published. In this instance, the principle of open-science-as-transparency is being used to determine an action that would be better determined by research communities themselves.
I am not arguing that sharing resources is unimportant and unnecessary, but rather that transparency requires greater theorisation as part of a bigger open knowledge project. The point that open science should not be about the indiscriminate sharing of resources is well theorised in Sabina Leonelli’s short book Philosophy of Open Science. Here, Leonelli writes that sharing does not constitute a ‘sufficient condition for conducting reliable
and responsible [open science]’ because it does not reflect the situated and contextual nature of research, or what she cites as the ‘well-justified, contextualized discrimination and judgement over the value and goals of research and its components, rather than absence of judgement, disregard for the specificity of research conditions and related attempts to ‘make everything available’ (p. 8). For Leonelli, open science is therefore about ‘judicious connection’ that emphasises the relational and human aspects of how science is conducted and shared. This understanding of openness pushes back on indiscriminate notions of openness as the sharing of everything and instead asks us to consider ideas of epistemic diversity and justice in our considerations of what should be shared and how.
For me, Leonelli’s theory of judicious connection is useful because it emphasises how openness is much more about governance — i.e., the decisions taken, by whom, in what situations, and for what reasons — than it is about sharing. Although sharing materials may allow us to make better judicious connections between discrete elements in the scientific process, the real work is about collectively figuring out ways to do this. Crucially, for my purposes, this requires us to foreground collective and community forms of governance over market forms, while also providing support for research communities to experiment with different approaches to openness. It should not, then, be left to organisations motivated by commercial returns to make these decisions, as this leads to interventions that are watered down or designed with other motivations in mind.
Viewed through the lens of judicious connection, Nature’s open review experiment is not problematic merely because it doesn’t go far enough. Instead, it is problematic as it is presupposed on the idea that transparency and governance are essentially the same thing: that getting everyone to share everything should be the object of our scientific governance processes. This approach to governance feels to me to be a cop-out as it doesn’t actually help us figure out how to make the relevant judicious connections required in our research. Rather, it leads to the idea that open research should be a free-for-all of data, code, review comments, publications, etc. dictated by market-led organisations who place their own restrictions on openness in each context. Openness can only ever be researcher-led and contextual, not market-driven and indiscriminate.