My First Time: Open Peer Review as a Social Scientist

The publications wall at the Public Governance Institute.

Slow-brewing motivations

My interest in open peer review has been brewing since the first year of my PhD. By open peer review, I mean open referee reports and open identities of authors and referees (cf. Ross-Hellauer, 2017). In this blog post, I present my opinions on how the design of open peer review improves on the flaws of the reality of anonymous peer review.

It remains to be seen how open peer review will play out at greater adoption rates. I regard it as highly promising not because of specific technical details, but because it takes peer review in the right direction: towards more accountability.

As a PhD candidate, I feel the need to future-proof my work through open research practices such as preregistration, open data, and open peer review. In public administration research, open-science practices are becoming more visible, but open peer review is not yet mainstream. It is noticeable in which direction the train is heading, and it is heading towards open research.

If I do not submit at least one manuscript for open peer review, I am afraid that future hiring and grant committees may ask why my PhD did not involve more up-to-date research practices. This year, I submitted my very first manuscript for open peer review. You can read the published manuscript on Open Research Europe.

Academic anxieties are best served with coffee.

My rather self-interested motivation was stirred by my experiences with anonymous peer review. Anonymous peer review is a relatively inefficient system that creates a lot of random judgments under the guise of anonymity and the weight of high throughput.

For example, I have had manuscripts rejected in the second round because one referee deemed the study not to be about a particular topic. I have seen review cycles for ten revised sentences take eight months. I have also read reviews so generic and detached that they seemed to be AI-generated.

These are examples from my personal experience. But they illustrate what everybody already knows about the inconsistency, long delays, and generative AI troubles of double-anonymous peer review procedures in legacy journals (Aczel et al., 2025).

Double-anonymous means that authors and referees are anonymized. It is also called “double-blind” review. Naturally, in many fields, a referee can simply google the title of the submitted manuscript to know who or which unit submitted it. In that sense, since the widespread adoption of the internet, it is questionable whether double-anonymous review actually still exists. It would be more accurate to refer to it as pseudonymized-author-and-anonymized-referee review — but that’s a mouthful.

Double-anonymous peer review: the good, the bad, the ugly

The good side of double-anonymous peer review is obvious. You can receive detailed feedback from two or more referees without having to pay for it. At the best journals, some referees read the manuscript and appendices very closely and provide careful, constructive, and genuinely useful comments that improve the paper. Peer review can improve a manuscript substantially when reviewers engage seriously over a longer period of time rather than quickly judging it (Garcia-Costa et al., 2022).

The bad side is that this system depends on unpaid labor. Authors benefit from referee reports, while referees usually get little or nothing in return. Unsurprisingly, review quality is highly uneven. Even strong papers can be rejected based on vague impressions dressed up in academic language.

The ugly side is the cumulative effect of this system on time-to-publication and unfair practices. Journals are finding it harder to recruit scholars to do unpaid work for them, both as referees and as editors (Aczel et al., 2025). This has slowed down peer review processes enormously if we also count the time lost when moving between journals.

In practice, large amounts of time are lost to waiting, reviewer recruitment, resubmissions at other journals, and repeated review cycles. Most importantly, these delays increase faster than the rate of substantive improvement to the manuscripts. Most often, repeated review cycles do not involve any changes to the underlying empirical research. The slowness of the overall system is especially damaging for scholars in the Global South who do not have the institutional security or resources to absorb multiple unsuccessful review processes.

The second ugly side of anonymous peer review is really ugly. Because the reviews and revisions remain hidden, unfair practices are difficult to detect or challenge. Anonymity protects unfair practices from scrutiny. We do not know whose manuscripts get published, how quickly, and at what cost (Björk and Solomon, 2013).

I am amazed at how social scientists have built an anonymized peer review system that is shielded from scrutiny, given everything we know about the importance of accountability (Aczel et al., 2025). By accountability, I mean literally having to present accounts of what everybody did during peer review.

Peer review is also a communication system.

A sustainable transition strategy for individual social scientists

For me, moving all my manuscripts to open peer review would not be sustainable. In academic hiring and grant evaluation, publications in top-ranked legacy journals still carry a strong status signal that opens doors (Morgan-Thomas et al., 2024). Everybody knows this, except for a goody two-shoes first-year PhD student perhaps.

Many scholars see only one strategy worth pursuing: getting published in the hierarchy of prestigious to less prestigious anonymous-review legacy journals. Stepping outside that hierarchy completely may be admirable in principle. But it would merely gain other scholars’ sympathy, disappointment, or even relief that you are out of the game.

Academic status hierarchies are sticky.

However, completely avoiding open peer review would be short-sighted. The European Commission, the ERC, Horizon Europe, and several major national funding agencies have expressed long-term support for Open Research Europe (Research Council of Norway, 2025). One should not underestimate the power of these funders:

• Swedish Research Council (SRC)
• Research Council of Norway (RCN)
• German Research Foundation (DFG)
• French National Research Agency (ANR)
• Austrian Science Fund (FWF)
• Slovenian Research and Innovation Agency (ARIS)
• Portuguese Foundation for Science and Technology (FCT)
• Spanish National Research Council (CSIC)

Open peer review has already gained ground in parts of the natural sciences, where open-science reforms often emerge earlier. Those reforms are adopted by the social sciences a few years later. The traditional anonymous peer review system is under visible strain since 2022, the year that ChatGPT was initially released.

That is why a mixed strategy seems the most sustainable and future-proof. My aim is to publish in both systems at the same time. Concretely, I try to submit one journal article per year for open peer review, while sending my other manuscripts to highly ranked toll journals with double-anonymous peer review procedures. If I only have one journal article in a given year, I submit it to a highly ranked toll journal.

While the direction is clear, this transition will take several more years. Think of how long it took for preregistration to become “a thing” in various social sciences, including public administration research (Belardinelli and Zhu, 2025). European funders now openly request open research, including open peer review and open-source publications, but this has not yet fundamentally changed academic status hierarchies.

The top scholars in the social sciences come from a generation who built their careers through top toll journals with anonymous peer review. For them, a transition to open peer review is not obviously in their own best interest (Morgan-Thomas et al., 2024). By contrast, today’s social science PhD candidates and postdocs are gaining direct experience with open peer review. But the current and future cohorts will only move into professorships and permanent research positions in about seven years’ time.

That is why I do not see a clean break ahead. Instead, we will experience a long period of overlap between two systems.

My experience with open peer review thus far

My experience with Open Research Europe has been very positive so far. The editorial and typesetting services have been the best I have seen up to now: rigorous, fast, and unusually clear in their communication.

In my case, the editorial team has taken the time to write detailed and helpful emails that explain exactly where the format of the manuscript could be improved. I have not yet completed the first round of open peer review, so I cannot say anything meaningful about the reviews themselves. I will update this blog post once the first referee reports start coming in.

Conclusion

I think it cannot hurt to submit one manuscript for open peer review. At the very least, it helps to future-proof your PhD for the case that open peer review becomes more mainstream.

At the same time, I found the overall experience nerve-wracking. It is not easy to step even slightly outside the dominant status hierarchies of the academic communities we belong to, or wish to belong to. Open peer review is still relatively novel in public administration research, compared to the natural and medical sciences (Wolfram et al., 2020).

I therefore had to remind myself to be rational and pragmatic. The best way to overcome my anxiety was simply to submit one manuscript to Open Research Europe.

PS: I do not discuss the problems that are induced by the journal-based structure of the old publishing system. For instance, having multiple competing journals leads to incentives for editors and referees to reject studies with null results for fear of deteriorating the journal impact factor. This is not the focus of this blog post. Nevertheless, it is worth noting that the gateway-based system of Open Research Europe improves upon the old journal-based system by removing incentives that lead to publication bias.

PPS: I also do not discuss issues of low-quality open access journals, where peer review is superficial or purely performative (Aczel et al., 2025). I do not have any experience with such journals. Again, the solution here seems to be open peer review. Who would want to be publicly disclosed as a superficial or performative referee who green-lighted a crappy manuscript?

Sometimes, one may not be able to give a full green light. Rather than making the choice binary, as with anonymized peer review, open peer review allows for nuanced referee decisions, such as “approved with reservations”. This nuance would also help non-academics immediately see that academic knowledge at the frontier is not black-and-white. “It’s published” does not mean “it’s true”.

PPPS: For a really good laundry list of all sorts of issues with the old peer review system, have a look at Aczel et al. (2025).

References

Aczel, B., Barwich, A.-S., Diekman, A. B., Fishbach, A., Goldstone, R. L., Gomez, P., Gundersen, O. E., von Hippel, P. T., Holcombe, A. O., Lewandowsky, S., Nozari, N., Pestilli, F., & Ioannidis, J. P. A. (2025). The present and future of peer review: Ideas, interventions, and evidence. Proceedings of the National Academy of Sciences, 122(5), e2401232121. https://doi.org/10.1073/pnas.2401232121

Belardinelli, P., & Zhu, X. (2025). Pre-registering public administration studies: Avoiding the poor practice of a “best-practice”. Journal of Behavioral Public Administration.

Björk, B.-C., & Solomon, D. (2013). The publishing delay in scholarly peer-reviewed journals. Journal of Informetrics, 7.

Garcia-Costa, D., Squazzoni, F., Mehmani, B., & Grimaldo, F. (2022). Measuring the developmental function of peer review: A multi-dimensional, cross-disciplinary analysis of peer review reports from 740 academic journals. PeerJ, 10, e13539.

Morgan-Thomas, A., Tsoukas, S., Dudau, A., & Gąska, P. (2024). Beyond declarations: Metrics, rankings and responsible assessment. Research Policy, 53(10), 105093.

Research Council of Norway. (2025). Statement of Intent on Open Research Europe. Zenodo. https://doi.org/10.5281/zenodo.14725287

Ross-Hellauer, T. (2017). What is open peer review? A systematic review. F1000Research, 6, 588.

Wolfram, D., Wang, P., Hembree, A., & Park, H. (2020). Open peer review: Promoting transparency in open science. Scientometrics, 125(2), 1033–1051.