About

Launched: 2017
Record Updated: Oct 04, 2024
Peer review system
PREreview.org is a platform for crowdsourcing open preprint reviews. PREreview designs and develops open source infrastructure to enable constructive feedback to preprints at a point in time in which it is needed, offers peer review training and mentorship, and partners with like-minded organizations to create opportunities for collaborative review experiences that defeat cultural and geographical barriers.

Mission

PREreview's mission is to bring more equity and transparency to scholarly peer review by supporting and empowering communities of researchers, particularly those at early stages of their career (ECRs) and historically excluded, to review preprints in a process that is rewarding to them.

Key Achievements

In 2023, we turned our full attention to the community, creating more opportunities for anyone to engage with us, provide feedback, share experiences and expectations that could guide our work. We now send out newsletters every two months to around 2000 individuals and provide weekly product updates on our blog. We invite community members to participate in design projects and one-on-one calls, with compensation offered.

Through community feedback, we introduced PREreview Clubs, where groups of people with similar affiliations, interests, or locations collaborate to give feedback to preprints, research articles posted online before undergoing journal-organized peer review.

We also launched Structured PREreviews, which use tested questions and additional comment fields to help users quickly evaluate research papers and offer more context and explanations for authors to improve their work.

Soon, authors submitting preprints to certain platforms can request feedback from PREreview community members and Clubs using a system called COAR Notify, which will create new opportunities for authors and reviewers to connect, share insights, and work together.

Technical Attributes

Maintenance Status

Actively Maintained

Open Code Repository

Implemented

Technical Documentation

Implemented

Open API

In Progress

Open Data Statement

Implemented

Open Product Roadmap

Implemented

Technical Attribute Statements

Technology Readiness Level

  • Actual system proven in operational environment

Code Licenses Used

  • MIT License

Content Licensing

https://creativecommons.org/licenses/by/4.0/

Standards

Integrations

Airtable, Azure, CiviCRM, Cloudinary, COAR Notify Protocol, Fathom, Ghost, Google Drive, OAuth, Sciety, Slack

Community Engagement

Code of Conduct

Implemented

Community Engagement

Implemented

Contribution Guidelines or Fora

Implemented

Community Statements

User Contribution Pathways

  • Contribute funds
  • Contribute to education or training
  • Contribute to peer reviews
  • Contribute to user research or user testing
  • Contribute to working groups or interest groups

More About Community Engagement

Community Engagement Activities:

Over the course of this year we have run 6 design sprints, covering three topic areas: Rapid PREreviews, Community Features, and Search and Discovery. Each topic was covered at two different time zones to accommodate broad participation across multiple time zones. We have had a total of 90 participants across all sprints from around 20 countries. Participants were compensated with $100 for their time and expertise.

Alongside sprints we have also run 9 community calls, again at different times of day to accommodate participation from our global community. We had 13 participants to community calls.

We have run 18 Live Reviews, previously called Live Streamed Journal Clubs) collaborating with 7 different partners to run either a stand-alone Live Review or more commonly a series of Live Reviews, engaging over 135 community members to collaborate and review together to provide constructive feedback to preprint authors.

PREreview’s Open Reviewers training takes various forms ranging from 2-hour standalone workshops to a more in-depth 4-6 multi-session workshops. To date we have run 21 training interventions and worked with 12 partner organizations to provide peer review training to approximately 300 participants (this number is estimated as for some workshops the partner organization has managed registration). Alongside working with partner organizations to provide dedicated training for their cohort, we also have delivered 4 free-to-attend 2-hour community workshops including 35 participants from around the world.

Included in this figure of workshops is Open Peer Reviewers in Africa, tailored to the region-specific context of African researchers. This was a collaboration between PREreview, AfricArXiv, Eider Africa, eLife, and the Training Centre in Communication Africa (TCC Africa). Together we co-created and openly shared tools and strategies for scholarly literature evaluation, and trained the first cohort of 11 African researchers with 5 trainees moving on to be open peer review trainers themselves (https://content.prereview.org/open-peer-reviewers-in-africa-a-train-of-trainer-program).

In addition to Open Reviewers training for reviewers of manuscripts, we have also collaborated with the Open Research Funders Group & Health Research Alliance to run two Open Grant Reviewers training workshops with 34 participants representing 11 funders. This included developing a dedicated Open Grant Reviewers toolkit and a series of 7 short videos with bite-size information that are freely and openly accessible for the wider community.

Finally in addition to these activities, we have launched a dedicated PREreview Community Slack channel which now has 65 active members and growing each week. Members can provide feedback to PREreview via our #community-feedback channel, hear about our news and share their own via the #community-news channel, learn about upcoming events, contribute to the community newsletter, share a review or request one on their own preprint.

POSI evaluation in progress.

Policies & Governance

Governance Summary

PREreview is governed by an Advisory Committee, and is a fiscally sponsored project of Code for Science & Society, a US-based 501(c)3

Policies

Commitment to Equity & Inclusion

Implemented

Privacy Policy

Implemented

Web Accessibility Statement

In Progress

Governance Structure & Processes

Implemented

Transparent Pricing and Cost Expectations

In Progress

Policy Statements

Board Structure

  • Advisory board or steering committee

Community Governance

  • Ad hoc

Additional Information

Organizational History

PREreview was born in 2017 as an open science project, founded by three early-career women scientists all in volunteer capacity. From the start, we embraced a shared leadership approach, capitalizing on our diverse skills, experiences, and personalities to strengthen our team.

Our first grant in 2018 was a $2,500 Mozilla Foundation “mini-grant” which went to supporting refreshments for a number of in-person preprint review events.
In March 2019, Daniela Saderi transitioned to a full-time role as the project director, supported by the Mozilla Foundation as a fellow for open science.

PREreview also became a fiscally sponsored project of the non-profit organization Code for Science and Society, firmly rooted in a non-profit model to align with our values of openness and community prioritization over commercialization.

Initial funding from the Alfred P. Sloan Foundation and the Mozilla Foundation enabled the creation of the first prototype of PREreview.org. Subsequent support from the Wellcome Trust allowed us to develop a second iteration and introduce new tools for collaborative peer review.

From 2020 and 2022, we entered a collaboration with the non-profit journal eLife to support efforts to expand peer-review training to more communities. To date, we continue to partner with eLife to advance open evaluation of preprints. In 2022, a significant two-year grant from the Chan-Zuckerberg Initiative, in collaboration with eLife, led to the development of the best-ever third version of PREreview.org, offering accessible tools and resources while facilitating increased community engagement and input.

Organizational Structure

Business or Ownership Model

Fiscal sponsorship (non-profit)

Current Affiliations

  • Code for Science and Society (PREreview's fiscal sponsor)

Funding

Primary Funding Source

  • Contributions

Funding Needs

Investing in PREreview means supporting an independent, non-for-profit organization dedicated to transforming scholarly evaluation through the lenses of openness, equity and collaboration.


Founded in 2017 by three early-career women scientists, PREreview primarily relies on grants and has had only one full-time employee, the director, with a volunteer leadership team for the first 3 years. In 2022, thanks to investment from the Chan-Zuckerberg Initiative, we added three new team members who brought us to new highs. We now offer services like peer-review training workshops and collaborative review sessions to research institutions, funders, and publishers, moving from grants to a service-based model while staying true to their mission.
However, we can't fully sustain ourselves with service contracts alone. We believe we are at a turning point, with strong technology and community support. We are asking for support from organizations, governments, and others who share our values to help transform research evaluation, with a focus on marginalized groups in academia.



Such support will open up new opportunities for us to partner and build trust with like-minded organizations who are active in their communities. In the next 2-3 years, we plan to improve and scale 3 pillars that support researchers' journeys towards more open, fair, and collaborative peer review: 1) inclusive and accessible training in open review practices, 2) an authoring platform integrated with the peer-review ecosystem that supports rigorous and ethical open peer review, and 3) collaborative groups we call clubs that will allow users from diverse groups to continue building their own, decentralized communities of open peer-review practice supported by our training and platform.