SIPS 2025 Online

oLT4: Metavaluation: An open, fair, and inclusive mechanism for valuing diverse scholarly contributions
2025-05-21 , Track 1 (Wed)

Metavaluation is a novel mechanism designed from first principles to overcome the collective action problem in academia by offering direct rewards for diverse contributions, including peer-reviews themselves. In this talk, I’ll give a brief overview of the framework, highlighting the key innovation: the use of pairwise comparisons as both an inclusive review protocol and a standard unit of value, serving to scale the relative value of all other contributions in a decentralised and autonomous manner. I’ll share data from several communities, within academia and beyond, as a testament to the models capacity to adapt to diverse contexts and foster coordination through interoperable metrics. I’ll close with a vision for how the model could connect diverse communities in the research space and beyond, fostering a virtuous cycle of value creation, before inviting participants to join us in a hackathon applying the framework to SIPS itself.


Landing page:

https://docs.google.com/document/d/1Xv9aFfpvPr04_IJMq2SR7B36lx4iordI/edit?usp=sharing&ouid=102720080386074581271&rtpof=true&sd=true