Cooper Smout
Sessions
Metavaluation is a novel mechanism designed from first principles to overcome the collective action problem in academia by offering direct rewards for diverse contributions, including peer-reviews themselves. In this talk, I’ll give a brief overview of the framework, highlighting the key innovation: the use of pairwise comparisons as both an inclusive review protocol and a standard unit of value, serving to scale the relative value of all other contributions in a decentralised and autonomous manner. I’ll share data from several communities, within academia and beyond, as a testament to the models capacity to adapt to diverse contexts and foster coordination through interoperable metrics. I’ll close with a vision for how the model could connect diverse communities in the research space and beyond, fostering a virtuous cycle of value creation, before inviting participants to join us in a hackathon applying the framework to SIPS itself.
Metavaluation is a novel mechanism designed from first principles to address the collective action problem in academia by offering direct rewards for diverse contributions—including peer reviews. The Metavaluation app, developed by the Open Heart Mind (OHM) community, is a free, open-source tool created to demonstrate the model and make it accessible to a wide range of communities.
In this hackathon, participants will use the Metavaluation app to nominate SIPS contributions and determine their relative value through an inclusive peer-review process. Every participant will have the opportunity to vote for their preferred contributions and be rewarded for their input. By the end of the session, we will generate a set of multidimensional scores that reflect the unique qualities of each contribution—including the evaluations themselves. This dataset may serve as a replicable template for future investigations, potentially fostering a virtuous cycle of value creation within the SIPS community.