2026-05-07 –, Track 1
Piloting—the preliminary testing of a study before full data collection—is integral to psychological research, yet these practices are rarely discussed and lack concrete guidance. Research methods textbooks seldom cover piloting, and when they do, it is often reduced to a brief footnote. Consequently, current techniques are tied to individual training and perceived field norms.
Piloting varies considerably within and across workflows (Pilot Reporting Task Force, 2024): researchers use different approaches, may understand "piloting" differently, and some are unaware that others pre-test their methods at all. This variability is not problematic in itself; rather, because so little attention is paid to these methods, we cannot fully learn from or develop them.
We therefore propose creating a living, open-access handbook to make piloting more visible, learnable, improvable, and valuable across psychological subfields. In this unconference, we aim to brainstorm how such a resource could be developed.
Mary Beth Neff, Agata Bochynska, Isaac Handley-Miner, Yashvin Seetahul, Priya Silverstein
Please classify your session as the theme it fits best in:: Pedagogy/Curriculum/Mentoring - Content related to educating students How will the session's content foster diversity & inclusion (e.g., who will present, who will it serve), and how will it improve psychological science?:A key goal of this handbook is to ensure it reflects the full breadth of psychological subfields, rather than defaulting to methods suited only to particular lab setups or well-resourced research contexts. Piloting needs also differ considerably across subfields and, without capturing this diversity, more visible areas risk inadvertently setting inappropriate standards for others.
Our working group already spans multiple countries, career stages, and diverse subfields across academia and industry. However, we recognize this is not sufficient: the handbook must reflect a wide range of voices to be genuinely useful across psychology. The unconference is therefore designed to broaden this knowledge base, and we welcome anyone committed to enhancing diversity and promoting transparent reporting of pilot studies!
Please note any pre-requisite knowledge/expertise you will expect from attendees (i.e., is the session most appropriate for someone who already has experience with a topic?).:No prior experience is needed
Mary Beth Neff is a (contract) Associate Professor at the University of Oslo, Norway and lead of the Pilot Reporting Task Force. Her research focuses broadly on method development, alongside more domain-specific work on developmental pragmatics, theory of mind, and common ground reasoning
I am a researcher and a research advisor focusing on implementing open research and reproducibility practices across disciplines at University of Oslo. My background is in cognitive psychology and linguistics with a special focus on the relationship between language and cognitive abilities in children and adults, including developmental conditions like autism. My current interest is also in meta-scientific assessments and evaluating implementation of open research.