Keynote Speaker: Repeatability: Redoing Research for Reproducibility, Robustness, and Replicability Reveals Reliability and Rigor Risks
Brian Nosek
Science progresses, in part, by identifying regularities in nature. Understanding the conditions under which a regularity recurs is a step toward predicting and explaining why it occurs. Challenges in establishing repeatability may be more pronounced than recognized when moving from the narrow case of reproducibility (applying the same analysis to the same data) to robustness (testing the same question with the same data using alternative analyses) and then to the broader concept of replicability (testing the same question with new data). Attempting to repeat evidence from a sample of papers from the social and behavioral sciences provides insight into the multidimensionality of repeatability as a fundamental concept for research progress.