2025-10-30 –, Main Hall
Statistical analysis can be a key tool for upping your security game, but there’s a lot of ways to end up with statistics-shaped outputs rather than useful information to support your work. In this talk a security expert (Kate Pearce) and a scientist (Lucy Stewart) team up to explain, with real-life examples, when and how statistics can help you figure out if that shiver down your spine is something to be concerned about – and when your priors are, actually, out to get you.
This talk digs into the practical realities of using data in security. We'll start by breaking down why "risk = probability x impact" doesn't mean what you think it means, how to define what you're even measuring (e.g., "attacks prevented"), and how your field of view can create dangerous false confidence.
We'll explore classic statistical traps with real-world security parallels—from the "average number of legs" to misleading metrics about medieval life expectancy—and show how to spot them in vendor reports and your own dashboards. You'll learn the importance of your null hypothesis (i.e., how to test if you’re wrong), and why in the real world of security, false positives can matter even more than false negatives.
This is a guide to using statistical tools as one part of a decision-making process, not as a machine for justifying what you already believe. We’ll ask: are you deciding, or just justifying? Is "data-driven" a lie? (Spoiler: mostly). You'll leave knowing how to communicate results honestly, how much effort to really put in, and when to ditch the complex models and trust your somewhat-informed vibes.
Its a super duo on stage get ready. They gave the first ever talk at Chcon in 2016, and this year they’re back!
Kate Pearce is the Head of Security at Trade Me. She is a recovered pentester and security conference speaker who has presented on network protocol abuses, didn't talk about gaslighting for security defence, and is rumored to have made a government approved TTRPG.
Dr Lucy Stewart is an environmental microbiologist, union organiser, and the current co-President of the New Zealand Association of Scientists. Sometimes she speaks at security conferences just to keep things interesting.