09-04, 11:30–12:00 (UTC), Track 3 (Oteiza)
Testing research code can be difficult, but is essential for robust results. Using Hypothesis, a tool for property-based testing, I'll show how testing can be both easier and dramatically more powerful - even for complex "black box" codes.
Code is now a critical part of almost all research, whether for communication or
for data collection and analysis. Unfortunately, producing reliably error-free
code remains an open problem in science to an even greater extent than other
applications. Soergal (2014) estimates that "any reported scientific result could
very well be wrong if data have passed through a computer, and that these
errors may remain largely undetected." - though some software errors are
much more dramatic, as with the crash of the Mars Climate Orbiter.
What can we do to reduce the rate of errors in our own code? There is no silver
bullet, but a more efficient way to create tests would certainly help...
The answer is to have a computer write your tests for you! Using Hypothesis,
you describe valid inputs - from 'an integer' to 'dataframes like this', as
complex and precise as needed - and write a test which should always pass...
then Hypothesis searches for the smallest inputs that cause an error.
This approach is called property based testing, and it regularly catches errors
that evaded every human review and hand-written test case (even in Numpy).
Even better, it rewards well-designed software - but can also do a quick check
of a script in just a few lines of code.
We'll cover the theory of property-based testing, a worked example, and then
jump into a whirlwind tour of the Hypothesis API: how to use, define, compose,
and infer strategies for input; properties and testing tactics for your code; and
how to debug your tests if everything seems to go wrong.
By the end of this talk, you'll be ready to find real bugs with Hypothesis in
anything from data pipelines to the core scientific Python libraries. Be the
change you want to see in your team's code - or test someone else's and help
push the world into a new age of reliable research software!
Testing research code can be difficult, but is essential for robust results. Using Hypothesis, a tool for property-based testing, I'll show how testing can be both easier and dramatically more powerful - even for complex "black box" codes.
Python Skill Level –professional
Domain Expertise –some
Domains –General-purpose Python
Zac is a researcher at the Australian National University’s 3A Institute, which is building a new applied science to 'manage the machines' - AI, cyber-physical systems, and other new technologies.
He started using Python to analyse huge environmental datasets, and contributing to libraries like Xarray to make such analysis easier for all scientists. Now, as a maintainer of Hypothesis, Pytest, and Trio, Zac is still passionate about making it easy to write software you can understand and rely on.
When not at a computer he can usually be found surrounded by books of all kinds, the Australian bush, or both.