Automated hypervisor testing and benchmarking on hardware
Automated testing is a very important tool in modern software engineering, and is often implemented through runners in virtualized environments. Low-level domains, such as kernel development, have unique requirements that are not covered in these environments, but automation of all different kinds of hardware comes with additional challenges. In this talk, we will deep-dive into some of the challenges we encountered during development and use of SoTest, a custom-made service for automatically testing and benchmarking virtualization workloads on a broad variety of hardware, including:
-
How can we make arbitrary hardware remote-controllable?
-
How do we integrate these tests into the developer workflow?
-
How can we optimize the throughput?
-
How do we enable performance benchmarking?
-
How do we handle, e.g., network infrastructure flakiness?