Preserving VFIO PCI Devices During Kernel Live Updates
Typically, updating a host kernel requires live-migrating virtual machines (VMs) to other hosts. However, this approach isn't feasible for VMs that rely on GPUs or for large-scale Language Model (LLM) training clusters spread across numerous hosts, where migration is complex and disruptive.
To address this challenge, Google is developing a Live Update mechanism [1]. This feature allows devices assigned to VMs or the Virtual Machine Monitor (VMM) via VFIO (Virtual Function I/O) to remain operational even as the host transitions to a new kernel using Kexec.
VFIO PCI device preservation is the key enabling technology here. It ensures that a PCI device can continue its direct memory access (DMA) and interrupt operations without being reset while the host kernel undergoes a Kexec-based update. Achieving this requires significant modifications to the VFIO, IOMMU (Input/Output Memory Management Unit), and PCI subsystems.
This talk will delve into Google's approach to preserving VFIO PCI devices during live kernel updates and the challenges encountered during its development.
[1] https://lore.kernel.org/lkml/20250515182322.117840-1-pasha.tatashin@soleen.com/