KVM Forum 2025

Exploring VM placement strategies for chiplet architectures
2025-09-05 , Room 1

Modern processors are increasingly adopting chiplet-based architectures that distribute CPU cores across multiple chiplets, each containing one or more core complexes (CCX) with a shared last-level cache. Inter-chiplet communication penalties can significantly degrade workload performance. While the current Linux scheduler has NUMA awareness and NUCA (non-uniform cache access) awareness through its scheduling domain hierarchy, it lacks adequate consideration for the significantly higher inter-CCX communication penalties inherent in chiplet architectures. This leads to suboptimal VM placement and therefore, degraded performance.

This talk proposes an enhanced VM scheduling framework designed for chiplet-based processors.
1. The framework utilizes lightweight monitoring techniques, such as hardware performance counters, to monitor cache efficiency, memory access patterns and inter-chiplet communication metrics. These insights help formulate informed policies for VM placement and vCPU group migrations.
2. The framework implements intelligent VM vCPU group placement strategies that optimize the initial allocation of vCPU groups and their associated contexts, such as vhost and other datapath threads, across chiplet boundaries. The algorithm balances maximizing chiplet locality while minimizing intra-cache contention. The algorithm also studies the dynamic behaviour of VM placements in overloaded cases.
3. These performance measurements guide the assignment of optimal virtual topologies to guests, improving performance through chiplet-locality-aware decisions starting at the guest scheduler level.

I am a Software Engineer on the AHV team at Nutanix. I mainly work in the virtualization domain, with a focus on NUMA systems, exposing the correct NUMA topology and improving vNUMA performance in virtualized environments. I am passionate about bridging the gap between hardware capabilities and software optimization to deliver measurable performance improvements for enterprise virtualization workloads.

I am a software engineer working for the hypervisor team in Nutanix with interests in areas of virtualization, performance and live migration. Apart from this I also like to explore new platform technologies.

I work for the AHV team at Nutanix where most of my work to date is around improving live migration algorithms. Other than live migration, my interests lie in systems and virtualization in general.

HyperVisor Engineer@Nutanix

Hypervisor Engineer @ Nutanix