JuliaCon 2025

Base Julia provides mostly low-level multi-threading and synchronization primitives -- task spawn, locks, conditions, etc. In building our knowledge graph application at RAI, we have found the need for higher level abstractions to build parallelism into our processing pipeline. In this lightning talk, we will describe TaskGroups: why we need them and what you can do with them.

Julia tasks are lightweight threads that Julia schedules MxN on OS threads. Tasks are cooperatively scheduled: there is no preemption; switches typically occur when waiting for events. In such a system, fairness is difficult to achieve--tasks can hog CPU time and starve other tasks.

We will describe the consequences of a lack of fairness in a real world system--RAI's knowledge graph system. We show how we were able to improve and discuss the implications for future work on Julia's scheduler.