When we started building ScramDB, we knew that the heart of any database system is its query execution engine. Rather than following a traditional interpreter approach, we designed ScramVM - a bytecode-based execution engine written in Rust that prioritizes throughput and CPU utilization.
ScramVM transforms SQL queries through multiple compilation stages: parsing, logical planning, physical planning, and finally bytecode generation. This approach gives us two major advantages. First, we can cache the compiled bytecode, dramatically speeding up repeated queries. Second, we can optimize execution at multiple levels, from query planning to the way we schedule work across CPU cores.
The most interesting challenge we tackled was parallelization. We implemented a morsel-driven execution model where the system breaks data into right-sized chunks ("morsels") and distributes them across worker threads. Our custom thread pool pins workers to specific CPU cores for better cache locality and NUMA awareness. This approach let us maximize hardware utilization without the complexity of managing distributed state across nodes.
One unexpected lesson was the importance of balancing abstraction with performance. For our CPU-bound query execution, we found that a specialized approach with careful thread management gave us better performance characteristics. We took our time testing various synchronization approaches and ultimately settled on an architecture where a coordinator dispatches work to workers via a custom scheduler that maintains awareness of the underlying hardware.
I'll share our experience with real-world queries, showing how these design decisions impact performance across different workload types. I'll also discuss the hardest bugs we encountered (stack management issues, anyone???) and how Rust's safety guarantees helped us avoid entire classes of common database engine errors while still allowing the low-level control we needed for optimal performance.