2026-06-06 –, Grand Hall 1
When building high-performance systems for analytical workload, we often focus on the efficiency of the algorithm, like reducing Big-O complexity or optimising numerical routines. Yet in real world workloads, the decisive factor is not just the algorithm but the shape of how the data is laid out, traversed, and distributed across processes.
This talk will cover aspects of mechanical sympathy, focussing on how structures in memory can benefit from cache-sensitive, SIMD-enabled (vector instructions) CPUs, constrained by memory bandwidth and optimised for predictable, contiguous access.
We will use real-world examples to show how minimising serialisation overhead and enabling efficient cross-process and cross-language data exchange reduces the cost of data movement across systems. Beyond single-system performance, we will examine why Arrow’s standardised, zero-copy columnar format is a critical enabler of distributed execution. We will see how columnar formats support scalable computation across threads, processes, and distributed nodes.
Everyday production-scale data and systems engineering still reflects a row-oriented mental model. Loops, iterations, mutations are seen as easy to read and are understandable. While these work for small datasets and toy models during explorations in notebooks, they fail to perform when workloads scale - be it for rolling analytics, high-throughput pipelines or multi-million row aggregations. This mismatch between row-wise thinking and modern CPU architecture becomes a structural bottleneck that becomes very costly to fix.
We’ll explore the shift from row-oriented design to columnar thinking, designing and developing high-performance workloads right from the onset. Using Arrow’s columnar memory format and Polars’ execution engine, armed with concrete examples from real-life quantitative calculations, we will examine how contiguous buffers, SIMD-compatible layouts, and lazy query planning are a natural combination for performant analytical workloads.
You’ll leave with:
1. A clear understanding of how columnar memory impacts execution, in contrast to row-oriented or traditional vectorised approaches.
2. Practical patterns for structuring column-first transformations.
3. Insights into how Arrow reduces data movement overhead in distributed systems.
4. Guidance on when lazy execution and query optimisation matters.
5. Ideal design principles for building scalable calculation pipelines with Polars and Arrow tools.
I am a senior engineering lead/executive director at Morgan Stanley.
I design and build large-scale, enterprise-ready, high-performance financial systems used in production environments where correctness, resilience, and speed matter. My work spans system design, hands-on engineering, and long-term platform evolution in regulated domains.
I place strong emphasis on clean, maintainable architecture—clear domain boundaries, explicit data contracts, and model-driven design. I optimise for systems that remain understandable and adaptable as complexity, scale, and regulatory demands increase.
A significant part of my work focuses on data analytics, complex data modelling, and financial mathematics—including forecasting, liquidity, risk, and regulatory calculations. I enjoy translating mathematically rich problem spaces and large datasets into precise, explainable, and production-grade implementations.
I work with a prototype-to-production mindset, leveraging modern cloud platforms, data tooling, and AI techniques to move quickly while preserving architectural discipline, observability, and operational robustness.