Taking Flight: Zero-Copy Data Transfer at Scale with Apache Arrow Flight and Friends
2026-09-12 –, Room A All times in Europe/AmsterdamWe all love Apache Arrow in our DataFrames and DuckDBs - but have you considered how the data is ending up on your machine?
Arrow Flight and friends was designed as a way of enabling end-to-end native Arrow communication and in this tutorial, we will go through the motivations behind the Arrow Flight protocol, introduce some of the derived protocols, and we end by building an Arrow Flight server from scratch and demonstrating how to interact with that server from a client.
This talk is for you if you've ever wondered how to implement high-speed data transfer for your data analytics.
In this talk, you will see why Arrow Flight makes building infrastructure to serve large amounts of data feasible. I will go through a brief breakdown of what Arrow is, and what benefits it brings before moving on to looking at the Arrow Flight specification.
By going through each of the components that makes up the Arrow Flight specification, you'll be able to start building your own Arrow flight server. I will take you through benchmarking Arrow Flight vs the classic REST Api and show the massive speed increases. I will go through how to build your own Arrow Flight server and demonstrate how easy it is to build a Client.
At the end, you'll have an understanding of what purpose Arrow Flight serves in your stack, and if you should take advantage of it.
Anders is the Head of Investments Engineering at Nordea Asset Management and organizer of Pydata Copenhagen Meetup. He has a background as a ML Tech Lead and Python Enabler with an interest in data engineering, ML and ML Engineering. Hailing from Stavanger, Norway, he is currently located in Copenhagen, Denmark