Thijs Nieuwdorp
Thijs Nieuwdorp is the DevRel Engineer at Polars in Amsterdam. His interest in the interaction between human and computer led him to an education in Artificial Intelligence at the Radboud University, after which he dove straight into the field of Data Science. At Xomnia he witnessed the birth of Polars as Ritchie Vink started working on it during his employment there , and has been using it in his projects ever since. Outside work Thijs enjoys exploring our world through hiking and traveling, and exploring other worlds through books, games, and movies.
Sessions
For data engineers and data scientists building Python pipelines workloads can hit the awkward middle: too large to handle comfortably with pandas on a single machine, too small for a Spark cluster to justify the operational cost. The usual choice is overprovisioned single machines on one end, or accept cluster maintenance and a second DataFrame API.
This talk argues that you don't need to choose. Polars' new streaming engine is the common core behind high performance single node processing with spill-to-disk capabilities, and distributed execution. The same API scales with your data.
Attendees will leave with actionable insights that help decide: when vertical scaling and spill-to-disk are enough (which gets you further than most assume), when distributed computing is actually justified, and technical knowledge of how Polars accomplishes both.
We'll follow one query across different dataset sizes, go through the technical challenges we overcame, and teach you to decide your optimal choice for each scenario.
What is an open source sprint?
An open source sprint is an event where programmers come together to make focused contributions to an open source project within a limited timeframe. Participants can work individually or in groups, collaborate directly on coding tasks, fix bugs, and develop new features for the project. The sprint environment offers hands-on coding, mentorship from experienced contributors, and fosters networking and community building among peers. Through time-boxed sessions, programmers can learn from each other, solve problems collaboratively, and quickly advance project goals in a welcoming, supportive setting.
Why should you join?
- Gain practical coding experience on real-world projects
- Receive guidance and mentorship from skilled contributors
- Make a meaningful impact on an open source project
- Expand your professional network and meet like-minded peers
- Learn best practices and new technologies in a supportive environment
Specifically: A hands-on sprint for people who use Polars and want to go one layer deeper. Pick one of two tracks: work on a "good first issue" from the Polars issue tracker, or build an expression plugin that adds your own custom functionality to Polars, running at native speed. You leave with either an open PR or a working plugin, and a setup you can keep using afterwards.
What is an open source sprint?
An open source sprint is an event where programmers come together to make focused contributions to an open source project within a limited timeframe. Participants can work individually or in groups, collaborate directly on coding tasks, fix bugs, and develop new features for the project. The sprint environment offers hands-on coding, mentorship from experienced contributors, and fosters networking and community building among peers. Through time-boxed sessions, programmers can learn from each other, solve problems collaboratively, and quickly advance project goals in a welcoming, supportive setting.
Why should you join?
- Gain practical coding experience on real-world projects
- Receive guidance and mentorship from skilled contributors
- Make a meaningful impact on an open source project
- Expand your professional network and meet like-minded peers
- Learn best practices and new technologies in a supportive environment
Specifically: A hands-on sprint for people who use Polars and want to go one layer deeper. Pick one of two tracks: work on a "good first issue" from the Polars issue tracker, or build an expression plugin that adds your own custom functionality to Polars, running at native speed. You leave with either an open PR or a working plugin, and a setup you can keep using afterwards.