We love Polars because it is written in Rust so we can use Rust's security and speed. However, it is not the most efficient if we still have to call in a Python function to perform specific aggregation. In this workshop, we will use the Polars plugin. You will be writing simple functions in Rust, and then you will use it together with Polars in your Python data pipeline.

Target Audience

Engineers and data scientists who use Polars and are confident to write a bit of Rust code. We expect you to have knowledge of Python and Polars and have a bit of Rust experience (or be able to pick it up relatively quickly). Not all concepts in Rust will be explained but we will link to material where you can find explanations.

Goal

To empower Polars users who want to do more and do better with Polars. For folks who don't mind learning a new programming language, it is also a good opportunity to learn and practice writing in Rust.

Preflight check

In this workshop, we expect you to have knowledge of Python and Polars and have a bit of Rust experience (or be able to pick it up relatively quickly). Not all concepts in Rust will be explained but we will link to material where you can find explanations.

Here are the things that you should have installed when you started this workshop:

• Install/ Update Rust(we are using rustc version 1.86.0 here)
• Make sure having Python 3.9 or above (assuming 3.13 in this workshop)
• Make sure using virtual environment (recommend using uv >= 0.4.25)

Windows checklist

In this workshop we recommend using Unix OS (Mac or Linux). If you use Windows, you may encounter problems with Rust and Maturin. To minimise issues that you may encounter, please go through the extra checklist below:

• Install the c++ build tools
• Check the dll files are linked correctly

Learning resources for Rust and PyO3

To wirte a Polars plugin, you will have to develop in Rust. If you are not familiar with Rust, we highly recommend you first check out some of the Rust learning resources so you can be prepare for the workshop. Here are some of our recommendations:

• The Rust Book
• Rustlings (Exerciese in Rust)
• Rust by Example
• Teach-rs (GitHub repo)

Another tool that we will be using will be PyO3 and Maturin. To learn more about them, please check out the following:

• The PyO3 user guide
• PyO3 101 - Writing Python modules in Rust

Setting up

1. create a new working directory

mkdir polars-plugin-101
cd polars-plugin-101

1. Set up virtual environment and activate it

uv venv .venv
source .venv/bin/activate
python -m ensurepip --default-pip

Note: the last command is needed as maturin develop cannot find pip otherwise

1. Install polars and maturin

uv pip install polars maturin

These are the versions that we are using here:

• maturin==1.8.3
• polars==1.27.1

Workshop materials: https://github.com/Cheukting/polars_plugin_101

Outline

• Introduction (15 mins):
  1. What is Polars plugin
  2. How does it work (using Maturin to develop packages)
  3. How to use it with Polars (exercises)
• Simple numerical functions (35 mins):
  1. Creating numerical functions with 1 input (exercise)
  2. Creating numerical functions with multiple inputs in the same row (exercise)
  3. Creating numerical functions that support multiple types (exercise)
• Advance usage with Polars plugin (40 mins):
  1. Creating functions with multiple inputs across different rows (exercise)
  2. Functions with user-set parameters (exercise)
  3. Working with strings and lists (exercise)