PyCon DE & PyData 2025

How Narwhals is silently bringing pandas, Polars, DuckDB, PyArrow, and more together
2025-04-25 , Zeiss Plenary (Spectrum)

If you were writing a data science tool in 2015, you'd have ensured it supported pandas and then called it a day.

But it's not 2015 anymore, we've fast-forwarded to 2025. If you write a tool which only supports pandas, users will demand support for Polars, PyArrow, DuckDB, and so many other libraries that you'll feel like giving up.

Learn about how Narwhals allows you to write dataframe-agnostic tools which can support all of the above, with zero dependencies, low overhead, static typing, and strong backwards-compatibility promises!


Suppose you want to write a data science tool to do feature engineering. Your experience may go like this:
- Expectation: you can focus on state-of-the art techniques for feature engineering.
- Reality: you keep having to make you codebase more complex because a new dataframe library has come out and users are demanding support for it.

Or rather, it might have gone like that in the pre-Narwhals era. Because now, you can focus on solving the problems which your tool set out to do, and let Narwhals handle the subtle differences between different kinds of dataframe inputs!

Narwhals is a lightweight and extensible compatibility layer between dataframe libraries. It is already used by several open source libraries including Altair, Marimo, Plotly, Scikit-lego, Vegafusion, and more. You will learn how to use Narwhals to build dataframe-agnostic tools.

This is a technical talk aimed at tool-builders. You'll be expected to be familiar with Python and dataframes. We will cover:
- 2-3 minutes: motivation. Why are there so many dataframe libraries?
- 2-3: minutes: life before vs after Narwhals - real-world examples of how the data landscape is changing
- 7-8 minutes: basics of Narwhals, wrapping native objects, expressions vs Series, lazy vs eager
- 7-8 minutes: advanced Narwhals concepts: row order, non-elementary group-by aggregations, multi-indices, null values, backwards-compatibility promises
- 2-3 minutes: what comes next?
- 5 minutes: engaging Q&A / awkward silence

Tool builders will benefit from the talk by learning how to build tools for modern dataframe libraries without sacrificing support for foundational classic libraries such as pandas.


Expected audience expertise: Domain:

Advanced

Expected audience expertise: Python:

Advanced

Public link to supporting material, e.g. videos, Github, etc.:

https://github.com/narwhals-dev/narwhals