PyCon LT 2023

Polars: done the fast, now the scale
2023-05-18 , Saphire B - PyData

DataFrame abstractions are one of the favorite data structures of many data scientists, data-engineers and programmers in general. They offer flexibility and intuitive reasoning on top of query processing.

However, the implementation of DataFrame abstractions have been lacking. On the single node they have been ignoring most research available in RDBMS research. Different from RDBMS, the most known python implementations don't control their own query engines, and are therefore always compromising control, performance and memory usage.

Polars is a DataFrame library that brings a very fast OLAP query engine to the DataFrame abstraction.

This talk we look at what polars has achieved since it's inception and what the future will hold in store.

<some extra characters because they were needed to fill the cell>


What is a level of your talk:

Intermediate

What topics define your talk the best?:

python, open source, PyData, optimization and speed, data science, machine learning, ML engineering, data engineering

Ritchie Vink is the author of polars. He has a background in machine learning/ data engineering and software engineering.

This speaker also appears in: