Daniel Finnan
Daniel Finnan is a 2nd year PhD candidate at the Lirsa laboratory, Conservatoire national des arts et métiers (CNAM), in Paris. His thesis focuses on decentralized finance, specifically decentralized exchanges, applying a quantitative methodology using blockchain data, techniques in data science, and time series econometrics. He codes in Python, R, and occasionally Rust and JavaScript, specifically using Python to manage data pipelines. He has a professional certification in full-stack development and holds a Master’s degree in Economics, with a specialization in Economic, Digital and Data strategies from CNAM’s department of Economics, Finance, Insurance and Banking.
Session
Do you find yourself weighing up the pros and cons of using nested types in the Polars library - pondering whether you should encode your variables in structures using lists, arrays or opt for a flat format without complex hierarchy? This talk focuses on the crucial design choices available, the performance implications, and how this impacts the logic of your queries, as well as code readability, when deciding how to implement your big data pipeline in Polars. The methods available for nested types in Polars have seen some significant additions over the last year, with powerful functionality, such as filtering and aggregation, released in the latest versions of the library. These provide much-needed shortcuts for queries interrogating complex nested structures that previously required sophisticated user-defined functions. It makes the use of nested types much easier and intuitive, but does this mean you should nest your data? Through practical examples you’ll learn some guidelines to help you decide.