2024-11-16 –, LT9
Language: English
Although Python is popular and known for rapid development and great ecosystem, performance may often be an issue when running certain types of tasks. For example, we ran into performance issues when we had to parse and chunk CSV files of arbitrary sizes.
Utilising C for core functions and a wrapper with Cython, we were able to quickly build a lightweight Python module that improved the performance of our use case!
Through this talk, we'd demonstrate how tools like Cython could make Python even more powerful.
Leo Chen is a software engineer in Bloomberg's Data Technologies Infrastructure Engineering group, where he works on data ETL pipelines and workflow orchestration platforms. Over the course of his seven-year career, he has experience with multiple programming languages, machine learning frameworks, and both frontend & backend development. He is a big Python enthusiast. Leo holds a master's degree in data science from the University of Sydney.