Barak Amar
Barak Amar is a principal engineer at lakeFS with over 25 years of experience spanning startups and enterprise environments. He specializes in distributed systems and backend architecture, designing scalable solutions. He is passionate about programming languages and contributes to open-source projects.
As part of the founding team at lakeFS, he has helped build the product while recently also gaining experience in product management.
When not on the keyboard, he is an avid runner who maintains a regular training schedule.
Session
The cloud native revolution has impacted all aspects of engineering, and data engineering is not exempt. One of the ongoing challenges in the data engineering world remains the local and distributed cloud native storage. In this talk we’ll explore working with distributed file systems in Python, through an intro to fsspec: a popular python library that is well-positioned to address the growing challenge of interacting with storage systems of different kinds in a consistent way.
In this talk we’ll show hands-on examples of working with fsspec with some of the most popular data tools in the Python community: Pandas, Tensorflow and PyArrow. We’ll demonstrate a real world implementation of fsspec and how it provides easy extensibility through open source tooling.
You’ll come away from this session with a better understanding for how to implement and extend fsspec to work with different cloud native storage systems.