2024-09-26 –, Gaston Berger
MAPIE (Model Agnostic Prediction Interval Estimator) is your go-to solution for managing uncertainties and risks in machine learning models. This Python library, nestled within scikit-learn-contrib, offers a way to calculate prediction intervals with controlled coverage rates for regression, classification, and even time series analysis. But it doesn't stop there - MAPIE can also be used to handle more complex tasks like multi-label classification and semantic segmentation in computer vision, ensuring probabilistic guarantees on crucial metrics like recall and precision. MAPIE can be integrated with any model - whether it's scikit-learn, TensorFlow, or PyTorch. Join us as we delve into the world of conformal predictions and how to quickly manage your uncertainties using MAPIE.
Link to Github: https://github.com/scikit-learn-contrib/MAPIE
This talk introduces MAPIE, an open-source Python library designed to quantify uncertainties and control risks in machine learning models. Learn how to compute conformal prediction intervals and control risks in various tasks such as regression, classification, time series, and even complex tasks like multi-label classification and semantic segmentation.
We will begin by discussing the importance of uncertainty quantification and risk control in machine learning models. Then, we will dive into the key features of MAPIE, including:
- Computing conformal prediction intervals for regression, classification, and time series tasks with guaranteed marginal coverage rates.
- Controlling risks for complex tasks such as multi-label classification and semantic segmentation in computer vision.
- Wrapping any machine learning model (scikit-learn, TensorFlow, PyTorch, etc.) with a scikit-learn-compatible wrapper for uncertainty quantification and risk control.
Throughout the talk, we will demonstrate MAPIE's capabilities with practical examples and code snippets. Attendees will learn how to apply MAPIE to their own models, ensuring more reliable and robust predictions.
This talk targets data scientists, machine learning engineers, and researchers with a basic understanding of machine learning concepts. Familiarity with scikit-learn and other popular machine learning libraries is helpful but not required.
By the end of the talk, attendees will have gained valuable insights into uncertainty quantification and risk control, as well as hands-on experience using MAPIE to bring uncertainty quantification to their machine learning models.
Approximate time breakdown:
0-3 minutes: Why perform uncertainty quantification?
3-15 minutes: Quick dive into the conformal prediction intervals for regression and classification.
15-20 minutes: Overview of a real-world application + Interactive demo of MAPIE.
20-30 minutes: Q&A and discussion.
Louis Lacombe, Data Scientist at Capgemini Invent, France.
Louis Lacombe, a graduate of the MSc program at the University of Bocconi, joined Quantmetry (now Capgemini Invent) in 2022. He began his journey in conformal predictions with the implementation of conformalized quantile regression in the MAPIE library.
Currently, he is one of the core developers of the library and focuses on enhancing its visibility by demonstrating diverse use cases and by integrating conformal predictions into clients' algorithms and advocating for a trusted AI approach.
Thibault Cordier is a Data and Research Scientist at Capgemini Invent, where he is a member of the Lab Invent team in France and serves as the technical leader of the MAPIE project.
Prior to joining the research team at Capgemini Invent, he earned his PhD in Computer Science in 2023 at Avignon University.
Up to now, his research has focused on distribution-free inference and conformal prediction, with applications in computer vision, natural language processing, and time series analysis.
Senior Data Scientist @ Capgemini Invent
Getting onboard to lead the team behind MAPIE, an open-source library within the sklearn-contrib ecosystem, focused on conformal predictions.
After earning a MSc in Computer Science from École Centrale, I spent a few years in product management before returning to more technical roles.
Let’s connect!