2024-09-26 –, Gaston Berger
Adaptive prediction intervals, which represent prediction uncertainty, are crucial for practitioners involved in decision-making. Having an adaptivity feature is challenging yet essential, as an uncertainty measure must reflect the model's confidence for each observation. Attendees will learn about state-of-the-art algorithms for constructing adaptive prediction intervals, which is an active area of research.
Proper prediction uncertainty estimation is critical, particularly when data-driven decision-making is in place. Point predictions alone are insufficient as they only represent the conditional expectation of a target variable. Therefore, to make well-informed decisions, one has to quantify the variance of probability distribution associated with a point prediction. Prediction intervals provide this valuable information.
Conformal prediction is a powerful distribution-free framework allowing to construct prediction intervals with valid marginal coverage. However, in real-life scenarios, predictive model performance is not homogeneous; there are regions in the input space where it performs better, leading to smaller prediction uncertainties that should be mapped to narrower prediction intervals. This capability to adapt interval lengths based on local performance is called adaptivity.
In this talk, targeted towards data scientists who develop and deploy predictive models, we review and benchmark algorithms for constructing adaptive prediction intervals. The key takeaway for the audience will be learning about the algorithms for building adaptive prediction intervals and understanding how to apply them in their own work.
Andro Sabashvili transitioned to a data science career after obtaining a PhD in theoretical physics. Andro's data science journey commenced at Bank of Georgia, where he sharpened his skills in developing end-to-end machine learning projects, such as credit risk, propensity, and churn models using advanced machine learning algorithms. After designing a novel algorithm to automate the hyperparameter optimization process, which is currently patent-pending, he joined the startup company Dressler Consulting, where he led a team of data scientists building an automated machine learning platform. In his current role as a Staff Data Scientist at Majid Al Futtaim, Andro is focused on scaling up sales forecasting for thousands of stores, marketing mix modeling, marketing campaign impact estimation, and customer segmentation.