Rayan Daod
Currently a Machine Learning Engineer at Atinary Technologies in Lausanne, Switzerland. Redesigned the training pipeline and introduced an inference infrastructure for the SDLabs platform. Now part of the research team to improve our bayesian optimization and transfer learning algorithms.
Before, I worked as an ML Research Intern at Bose and Logitech on lightweight, causal speech enhancement and source separation under strict latency and memory constraints.
Before that I obtained a MSc in Computer Science from EPFL, with focus on Machine Learning and Digital Signal Processing.
I travel and produce electronic music during my free time!
rayan.daodnathoo.com
Session
Serving user-specific ML models at scale while satisfying low-latency requirements is non-trivial, especially when the model training code is regularly updated, making models trained weeks apart require potentially incompatible environments. How do you build a scalable inference pipeline with per-user models while still moving fast, when the runtime keeps evolving? In this talk, I’ll show how our small team built and maintains a Python inference pipeline that can serve thousands of user-specific models from a single service using a dynamic model loading approach. There is still one challenge to tackle: enabling training code and dependency update without risking incompatible inference environments. We’ll compare different strategies and explain the solution we chose to stay agile while preserving a smooth UX and DevEx. Finally, we will walk through the end-to-end architecture built using open source tools such as MLflow, MLServer, KServe, and supporting both local development and production-scale deployments.