Daria Mustafina
I'm a Senior Data Scientist at Eneco, one of the Netherlands' leading energy companies, where I build large-scale forecasting systems that predict the energy demand and generation of millions of customers across the country. My work sits at the intersection of applied machine learning and the energy transition — turning smart meter and portfolio data into forecasts that keep the grid balanced and support the shift to renewable energy. I'm passionate about building scalable, production-grade data science solutions that solve real energy problems
Session
Standard MLOps stacks assume you are operating tens or hundreds of models. What happens when you need to train, validate, and serve thousands of forecasting models on a weekly cadence, producing millions of forecasts for a national energy portfolio? Tools like MLflow start to buckle, per-model evaluation stops being meaningful, and the choice between one global model and many local ones becomes an architectural decision rather than a modeling one.
This talk is a systems architecture case study of the platform we built when off-the-shelf MLOps stopped working. We focus on three concrete, transferable patterns:
- Heterogeneous parallel training — running Spark ML global models and thousands of local models (via applyInPandas) in a single unified workflow, with a four-axis decision framework (data volume per entity, signal heterogeneity, cold-start behavior, operational cost) for routing each segment.
- A model registry that scales beyond MLflow — what we kept, what we dropped, and the schema that lets us manage versions, parameters, and artifacts for a vast portfolio without metadata-store collapse.
- Portfolio-level promotion gates — why per-model accuracy metrics mislead at scale, and how we run champion/challenger experiments with strict temporal integrity and aggregate decision criteria.
Attendees will leave with patterns they can apply to any large-scale time-series system, not just energy forecasting.