Tobias Senst
Tobias Senst is a Senior Machine Learning Engineer at idealo internet GmbH. He received his PhD from Technische Universität Berlin and has more than 10 years of experience in computer vision and video analytics research.
At idealo, he transitioned from working with images and videos to natural language processing. For the past five years, he has worked on high-performance machine learning models and automated continuous training, development, and integration (CI/CD/CT) pipelines in the field of cataloging. His professional focus is on MLOps, ML testing, and research.
Session
E-commerce cataloging at idealo operates at extreme scale: 4.5 billion offers from 50,000+ shops across six countries, with peak ingestion rates of 4.8 million offers per minute. While large language models (LLMs) provide strong classification accuracy, they are too slow and costly for billion-scale real-time processing. This talk shows how idealo builds a cost-efficient, high-throughput machine learning system that leverages LLM knowledge without deploying full models in production.
We present how knowledge distillation from a large e5 instruction model enables a compact multilingual MiniLM encoder to achieve high accuracy, and how optimized inference runtimes and specialized hardware such as AWS Neuron help meet strict latency and cost requirements. Beyond modeling, we highlight key operational challenges: constructing training datasets from massively imbalanced data, selecting the right encoder architecture from today’s model landscape, and designing a robust MLOps lifecycle with automated data sampling, training, deployment, and monitoring.
Attendees will learn practical techniques for scaling ML systems under real-world constraints, how to extract value from LLMs when they are too large to serve directly, and how to transition research prototypes into reliable, high-volume production pipelines.