Kai Jeggle
Kai Jeggle is a Meteo AI Scientist at Dexter Energy, where he turns high-dimensional weather and satellite data into insights for energy price and power generation forecasting using geospatial AI. He holds a PhD from ETH Zurich, where his research focused on combining machine learning with atmospheric physics, including time at the European Space Agency's Phi-Lab. He previously also worked as a software engineer at Hopsworks, an MLOps startup in Stockholm. Kai has been active in the AI for climate community by leading initiatives at Climate Change AI, a global non-profit working at the intersection of machine learning and climate action.
Session
Representation learning has transformed AI for language and vision. Large foundation models learn abstract, reusable features that can transfer to arbitrary downstream tasks. But what about other real world problems ?
Many practitioners work with inputs that are high-dimensional containing complex non-linear patterns: weather or satellite maps, sensor networks, graph data or medical scans. In contrast, prediction targets are often simple and low-dimensional. Learning directly from raw high-dimensional inputs to low-dimensional targets is tempting, but often fails in practice: limited labeled data forces models to simultaneously discover useful representations and solve the prediction task.
This talk presents the embed first, predict later paradigm and shows that it works just as powerfully for other domains as it does for text and images, using energy forecasting from weather data as our running example. We show how a pre-trained open source foundation model learns abstract representations of weather such as storms or cloud systems, and how these embeddings can be fed into a multitude of lightweight downstream tasks. The result is a modular and reusable ML pipeline that generalizes well.
You'll leave with three concrete things: (1) the building blocks of an embedding-first pipeline, illustrated through a real-world implementation, (2) practical tips and tricks for visualizing and inspecting high-dimensional embedding spaces and (3) an intuition for when this paradigm can be applied to your own domain and pointers on how to get started.