2026-09-11 –, Unconference
In the Netherlands, groundwater level time-series stored across partially overlapping databases need to be harmonised before they can be used in hydrogeological models for studying and predicting Dutch hydrology. Manual harmonisation does not scale: the search space contains millions of time-series, and a false match can propagate into model inputs. In this setting, missing a match is often less harmful than accepting the wrong one.
That makes this a certainty-first matching problem rather than a task of retrieving the closest match. Spatial proximity is not sufficient in dense monitoring networks, and a single similarity score breaks down under common artifacts such as vertical shifts and gaps introduced or removed during quality control. More generic approaches, including spectral representations and learned embeddings, can also fail when they compress away details that remain critical in the time-series.
We compare complementary matching scores based on metadata fingerprinting, Dynamic Time Warping, and point-wise value comparison. We found that no method dominates across archetypes: each succeeds on some data pathologies and fails on others. We show how these failure modes can in turn be used to design a more reliable matching workflow.
Attendees working on time-series matching, record linkage, or other large-scale data harmonisation problems applied to real-world data shaped by natural variability, human error, and imperfect observation histories, will leave with a practical framework for deciding when to trust different matching scores and how to combine them into an interpretable, scalable workflow that favors certainty over forced decisions.
We present a practical framework for harmonising partially overlapping groundwater level time-series from two different sources. The case study comes from Dutch groundwater databases, where records that describe the same physical process may differ in metadata, temporal coverage, quality-control history, and observation artifacts. The stakeholder constraint is central: these harmonised records are used as input to hydrogeological models, so false matches are more harmful than missing matches. This makes the task a certainty-first matching problem rather than one of simply retrieving the closest candidate.
The talk compares three complementary matching scores:
- metadata fingerprinting: used to encode structured record similarity and constrain candidate selection;
- Dynamic Time Warping (DTW): used for shape-based comparison of time-series under imperfect alignment;
- Point-wise value comparison: used as a simpler and more controllable baseline in cases where direct time-series comparison remains meaningful.
We show that no method dominates across the full range of real-world artifacts present in the data. In particular, we will inspect records showing recurring archetypes such as vertical shifts, gap creation/filling during quality control, and censored values to explain why each method succeeds in some cases and fails in others. This is not a “best metric” talk. The main contribution is a method-aware, certainty-first workflow: instead of trusting a single score, we use failure modes observed across archetypes to reason about which scores are informative and when a match should remain unresolved.
To ground the discussion technically, we present evaluation results on curated datasets derived from verified migration records, where the true matches are known. The performance is assessed using precision retrieval metrics (including precision at 1 and precision at 3), and use selected examples to show why nearest spatial candidates, compressed representations, or shape-based similarity can each break down under realistic data pathologies. We discuss which parts of the workflow are domain-specific and which generalize to other time-series matching and data harmonisation problems.
The talk is aimed at data scientists, ML practitioners, data engineers, and researchers working with time-series matching, record linkage, or large-scale harmonisation of real-world observational data. The value for attendees is practical: how to think about matching when precision matters more than recall, how to identify archetypes that systematically break common comparison methods, and how to combine interpretable scores into a scalable workflow that can abstain when uncertainty is high.
I work as a Data Scientist at TNO-GDN (the geological survey of the Netherlands). I prototype and develop end-to-end solutions using data science and AI tools to solve challenges related to data management.
I have a scientific background in mechanics of granular materials.
In my free time I love to do sports and contribute to open-source projects.