SciPy 2026

FAIRer Data: The case for Data Advertising in the age of Agentic AI
2026-07-15 , Thomas Swain Room

For scientists wanting to work with and analyse earth science data, the standard remains delivering tooling via python packages, and data via HPC or the cloud. For data siloed on an HPC system, this presents a barrier to findability and accessibility. However, with agentic AI now widely available, the cost of learning a new tech stack or toolchain to deliver this data has plummeted.

In this talk, I'll outline how we utilised agentic AI to translate an intake catalog into an interactive, single page web application, maximising data discoverability whilst leaning on our existing data infrastructure and established Python API's to constrain the scope, keep the wrapper thin, and the code from becoming spaghettified, unmaintainable AI slop.


FAIR (Findable, Accessible, Interoperable, Reusable) data is widely agreed on as the benchmark for data management and stewardship. Unfortunately, findability is often an afterthought - and datasets that can't be found are difficult to access!
In the earth sciences, we commonly assume that if a dataset can be found by someone who already knows what they are looking for or where to look for it, then it is findable. For small and tight knit research communities, or cloud distributed datasets, this might be the case.
However, within the earth sciences, many datasets remained siloed on HPC systems, where it is often assumed that tribal knowledge that can be obtained from a supervisor, colleague, or collaborator is sufficient to guide new users through these systems.
Worse yet, users are often expected to obtain login access to an HPC simply to discover which datasets are available, before they can even determine whether the data are relevant to their needs.
In practice, this assumption means that datasets are only available to an in group of users who are already familiar with them.

Why is this so often the case? The defacto tool for data analysis in the earth sciences is Python, but the best way to advertise and distribute datasets is through the web. If we want to distribute the data ourselves, without getting experienced web developers involved, this leaves us with a few options: static site generation through tools like readthedocs, writing a python web server, or going all in and learning enough JavaScript to create an interactive data exploration tool.

The key issue? The better the interface, the more time and effort you'd need to sink into learning a new tech stack, toolchain, and way of thinking. The result of this - lots of clunky interfaces to explore and obtain data.

Whilst this is still true, with AI agents now widely available, the cost of learning a new tech stack or creating new data delivery tools has plummeted. For an experienced developer with a hoard of well structured data, 'vibe-coding' a wrapper to advertise and distribute that data is now a serious option.

In this talk, I'll walk through how we created a tool for advertising Australia's trove of earth science data, making it easy to find and discover for anyone with a browser and an internet connection - not just those who already had the right HPC login. Expect to learn:

  • Why well structured data, metadata, and documentation are more important than ever - not less - in this new data landscape.
  • How we used intake, duckdb-wasm, polars and Vue to create a tool that blends cloud and HPC data delivery.
  • Why the proliferation of social media and gamification of content has made data advertising more important than ever.
  • How keeping wrappers thin and focusing on the interactive experience lets the data do the talking.
  • How we went about testing, gathering feedback, and iterating on an interactive tool in an area where users expect to be provided with static content or a Python API.

Intended Audience: Earth Scientists, people interested in data sharing, people looking to use emerging tools to make their work more impactful

Charles is a Research Software Engineer at ACCESS-NRI, where he works in the Model Evaluation and Diagnostics team, helping make it easier to access and analyse climate data. He has a PhD in Oceanography, where he first discovered his love of wrangling and disseminating data.

When not in front of a computer, he enjoys routinely injuring himself in a variety of sports.

This speaker also appears in: