Tiled is a full-fledged data management service designed specifically to help scientists store, find, and access scientific data at scale easily.
The concept of data structures is the cornerstone of Tiled; it allows us to abstract the inherent diversity of various file formats and data storage types to a handful of scientifically meaningful representations: arrays, tables, nested hierarchies, and even awkward, ragged, and sparse arrays. Tiled provides a consistent API to such disparate datasets and naturally integrates with the SciPy ecosystem, including NumPy, pandas, xarray, Dask, and more. The users can slice, convert, and retrieve only the data they need, or even subscribe to live streams from external instruments and send updates to a dashboard. Importantly, Tiled supports operations with rich metadata – including search – making the data registered in Tiled discoverable and interactable with minimal overhead, by the human users and AI agents alike. Tiled runs equally well on a private laptop or in a large facility’s data center. Its built-in authentication and authorization mechanisms make the data access controllable and secure. Finally, Tiled is a fully open-source project developed under a multi-institutional governance model, which reflects our commitment to open science and the FAIR principles in scientific computing.
In this talk we will introduce Tiled’s architecture, demonstrate its most popular use cases using the native Python client, discuss deployment and integration strategies, and show how it can simplify practical scientific data workflows.