Melissa DeLucchi
Melissa (they/she) is Principal Software Engineer for LINCC Frameworks (LSST Interdisciplinary Network for Collaboration and Computing) at Carnegie Mellon University in Pittsburgh, PA. They additionally serve on the executive council for the LSST Informatics and Statistics Science Collaboration.
Session
Current generation surveys will introduce unprecedented challenges in the joint analysis of astronomical datasets. We present Hierarchical Adaptive Tiling Scheme (HATS, formerly referred to as hipscat), an advanced spatial partitioning of large datasets using Parquet storage. LSST Interdisciplinary Network for Collaboration and Computing (LINCC) Frameworks has built the LSDB library on top of the HATS format for efficient and scalable cross-matching and analysis of big datasets, to enable catalog builders to provide archival and future catalogs in HATS format. We have created a global partnership to provide survey data in HATS (including STScl, IPAC, CDS, and S-PLUS).
In this talk, we will showcase: an overview of the tiling scheme and its power in driving massive cross-matches; an introduction to the LSDB API; a demonstration of its use on DP1 and crossmatches to publicly-available HATS catalogs like Gaia, ZTF, and PanStarrs.