Gilles Landais
Sessions
We explored a way to wrap in database large astronomical tables stored in a remote repository and in a suitable format. The architecture is based on ATSS (Astronomical Table Serialisation System) which exploits binary file allowing a powerful position indexation well suited to intensive use. This binary format is used in the VizieR catalogue service which duplicates large tables in ATSS and in a postgreSQL database to provide an efficient conesearch service and the SQL flexibility needed by the IVOA Table Access Protocol (TAP). The database architecture requires an important capacity to deal with datasets exceeding already the billion of records (Gaia, SDSS, GSC, etc). Some new surveys such as LSST will drive a significant increase in volume which may require a redesign of the architecture. We explored a cost-effective solution in term of funds and maintenance that consists in wrapping ATSS binary data as foreign table. We developed a PostgreSQL extension using the Foreign-Data-Wrapper that maps SQL with the API of the binary file including its cone search and indexation capabilities. The solution has been tested with a selection of large tables in a TAP service, we will discuss the technology, its potential but also the limitation of the current development.
Model Instances in VOTables (MIVOT) defines a syntax to map VOTable data to any data model serialised in VO-DML (Virtual Observatory Data Modeling Language).
This annotation schema operates as a bridge between data and models. It associates both VOTable metadata and data to data model elements (class, attributes, types, etc.). It also brings up VOTable data or metadata that were possibly missing in the table, e.g., fine tuned coordinate system description, or curation tracing. MIVOT became an IVOA recommendation in May 2023.
Having this standard was necessary to exercise data models against real data and to make the data interpretation easier by using code working with common data models.
This paper presents our ongoing developments : reading and writing MIVOT annotations with the CDS RUST library, reading and interpreting annotations with AstroPy/PyVO and creating an add-on for the VOLLT TAP library enable to annotate query responses on the fly.