The ESO Data Processing System (EDPS) is a new software infrastructure to run ESO science data processing modules (“pipeline recipes”) that is currently in an advanced status of development. These recipes are used for quality control purposes, unsupervised production of science and calibration products for the archive, and are embedded in an interactive data reduction system that is also offered to external science users. EDPS aims to unify all these use cases and thereby replace all dedicated systems that are currently used at the European Southern Observatory (ESO).
EDPS is designed to handle all present and future instruments from ESO's La Silla, Paranal and, in perspective, Armazones (ELT) Observatory. They range from simple imagers, to 1D and 3D spectrographs to optical interferometry (VLTI), across the entire optical and infrared wavelength interval. Instrument-specific workflows can be configured in EDPS using a flexible DSL (Domain Specific Language) written in Python.
In this talk I will present the high-level requirements of EDPS as well as its design, focusing in particular on the data organisation and processing model and on the workflow specification language.