Astronomical Data Analysis Software & Systems XXXIV

Automated Deployment of Radio Astronomy Pipeline on CPU-GPU Processing Systems: DiFX as a Case Study

The next generation of radio telescopes, such as the Square Kilometre Array (SKA), will require peta-FLOPS processing power to handle the massive amount of data acquired. A new generation of computing pipelines are required to address the SKA challenges leading to the integration of the pipelines on a dedicated heterogeneous High-Performance Computing (HPC) system. The tight real-time and energy constraints are driving the community to study the use of hardware accelerators like GPUs in the computing system. Allocating resources, such as processor times, memory, or communication bandwidth, to support complex algorithms in such systems is known as an NP-complete problem. Existing tools such as Dask and Data Activated 流 Graph Engine (DALiuGE) rely on dataflow Model of Computation (MoC) and have proven to be an efficient solution to specify parallel algorithms and automate their deployment. These models are efficient programming paradigms for expressing the parallelism of an application. However, state-of-the-art dataflow resource allocation only targets CPUs and usually relies on complex graph transformations resulting in a time-consuming process. This paper introduces an automated dataflow resource allocation method and code generation for heterogeneous CPU-GPU systems. Our method efficiently and quickly manages pre-scheduling graph complexity, and optimizes the dataflow model to the target architecture. Experimental results show that the proposed method improves resource allocation and speeds up the process by a factor of 13 compared to the best existing method on a basic architecture. Moreover, the execution times of the obtained implementations are comparable to those of manual implementations.

See also: