JuliaCon 2023

Computational workflows and pipelines are essential in large-scale data analysis and bioinformatics. Here, we present Pipelines.jl and JobSchedulers.jl, as the reproducible and scalable pipeline builder and workload manager. Pipelines.jl provides simple but powerful methods to wrap Julia function or external command, and provide data and dependency validation, and job re-try or skipping. JobSchedulers.jl can allocate CPU and memory to jobs and defer a job until other jobs are finished.