JuliaCon 2022 (Times are UTC)

Julia has already succeeded by empowering many scientists and engineers to author their own high-performance compute kernels without the usual ergonomics/composability sacrifices that high-performance code often entails. However, actually leveraging these kernels within production contexts often requires packaging them into an automated service, usually within the context of wider automated pipelines. It is not surprising that in the past few years, many new capabilities and packages have emerged that facilitate this by enabling Julia to be executed atop Kubernetes, interop with tabular data sources/sinks via Apache Arrow, and integrate with other popular cloud-native technologies. This blossoming ecosystem within the wider Julia community demonstrates both the desire and opportunity for Julia's usage in production data engineering contexts.

Topics of discussion for this BoF include:

• current data engineering efforts/challenges faced by industry Julia users maintaining production systems
• containerization of Julia processes and Julia-functions-as-a-service
• executing Julia-based jobs/services via Kubernetes
• Julia-centric workflow/dataflow orchestration
• the intersection of Julia's tabular data ecosystem and enterprise data architectures

Our goal is two-fold:

• uncover the shared data engineering problems, tools, and opportunities that characterize Julia's nascent Data Engineering community
• identify concrete opportunities for open-source and cross-organization collaboration (hackathons, blogs, package development, etc.)