2021-07-29 –, Blue
PackageCompiler.jl has become the de facto method for creating standalone Julia applications. In this talk, we will demonstrate the use of PackageCompiler.jl to produce shared library bundles. This functionality was added recently and allows the easy creation of location-independent dynamic libraries which can be linked to and called from C, C++, Rust, or other languages which can link to and use C libraries.
Julia has been touted as a great solution to the two-language problem (and it is). But for many, interacting with code in other languages is a necessity.
Numerous packages exist which aid interoperability with other languages, including C (Clang.jl), C++ (CxxWrap.jl), Java (JavaCall.jl), Matlab (Matlab.jl / Mex.jl), Python (PyCall.jl / pyjulia), R (RCall.jl / JuliaCall), Mathematica (MathLink.jl), and rust (jlrs).
Many of these packages focus on calling out to code in other languages from Julia, but there is also support for calling Julia code from other languages, especially for those that have the ability to call C functions, and that is what we will focus on here.
The Julia manual has a full section on Embedding Julia. Until now, this has been the standard way to embed and call Julia from other languages. Using the ideas here, along with custom Julia sysimage generation with PackageCompiler.jl, one of us created a proof-of-concept repository for creating a shared library from Julia code for C or other languages (https://github.com/simonbyrne/libcg).
One downside of this work is that the library was not easy to relocate--it contained hard-coded paths to the Julia runtime. We wanted the ability to create a relocatable shared library.
PackageCompiler.jl already allowed the creation of “apps”--bundles of files, including an executable--which could be relocated and moved to other machines (with some minor caveats). We extended this functionality to create relocatable shared libraries with a create_library function.
The actual act of creating a shared library with PackageCompiler.jl is very much like creating an “app”, and has a very similar output--a bundle of directories which include the shared library and enough of the Julia runtime to run. This bundle can be zipped or tarred up, sent to other computers, and installed in any location that a linker can find it. The user has the option of setting the library version (on Mac and Linux), and can include C header files for the Julia functions she has exported in the shared library.
For this talk, we will give a brief overview of the create_library functionality, discuss situations in which it might be used, show how to use it, and discuss its limitations.
I'm a contributor to the Julia language and many packages.
Simon is the lead software engineer at the CliMA project.
Kevin Squire is a Senior Software Engineer at Second Spectrum, Inc., where he works in the AI group on machine learning algorithms and infrastructure. Previously, he completed a postdoc in Computational Genetics at UCLA and taught Computer Science at the Naval Postgraduate School. He received his PhD in Electrical Engineering from the University of Illinois. Kevin has been interested in Julia since its first public release and was an early contributor to the language.