2024-07-12 –, If (1.1)
Notebooks like Pluto provide a great environment for enhancing code with explanations and interactive explorability. While popular with experimental research code, this is rarley found in package implementations. Since Pluto notebooks are actually plain Julia code under the hood, they can be used to implement a package while still being a descriptive place for exploration. In this talk, we learn how to use packages like PlutoDevMacros and PlutoTest to go from quick and dirty to reusable!
We have come a long way since Donald Knuth's WEB programming language for literate programming from the 1980s. His vision of interweaving code for the computer and text for humans is still popular today, however. Nowadays, notebooks like Jupyter and Pluto are the most popular tools for that purpose. While Jupyter notebooks are stored as JSON files and can therefore only be interpreted as notebooks, Pluto uses plain Julia files and annotates it with some comments to specify the notebook structure. This means that other Julia code does not have to concern itself with the Pluto-origin of that file and can just interact with it as with any other Julia file. Among many others, one benefit of this fact is that the Pluto notebook can contain the implementation of a reusable package. Such a package often naturally evolves from a script or notebook that answers one specific research question when the broader context and applicability become evident. Usually, this evolution then means starting from scratch and reimplementing the logic from the exploration. The extracted package will also miss all the deeper explanations and derivations of the implemented algorithms that were present in the notebook, making it literal in Knuth's sense. Additionally, more often than not, one will continue to experiment with the original code and, at this point, much effort is necessary to keep the code bases in sync. Therefore, transforming the explorative notebook into the package itself can be an alternative worth considering. A growing package quickly becomes impractical to maintain in a single file or notebook. Here, packages like PlutoDevMacros and PlutoLinks help because they are able to connect multiple Pluto notebooks with eachother. With PlutoTest, we can even add interactive testing for our code.
In this talk, we follow an example that starts as a typical research notebook and then showcases the evolution towards a reusable package by employing the mentioned packages step by step. The devolper experience is not yet perfect at this point, so the purpose of this talk is merely to suggest a workflow for literate package development and hint at where further improvements could be beneficial.
Hi! I'm a PhD student at University Hospital Jena, Germany, conducting research on computational structural biology. We are interested in reconstructing the conformational movement of large biomolecules from cryo-EM images. Methodically, we employ Bayesian probabilistic inference. If that's up your alley, let's have a chat!
At JuliaCon, I will be talking about another passion of mine, how code and text fit together.