JuliaCon 2025

What can be gained from embedding formalized metadata within comments? Lets consider four, seemingly unrelated use cases which together ought to resonate with the vast majority of JuliaCon's attendees:

1. Over time a large number of Julia files can accumulate. Wouldn't it be wonderful to speedily retrieve any, long forgotten poignant example, powerful code snippet, insightful explanatory statement or laboriously tweaked plot related to some task at hand? An instrument solving this problem might prove even more invaluable when collaborating in a group sharing a repository of files.

2. One might want to (inter)link a certain fragment of one file with another fragment, possibly contained in some other file. E.g.: linking an intricate piece of code to an enlightening explanation or a clarifying toy example.

3. Would you like to harness the power of Documenter, enjoying simplified control while sticking to pure Julia files rather than having to deal with additional markdown documents? More generally, a Julia file can serve as a source for all kinds of outputs.

4. Being able to control how individual chunks of code / text are being executed, respectively rendered, can be beneficial. E.g.: certain passages of a file might be intended for personal usage only. Marking them as such would allow automated removal before sharing.

To illustrate how a set of meta instructions can offer a unified, efficient approach, lets introduce GoLiterate. This is an extended adaptation of the well established Literate by Fredrik Ekre et al. Thanks :-)

The defining concept of Literate consists of storing chunks of code and text together in pure Julia files. These serve as source files which can subsequently be converted into a variety of formats such as notebooks, standard markdown or even documenter markdown files. A suitable system for metadata would allow exerting precise control over these format's wide ranging capabilities from within the Julia source file. GoLiterate is a fork of Literate to showcase this.

So what does the proposed schema for metadata look like?

Any line starting with #~ is considered to be a metaline i.e.: it will be processed as metadata. Contiguous metalines constitute a coherent block of metadata separating it from others. By positioning such a block just above a chunk of code or text [no vertical spacing in between] the former gets attached to the latter – similar to how docstrings get associated with Julia structures following them. For the purpose of inheritance, a hierarchy of blocks of metadata can be defined by prepending one or more # to the first metaline of a given block which works analogously to markdown's syntax for titles.

The current implementation of GoLiterate accepts standard metalines [more below] with its content consisting of key-value pairs where the keys are restricted to one word and the values may include general strings or vectors. Like this a name or title can be provided and labels can be applied. A UID is automatically generated and assigned to each block of metadata. This way a targeted search for particular sections of code or text becomes possible as well as interlinking various sections.

The execution of some tasks may be so common that they are deserving of a simplified syntax to facilitate their usage. Others may benefit or even necessitate custom tailored metadata to enable full functionality, as is the case for instructions passed to Documenter. The current schema resolves this by introducing specialized metalines. E.g.: a leading #~d instructs the parser to process the subsequent content for the usage with Documenter. Appending a ! to a metaline declaration, #~!, causes its content to be retained but not processed.

All this is readily exemplified:

# Start file.jl

#~  [Metadata Block 1]
#~

 ... code/text

##~ [Metadata Block 2]
#~!
#~
"""
    addone(x)
This is a docstring for addone().
"""
function addone(x)
    x += 1
end

 ... code/text

#~   [Metadata Block 3]
#~d jldoctest; output = false
addone(2)

# output

3

# End file.jl

Above three distinct blocks of metadata are presented. The second of which can inherit attributes from the first. While the second block is attached to a chunk and thus gets applied to it, the first block has no direct effect on any code or text chunk. Its attributes are only passed on indirectly through inheritance.

Block 3 does not inherit anything from the blocks before since it's not a sub-block / sub-section of them. However, since it contains a specialized metaline meant for Documenter, the chunk of code following it will be fenced by ```jldoctest; output = false … ``` when targeting Documenter.

Time permitting, the talk will be expanding on this brief overview.

In the near future, AI will automate much of the labeling. Crucially, if this black box is confined within a formalized framework of metadata, it can become train- and tunable.