Fabian Gans
I am a physicist by training and am currently studying Global Biogeochemical Cycles in the Earth System using Remote Sensing, Meteorological and other data sets based at the Max-Planck-Institute for Biogeochemistry, Jena, Germany.
My first commit to my first Julia package dates back to the year 2012 and since then I have authored and contributed to packages in the Julia Geodata and processing ecosystem, examples are NetCDF.jl, Zarr.jl, DiskArrays.jl, YAXArrays.jl EarthDataLab.jl and others. Some may know me under my github tag @meggart
Sessions
Julia's elegance and speed lends itself to applications in scientific computing that requires a strong scientific data ecosystem. Building upon prior successful minisymposia on tabular data, the Scientific Data minisymposium will extend the scope to dealing with annotated, hiearchical, and n-dimensional data. Additionally, the symposium will invite talks on processing large datasets using advanced chunking techniques, distribution of big data via cloud computing and data formats.
DiskArrays.jl provides the AbstractDiskArray interface for chunked and compressed n-dimensional arrays with slow random access. Implementing the interface gives access to a wide set of indexing, views, reductions and broadcasting. Downstream packages can optimize parallel operations on these chunked arrays using the AbstractDiskArray interface.
PyramidScheme.jl is a package to easily and efficiently compute pyramids of an array which might be larger than RAM.
PyramidScheme.jl provides the Pyramid struct to handle the layers of a pyramid.
A pyramid is a collection of subarrays of a larger array so that every layer is half the size in every dimension so that pixels are combined to get to the next level of the pyramid.
These different layers allow to lazily give an overview of the data without having to load the whole array into memory.