2024-07-10 –, Function (4.1)
We present the package DiffPointRasterisation.jl, which aims to provide a differentiable rasterization medthod for point cloud data.
The focus of the package is on rasterizing volumetric 3D point cloud data either to 3D voxel grids or to 2-dimensional (raster) images.
This enables gradient-based 3D reconstruction algorithms for e.g. electron tomography.
Differentiable rendering is concerned with making the 3D graphics pipeline differentiable.
While this is a very broad and hot topic in Computer Graphics and Computer Vision, the focus of the package DiffPointRasterisation.jl is solely on rasterizing volumetric 3D point cloud data either to 3D voxel grids or to 2-dimensional (raster) images.
The implementation and interface however are dimension-independent and could also be used for arbitrary-dimensional data.
The package provides fast implementations for the forward- (rendering) and backward- (gradient calculation) process both on CPU and on GPU (via CUDA.jl and KernelAbstractions.jl).
Gradients/pullbacks are provided via explicit functions for fast allocation-free calculations, but are also integrated into the ChainRules.jl ecosystem for automatic differentiation.
We give a quick overview of the package's capabilities and dive into its single-function interface.
We briefly show how we use DiffPointRasterisation.jl in the process of tomographic reconstruction from cryo-electron microscopy data, and conclude by discussing how the package fits into Julia's differentiable computing landscape.
