PyConDE & PyData Berlin 2024

In this talk, we explore a new method to approximate Gaussian processes using spectral analysis methods, known as the Hilbert Space Gaussian process (HSGP) approximation. This technique allows us to use and fit Gaussian processes at scale for concrete applications. We provide a basic introduction to the ideas behind the method and make them tangible by implementing them ourselves using Numpyro. We then present two concrete examples in practice using both Numpyro and PyMC. Namely time-varying coefficient regression and time series forecasting.

Idea about the approximation idea: The core of this method relies on the Laplacian's spectral decomposition to approximate kernels' spectral measures as a function of basis functions. The key observation is that the basis functions in the reduced-rank approximation do not depend on the hyperparameters of the covariance function for the Gaussian process. This allows us to speed up the computations tremendously.

References
- Hilbert space methods for reduced-rank Gaussian process regression (https://link.springer.com/article/10.1007/s11222-019-09886-w)
- Practical Hilbert space approximate Bayesian Gaussian processes for probabilistic programming (https://link.springer.com/article/10.1007/s11222-022-10167-2 )
- Example: Hilbert space approximation for Gaussian processes (https://num.pyro.ai/en/stable/examples/hsgp.html)
- PyMCon Web Series - Introduction to Hilbert Space GPs in PyMC - Bill Engels (https://www.youtube.com/watch?v=ri5sJAdcYHk )