GPs & Functionality

Gaussian Processes (GPs) can conveniently be used for Bayesian supervised learning, such as regression and classification. In its simplest form, GP inference can be implemented in a few lines of code. However, in practice, things typically get a little more complicated: you might want to use expressive covariance and mean functions, learn good values for hyperparameters, use non-Gaussian likelihood functions (rendering exact inference intractable), use approximate inference algorithms, or combinations of many or all of the above.

A comprehensive introduction to Gaussian Processes for Machine Learning is provided in the GPML book by Rasmussen and Williams, 2006.

Inference and Likelihood for Standard GPs

The following combinations of inference methods and likelihood functions are possible:

  • Regression:
    • Exact inference and Gaussian likelihood (default)
    • EP inference and Laplace likelihood
  • Classification:
    • EP inference and Erf likelihood (default)
    • Laplace inference and Erf likelihood

List of Functionality

This table lists the functionality implemented in pyGPs.

Functionality Kernel Mean Likelihood Inference Optimizer
Simple Constant Constant Gaussian Exact Minimize
Linear Linear Cumulative Gaussian (Erf) EP CG
Linear ard One Laplace Laplace SCG
Matérn (1,3,5) Zero     BFGS
Piecewise Poly
RBF iso
RBF ard
RBF unit
RQ iso
RQ ard
Spectral Mixture
Composite Sum (+) Sum (+)      
Product (*) Product (*)
Scale (*) Scale (*)
  Power (**)
Sparse GP       FITC Exact  
FITC Laplace
Graphs Diffusion        
VN Diffusion
Pseudo Inv Laplace
Regularized Laplace
p-step Random Walk
Inverse Cosine
Propagation Kernel
Other Customized Customized      
Precomputed Kernel  

pyGPs also provide cross-validation and some built-in evalation methods.