Covariance Kernels

Covariance kernels for defining Gaussian Processes.

Gaussian processes are defined by their mean function and covariance kernel. In this group, we provide tools for constructing and using covariance kernels. Several simple kernels are provided as well as tools for combining simple kernels into more complicated kernels.

Templates are used extensively in this module to maximize performance (by reducing virtual function calls). Thus, using the c++11 "auto" keyword can result in much cleaner and readable code.

Combining Kernels

As an example of constructing a covariance kernel that is constructed from several simple kernels, consider a kernel of the form

\[ k(x_1,x_2) = k_1(x_1,x_2)\, k_2(x_1,x_2) + k_3(x_1,x_2), \]

where \(k_1\) is a squared exponential kernel, \(k_2\) is a periodic kernel and \(k_3\) is a white noise kernel. Assume the dimension of \(x_1\) and \(x_2\) is stored in a variable called dim. Then, the combined kernel \(k(x,y)\) can be constructed with the following snippet:

#include "MUQ/Approximation/GaussianProcesses/CovarianceKernels.h"
 
// Other setup ...
 
auto k1 = SquaredExpKernel(dim, var1, length1);
auto k2 = PeriodicKernel(dim, var2, length2, period);
auto k3 = WhiteNoiseKernel(dim, var3);
 
auto k = kernel1*kernel2 + kernel3;

or, more succinctly, as

auto k = SquaredExpKernel(dim, var1, length1) * PeriodicKernel(dim, var2, length2, period) + WhiteNoiseKernel(dim, var3);

In either case, the k variable is an instance of "SumKernel<ProductKernel<SquaredExpKernel,PeriodicKernel>, WhiteNoiseKernel>". The "auto" keyword allows us to avoid typing this long type.

Anisotropic Kernels

In many cases, the correlation of a GP in one dimension will be different that the correlation is some other direction. For example, let's say we have two spatial variables \(x\) and \(y\) as well as a kernel of the form

\[ k([x_1,y_1], [x_2, y_2]) = k_x(x_1, x_2)\, k_y(y_1, y_2). \]

Such kernels commonly arise when modeling anisotropic media (e.g., hydraulic conductivity fields). In MUQ, it is possible to specify the dimensions that are used by a kernel. For example, if \(k_1\) and \(k_2\) were both squared exponential kernels, than \(k\) could be defined as

// Only keep the 0 index
std::vector<unsigned> indsx = {0};
auto kx = SquaredExpKernel(2, indsx, varx, Lx);
 
// Only keep the 1 index
std::vector<unsigned> inds2 = {1};
auto ky = SquaredExpKernel(2, indsy, vary, Ly);
 
auto k = k1 * k2;

It is also possible to define more complicated relationships. For example, consider a third component \(z\), and let the kernel be defined as

\[ k([x_1,y_1,z_1], [x_2, y_2, z_2]) = k_{xy}([x_1,y_1], [x_2,y_2])\, k_z(z_1, z_2). \]

This kernel might be constructed with

// Keep both the 0 and 1 indices
std::vector<unsigned> indsxy = {0,1};
auto kx = SquaredExpKernel(3, indsxy, varxy, Lxy);
 
// Only keep the 2 index
std::vector<unsigned> inds2 = {2};
auto kz = SquaredExpKernel(3, indsz, varz, Lz);
 
auto k = kxy * kz;

Classes
class	muq::Approximation::ConstantKernel
class	muq::Approximation::KernelBase
	Base class for all covariance kernels. More...
class	muq::Approximation::KernelImpl< ChildType >
	Base class in CRTP pattern for covariance kernels. More...
class	LinearKernel
class	muq::Approximation::LinearTransformKernel
class	muq::Approximation::MaternKernel
class	muq::Approximation::SquaredExpKernel
class	muq::Approximation::SumKernel
class	muq::Approximation::WhiteNoiseKernel