# Implementations¶

 Latent([mean_func, cov_func]) Latent Gaussian process. Marginal([mean_func, cov_func]) Marginal Gaussian process. LatentKron([mean_func, cov_funcs]) Latent Gaussian process whose covariance is a tensor product kernel. MarginalKron([mean_func, cov_funcs]) Marginal Gaussian process whose covariance is a tensor product kernel. MarginalSparse([mean_func, cov_func, approx]) Approximate marginal Gaussian process. TP([mean_func, cov_func, nu]) Student's T process prior.
class pymc3.gp.gp.Latent(mean_func=<pymc3.gp.mean.Zero object>, cov_func=<pymc3.gp.cov.Constant object>)

Latent Gaussian process.

The gp.Latent class is a direct implementation of a GP. No additive noise is assumed. It is called “Latent” because the underlying function values are treated as latent variables. It has a prior method and a conditional method. Given a mean and covariance function the function $$f(x)$$ is modeled as,

$f(x) \sim \mathcal{GP}\left(\mu(x), k(x, x')\right)$

Use the prior and conditional methods to actually construct random variables representing the unknown, or latent, function whose distribution is the GP prior or GP conditional. This GP implementation can be used to implement regression on data that is not normally distributed. For more information on the prior and conditional methods, see their docstrings.

Parameters
cov_func: None, 2D array, or instance of Covariance

The covariance function. Defaults to zero.

mean_func: None, instance of Mean

The mean function. Defaults to zero.

Examples

# A one dimensional column vector of inputs.
X = np.linspace(0, 1, 10)[:, None]

with pm.Model() as model:
# Specify the covariance function.

# Specify the GP.  The default mean function is Zero.
gp = pm.gp.Latent(cov_func=cov_func)

# Place a GP prior over the function f.
f = gp.prior("f", X=X)

...

# After fitting or sampling, specify the distribution
# at new points with .conditional
Xnew = np.linspace(-1, 2, 50)[:, None]

with model:
fcond = gp.conditional("fcond", Xnew=Xnew)

conditional(name, Xnew, given=None, **kwargs)

Returns the conditional distribution evaluated over new input locations Xnew.

Given a set of function values f that the GP prior was over, the conditional distribution over a set of new points, f_* is

$f_* \mid f, X, X_* \sim \mathcal{GP}\left( K(X_*, X) K(X, X)^{-1} f \,, K(X_*, X_*) - K(X_*, X) K(X, X)^{-1} K(X, X_*) \right)$
Parameters
name: string

Name of the random variable

Xnew: array-like

Function input values.

given: dict

Can optionally take as key value pairs: X, y, noise, and gp. See the section in the documentation on additive GP models in PyMC3 for more information.

**kwargs

Extra keyword arguments that are passed to MvNormal distribution constructor.

prior(name, X, reparameterize=True, **kwargs)

Returns the GP prior distribution evaluated over the input locations X.

This is the prior probability over the space of functions described by its mean and covariance function.

$f \mid X \sim \text{MvNormal}\left( \mu(X), k(X, X') \right)$
Parameters
name: string

Name of the random variable

X: array-like

Function input values.

reparameterize: bool

Reparameterize the distribution by rotating the random variable by the Cholesky factor of the covariance matrix.

**kwargs

Extra keyword arguments that are passed to distribution constructor.

class pymc3.gp.gp.LatentKron(mean_func=<pymc3.gp.mean.Zero object>, cov_funcs=<pymc3.gp.cov.Constant object>)

Latent Gaussian process whose covariance is a tensor product kernel.

The gp.LatentKron class is a direct implementation of a GP with a Kronecker structured covariance, without reference to any noise or specific likelihood. The GP is constructed with the prior method, and the conditional GP over new input locations is constructed with the conditional method. conditional and method. For more information on these methods, see their docstrings. This GP implementation can be used to model a Gaussian process whose inputs cover evenly spaced grids on more than one dimension. LatentKron is relies on the KroneckerNormal distribution, see its docstring for more information.

Parameters
cov_funcs: list of Covariance objects

The covariance functions that compose the tensor (Kronecker) product. Defaults to [zero].

mean_func: None, instance of Mean

The mean function. Defaults to zero.

Examples

# One dimensional column vectors of inputs
X1 = np.linspace(0, 1, 10)[:, None]
X2 = np.linspace(0, 2, 5)[:, None]
Xs = [X1, X2]
with pm.Model() as model:
# Specify the covariance functions for each Xi
cov_func1 = pm.gp.cov.ExpQuad(1, ls=0.1)  # Must accept X1 without error
cov_func2 = pm.gp.cov.ExpQuad(1, ls=0.3)  # Must accept X2 without error

# Specify the GP.  The default mean function is Zero.
gp = pm.gp.LatentKron(cov_funcs=[cov_func1, cov_func2])

# ...

# After fitting or sampling, specify the distribution
# at new points with .conditional
# Xnew need not be on a full grid
Xnew1 = np.linspace(-1, 2, 10)[:, None]
Xnew2 = np.linspace(0, 3, 10)[:, None]
Xnew = np.concatenate((Xnew1, Xnew2), axis=1)  # Not full grid, works
Xnew = pm.math.cartesian(Xnew1, Xnew2)  # Full grid, also works

with model:
fcond = gp.conditional("fcond", Xnew=Xnew)

conditional(name, Xnew, **kwargs)

Returns the conditional distribution evaluated over new input locations Xnew.

Xnew will be split by columns and fed to the relevant covariance functions based on their input_dim. For example, if cov_func1, cov_func2, and cov_func3 have input_dim of 2, 1, and 4, respectively, then Xnew must have 7 columns and a covariance between the prediction points

cov_func(Xnew) = cov_func1(Xnew[:, :2]) * cov_func1(Xnew[:, 2:3]) * cov_func1(Xnew[:, 3:])


The distribution returned by conditional does not have a Kronecker structure regardless of whether the input points lie on a full grid. Therefore, Xnew does not need to have grid structure.

Parameters
name: string

Name of the random variable

Xnew: array-like

Function input values. If one-dimensional, must be a column vector with shape (n, 1).

**kwargs

Extra keyword arguments that are passed to MvNormal distribution constructor.

prior(name, Xs, **kwargs)

Returns the prior distribution evaluated over the input locations Xs.

Parameters
name: string

Name of the random variable

Xs: list of array-like

Function input values for each covariance function. Each entry must be passable to its respective covariance without error. The total covariance function is measured on the full grid cartesian(*Xs).

**kwargs

Extra keyword arguments that are passed to the KroneckerNormal distribution constructor.

class pymc3.gp.gp.Marginal(mean_func=<pymc3.gp.mean.Zero object>, cov_func=<pymc3.gp.cov.Constant object>)

Marginal Gaussian process.

The gp.Marginal class is an implementation of the sum of a GP prior and additive noise. It has marginal_likelihood, conditional and predict methods. This GP implementation can be used to implement regression on data that is normally distributed. For more information on the prior and conditional methods, see their docstrings.

Parameters
cov_func: None, 2D array, or instance of Covariance

The covariance function. Defaults to zero.

mean_func: None, instance of Mean

The mean function. Defaults to zero.

Examples

# A one dimensional column vector of inputs.
X = np.linspace(0, 1, 10)[:, None]

with pm.Model() as model:
# Specify the covariance function.

# Specify the GP.  The default mean function is Zero.
gp = pm.gp.Marginal(cov_func=cov_func)

# Place a GP prior over the function f.
sigma = pm.HalfCauchy("sigma", beta=3)
y_ = gp.marginal_likelihood("y", X=X, y=y, noise=sigma)

...

# After fitting or sampling, specify the distribution
# at new points with .conditional
Xnew = np.linspace(-1, 2, 50)[:, None]

with model:
fcond = gp.conditional("fcond", Xnew=Xnew)

conditional(name, Xnew, pred_noise=False, given=None, **kwargs)

Returns the conditional distribution evaluated over new input locations Xnew.

Given a set of function values f that the GP prior was over, the conditional distribution over a set of new points, f_* is:

$f_* \mid f, X, X_* \sim \mathcal{GP}\left( K(X_*, X) [K(X, X) + K_{n}(X, X)]^{-1} f \,, K(X_*, X_*) - K(X_*, X) [K(X, X) + K_{n}(X, X)]^{-1} K(X, X_*) \right)$
Parameters
name: string

Name of the random variable

Xnew: array-like

Function input values. If one-dimensional, must be a column vector with shape (n, 1).

pred_noise: bool

Whether or not observation noise is included in the conditional. Default is False.

given: dict

Can optionally take as key value pairs: X, y, noise, and gp. See the section in the documentation on additive GP models in PyMC3 for more information.

**kwargs

Extra keyword arguments that are passed to MvNormal distribution constructor.

marginal_likelihood(name, X, y, noise, is_observed=True, **kwargs)

Returns the marginal likelihood distribution, given the input locations X and the data y.

This is integral over the product of the GP prior and a normal likelihood.

$y \mid X,\theta \sim \int p(y \mid f,\, X,\, \theta) \, p(f \mid X,\, \theta) \, df$
Parameters
name: string

Name of the random variable

X: array-like

Function input values. If one-dimensional, must be a column vector with shape (n, 1).

y: array-like

Data that is the sum of the function with the GP prior and Gaussian noise. Must have shape (n, ).

noise: scalar, Variable, or Covariance

Standard deviation of the Gaussian noise. Can also be a Covariance for non-white noise.

is_observed: bool

Whether to set y as an observed variable in the model. Default is True.

**kwargs

Extra keyword arguments that are passed to MvNormal distribution constructor.

predict(Xnew, point=None, diag=False, pred_noise=False, given=None)

Return the mean vector and covariance matrix of the conditional distribution as numpy arrays, given a point, such as the MAP estimate or a sample from a trace.

Parameters
Xnew: array-like

Function input values. If one-dimensional, must be a column vector with shape (n, 1).

point: pymc3.model.Point

A specific point to condition on.

diag: bool

If True, return the diagonal instead of the full covariance matrix. Default is False.

pred_noise: bool

Whether or not observation noise is included in the conditional. Default is False.

given: dict

Same as conditional method.

predictt(Xnew, diag=False, pred_noise=False, given=None)

Return the mean vector and covariance matrix of the conditional distribution as symbolic variables.

Parameters
Xnew: array-like

Function input values. If one-dimensional, must be a column vector with shape (n, 1).

diag: bool

If True, return the diagonal instead of the full covariance matrix. Default is False.

pred_noise: bool

Whether or not observation noise is included in the conditional. Default is False.

given: dict

Same as conditional method.

class pymc3.gp.gp.MarginalKron(mean_func=<pymc3.gp.mean.Zero object>, cov_funcs=<pymc3.gp.cov.Constant object>)

Marginal Gaussian process whose covariance is a tensor product kernel.

The gp.MarginalKron class is an implementation of the sum of a Kronecker GP prior and additive white noise. It has marginal_likelihood, conditional and predict methods. This GP implementation can be used to efficiently implement regression on data that are normally distributed with a tensor product kernel and are measured on a full grid of inputs: cartesian(*Xs). MarginalKron is based on the KroneckerNormal distribution, see its docstring for more information. For more information on the prior and conditional methods, see their docstrings.

Parameters
cov_funcs: list of Covariance objects

The covariance functions that compose the tensor (Kronecker) product. Defaults to [zero].

mean_func: None, instance of Mean

The mean function. Defaults to zero.

Examples

# One dimensional column vectors of inputs
X1 = np.linspace(0, 1, 10)[:, None]
X2 = np.linspace(0, 2, 5)[:, None]
Xs = [X1, X2]
y = np.random.randn(len(X1)*len(X2))  # toy data
with pm.Model() as model:
# Specify the covariance functions for each Xi
cov_func1 = pm.gp.cov.ExpQuad(1, ls=0.1)  # Must accept X1 without error
cov_func2 = pm.gp.cov.ExpQuad(1, ls=0.3)  # Must accept X2 without error

# Specify the GP.  The default mean function is Zero.
gp = pm.gp.MarginalKron(cov_funcs=[cov_func1, cov_func2])

# Place a GP prior over the function f.
sigma = pm.HalfCauchy("sigma", beta=3)
y_ = gp.marginal_likelihood("y", Xs=Xs, y=y, sigma=sigma)

# ...

# After fitting or sampling, specify the distribution
# at new points with .conditional
# Xnew need not be on a full grid
Xnew1 = np.linspace(-1, 2, 10)[:, None]
Xnew2 = np.linspace(0, 3, 10)[:, None]
Xnew = np.concatenate((Xnew1, Xnew2), axis=1)  # Not full grid, works
Xnew = pm.math.cartesian(Xnew1, Xnew2)  # Full grid, also works

with model:
fcond = gp.conditional("fcond", Xnew=Xnew)

conditional(name, Xnew, pred_noise=False, **kwargs)

Returns the conditional distribution evaluated over new input locations Xnew, just as in Marginal.

Xnew will be split by columns and fed to the relevant covariance functions based on their input_dim. For example, if cov_func1, cov_func2, and cov_func3 have input_dim of 2, 1, and 4, respectively, then Xnew must have 7 columns and a covariance between the prediction points

cov_func(Xnew) = cov_func1(Xnew[:, :2]) * cov_func1(Xnew[:, 2:3]) * cov_func1(Xnew[:, 3:])


The distribution returned by conditional does not have a Kronecker structure regardless of whether the input points lie on a full grid. Therefore, Xnew does not need to have grid structure.

Parameters
name: string

Name of the random variable

Xnew: array-like

Function input values. If one-dimensional, must be a column vector with shape (n, 1).

pred_noise: bool

Whether or not observation noise is included in the conditional. Default is False.

**kwargs

Extra keyword arguments that are passed to MvNormal distribution constructor.

marginal_likelihood(name, Xs, y, sigma, is_observed=True, **kwargs)

Returns the marginal likelihood distribution, given the input locations cartesian(*Xs) and the data y.

Parameters
name: string

Name of the random variable

Xs: list of array-like

Function input values for each covariance function. Each entry must be passable to its respective covariance without error. The total covariance function is measured on the full grid cartesian(*Xs).

y: array-like

Data that is the sum of the function with the GP prior and Gaussian noise. Must have shape (n, ).

sigma: scalar, Variable

Standard deviation of the white Gaussian noise.

is_observed: bool

Whether to set y as an observed variable in the model. Default is True.

**kwargs

Extra keyword arguments that are passed to KroneckerNormal distribution constructor.

predict(Xnew, point=None, diag=False, pred_noise=False)

Return the mean vector and covariance matrix of the conditional distribution as numpy arrays, given a point, such as the MAP estimate or a sample from a trace.

Parameters
Xnew: array-like

Function input values. If one-dimensional, must be a column vector with shape (n, 1).

point: pymc3.model.Point

A specific point to condition on.

diag: bool

If True, return the diagonal instead of the full covariance matrix. Default is False.

pred_noise: bool

Whether or not observation noise is included in the conditional. Default is False.

predictt(Xnew, diag=False, pred_noise=False)

Return the mean vector and covariance matrix of the conditional distribution as symbolic variables.

Parameters
Xnew: array-like

Function input values. If one-dimensional, must be a column vector with shape (n, 1).

diag: bool

If True, return the diagonal instead of the full covariance matrix. Default is False.

pred_noise: bool

Whether or not observation noise is included in the conditional. Default is False.

class pymc3.gp.gp.MarginalSparse(mean_func=<pymc3.gp.mean.Zero object>, cov_func=<pymc3.gp.cov.Constant object>, approx='FITC')

Approximate marginal Gaussian process.

The gp.MarginalSparse class is an implementation of the sum of a GP prior and additive noise. It has marginal_likelihood, conditional and predict methods. This GP implementation can be used to implement regression on data that is normally distributed. The available approximations are:

• DTC: Deterministic Training Conditional

• FITC: Fully independent Training Conditional

• VFE: Variational Free Energy

Parameters
cov_func: None, 2D array, or instance of Covariance

The covariance function. Defaults to zero.

mean_func: None, instance of Mean

The mean function. Defaults to zero.

approx: string

The approximation to use. Must be one of VFE, FITC or DTC.

References

• Quinonero-Candela, J., and Rasmussen, C. (2005). A Unifying View of Sparse Approximate Gaussian Process Regression.

• Titsias, M. (2009). Variational Learning of Inducing Variables in Sparse Gaussian Processes.

Examples

# A one dimensional column vector of inputs.
X = np.linspace(0, 1, 10)[:, None]

# A smaller set of inducing inputs
Xu = np.linspace(0, 1, 5)[:, None]

with pm.Model() as model:
# Specify the covariance function.

# Specify the GP.  The default mean function is Zero.
gp = pm.gp.MarginalSparse(cov_func=cov_func, approx="FITC")

# Place a GP prior over the function f.
sigma = pm.HalfCauchy("sigma", beta=3)
y_ = gp.marginal_likelihood("y", X=X, Xu=Xu, y=y, sigma=sigma)

...

# After fitting or sampling, specify the distribution
# at new points with .conditional
Xnew = np.linspace(-1, 2, 50)[:, None]

with model:
fcond = gp.conditional("fcond", Xnew=Xnew)

conditional(name, Xnew, pred_noise=False, given=None, **kwargs)

Returns the approximate conditional distribution of the GP evaluated over new input locations Xnew.

Parameters
name: string

Name of the random variable

Xnew: array-like

Function input values. If one-dimensional, must be a column vector with shape (n, 1).

pred_noise: bool

Whether or not observation noise is included in the conditional. Default is False.

given: dict

Can optionally take as key value pairs: X, Xu, y, noise, and gp. See the section in the documentation on additive GP models in PyMC3 for more information.

**kwargs

Extra keyword arguments that are passed to MvNormal distribution constructor.

marginal_likelihood(name, X, Xu, y, noise=None, is_observed=True, **kwargs)

Returns the approximate marginal likelihood distribution, given the input locations X, inducing point locations Xu, data y, and white noise standard deviations sigma.

Parameters
name: string

Name of the random variable

X: array-like

Function input values. If one-dimensional, must be a column vector with shape (n, 1).

Xu: array-like

The inducing points. Must have the same number of columns as X.

y: array-like

Data that is the sum of the function with the GP prior and Gaussian noise. Must have shape (n, ).

noise: scalar, Variable

Standard deviation of the Gaussian noise.

is_observed: bool

Whether to set y as an observed variable in the model. Default is True.

**kwargs

Extra keyword arguments that are passed to MvNormal distribution constructor.

class pymc3.gp.gp.TP(mean_func=<pymc3.gp.mean.Zero object>, cov_func=<pymc3.gp.cov.Constant object>, nu=None)

Student’s T process prior.

The usage is nearly identical to that of gp.Latent. The differences are that it must be initialized with a degrees of freedom parameter, and TP is not additive. Given a mean and covariance function, and a degrees of freedom parameter, the function $$f(x)$$ is modeled as,

$\begin{split}f(X) \sim \mathcal{TP}\left( \mu(X), k(X, X'), \\nu \\right)\end{split}$
Parameters
cov_funcNone, 2D array, or instance of Covariance

The covariance function. Defaults to zero.

mean_funcNone, instance of Mean

The mean function. Defaults to zero.

nufloat

The degrees of freedom

References

• Shah, A., Wilson, A. G., and Ghahramani, Z. (2014). Student-t Processes as Alternatives to Gaussian Processes. arXiv preprint arXiv:1402.4306.

conditional(name, Xnew, **kwargs)

Returns the conditional distribution evaluated over new input locations Xnew.

Given a set of function values f that the TP prior was over, the conditional distribution over a set of new points, f_* is

Parameters
name: string

Name of the random variable

Xnew: array-like

Function input values.

**kwargs

Extra keyword arguments that are passed to MvNormal distribution constructor.

prior(name, X, reparameterize=True, **kwargs)

Returns the TP prior distribution evaluated over the input locations X.

This is the prior probability over the space of functions described by its mean and covariance function.

Parameters
name: string

Name of the random variable

X: array-like

Function input values.

reparameterize: bool

Reparameterize the distribution by rotating the random variable by the Cholesky factor of the covariance matrix.

**kwargs

Extra keyword arguments that are passed to distribution constructor.