Diagnostics

Convergence diagnostics and model validation

pymc3.diagnostics.geweke(pymc3_obj, *args, **kwargs)

Return z-scores for convergence diagnostics.

Compare the mean of the first % of series with the mean of the last % of series. x is divided into a number of segments for which this difference is computed. If the series is converged, this score should oscillate between -1 and 1.

Parameters:
  • x (array-like) – The trace of some stochastic parameter.
  • first (float) – The fraction of series at the beginning of the trace.
  • last (float) – The fraction of series at the end to be compared with the section at the beginning.
  • intervals (int) – The number of segments.
Returns:

scores (list [[]]) – Return a list of [i, score], where i is the starting index for each interval and score the Geweke score on the interval.

Notes

The Geweke score on some series x is computed by:

\[\frac{E[x_s] - E[x_e]}{\sqrt{V[x_s] + V[x_e]}}\]

where \(E\) stands for the mean, \(V\) the variance, \(x_s\) a section at the start of the series and \(x_e\) a section at the end of the series.

References

Geweke (1992)

pymc3.diagnostics.gelman_rubin(mtrace, varnames=None, include_transformed=False)

Returns estimate of R for a set of traces.

The Gelman-Rubin diagnostic tests for lack of convergence by comparing the variance between multiple chains to the variance within each chain. If convergence has been achieved, the between-chain and within-chain variances should be identical. To be most effective in detecting evidence for nonconvergence, each chain should have been initialized to starting values that are dispersed relative to the target distribution.

Parameters:
  • mtrace (MultiTrace) – A MultiTrace object containing parallel traces (minimum 2) of one or more stochastic parameters.
  • varnames (list) – Names of variables to include in the rhat report
  • include_transformed (bool) – Flag for reporting automatically transformed variables in addition to original variables (defaults to False).
Returns:

Rhat (dict) – Returns dictionary of the potential scale reduction factors, \(\hat{R}\)

Notes

The diagnostic is computed by:

\[\hat{R} = \frac{\hat{V}}{W}\]

where \(W\) is the within-chain variance and \(\hat{V}\) is the posterior variance estimate for the pooled traces. This is the potential scale reduction factor, which converges to unity when each of the traces is a sample from the target posterior. Values greater than one indicate that one or more chains have not yet converged.

References

Brooks and Gelman (1998) Gelman and Rubin (1992)

pymc3.diagnostics.effective_n(mtrace, varnames=None, include_transformed=False)

Returns estimate of the effective sample size of a set of traces.

Parameters:
  • mtrace (MultiTrace) – A MultiTrace object containing parallel traces (minimum 2) of one or more stochastic parameters.
  • varnames (list) – Names of variables to include in the effective_n report
  • include_transformed (bool) – Flag for reporting automatically transformed variables in addition to original variables (defaults to False).
Returns:

n_eff (float) – Return the effective sample size, \(\hat{n}_{eff}\)

Notes

The diagnostic is computed by:

\[\hat{n}_{eff} = \frac{mn}{1 + 2 \sum_{t=1}^T \hat{\rho}_t}\]

where \(\hat{\rho}_t\) is the estimated autocorrelation at lag t, and T is the first odd positive integer for which the sum \(\hat{\rho}_{T+1} + \hat{\rho}_{T+1}\) is negative.

References

Gelman et al. (2014)