Welcome to PyMC3’s documentation!

PyMC3 is a Python package for Bayesian statistical modeling and Probabilistic Machine Learning focusing on advanced Markov chain Monte Carlo (MCMC) and variational inference (VI) algorithms. Its flexibility and extensibility make it applicable to a large suite of problems.

Check out the getting started guide, or interact with live examples using Binder!

Features

• Intuitive model specification syntax, for example, x ~ N(0,1) translates to x = Normal('x',0,1)
• Powerful sampling algorithms, such as the No U-Turn Sampler, allow complex models with thousands of parameters with little specialized knowledge of fitting algorithms.
• Variational inference: ADVI for fast approximate posterior estimation as well as mini-batch ADVI for large data sets.

• Relies on Theano which provides:

  • Computation optimization and dynamic C compilation
  • Numpy broadcasting and advanced indexing
  • Linear algebra operators
  • Simple extensibility

• Transparent support for missing value imputation

Getting started

If you already know about Bayesian statistics:

• API quickstart guide
• The PyMC3 tutorial
• PyMC3 examples and the API reference

Learn Bayesian statistics with a book together with PyMC3:

• Probabilistic Programming and Bayesian Methods for Hackers: Fantastic book with many applied code examples.
• PyMC3 port of the book “Doing Bayesian Data Analysis” by John Kruschke as well as the second edition: Principled introduction to Bayesian data analysis.
• PyMC3 port of the book “Statistical Rethinking A Bayesian Course with Examples in R and Stan” by Richard McElreath
• PyMC3 port of the book “Bayesian Cognitive Modeling” by Michael Lee and EJ Wagenmakers: Focused on using Bayesian statistics in cognitive modeling.
• Bayesian Analysis with Python by Osvaldo Martin (and errata): Great introductory book.

PyMC3 talks

There are also several talks on PyMC3 which are gathered in this YouTube playlist

Installation

The latest release of PyMC3 can be installed from PyPI using pip:

pip install pymc3

Note: Running pip install pymc will install PyMC 2.3, not PyMC3, from PyPI.

Or via conda-forge:

conda install -c conda-forge pymc3

The current development branch of PyMC3 can be installed from GitHub, also using pip:

pip install git+https://github.com/pymc-devs/pymc3

To ensure the development branch of Theano is installed alongside PyMC3 (recommended), you can install PyMC3 using the requirements.txt file. This requires cloning the repository to your computer:

git clone https://github.com/pymc-devs/pymc3
cd pymc3
pip install -r requirements.txt

However, if a recent version of Theano has already been installed on your system, you can install PyMC3 directly from GitHub.

Another option is to clone the repository and install PyMC3 using python setup.py install or python setup.py develop.

Dependencies

PyMC3 is tested on Python 2.7 and 3.6 and depends on Theano, NumPy, SciPy, Pandas, and Matplotlib (see requirements.txt for version information).

Optional

In addtion to the above dependencies, the GLM submodule relies on Patsy.

Citing PyMC3

Salvatier J., Wiecki T.V., Fonnesbeck C. (2016) Probabilistic programming in Python using PyMC3. PeerJ Computer Science 2:e55 DOI: 10.7717/peerj-cs.55.

Contact

We are using discourse.pymc.io as our main communication channel. You can also follow us on Twitter @pymc_devs for updates and other announcements.

To ask a question regarding modeling or usage of PyMC3 we encourage posting to our Discourse forum under the “Questions” Category. You can also suggest feature in the “Development” Category.

To report an issue with PyMC3 please use the issue tracker.

Finally, if you need to get in touch for non-technical information about the project, send us an e-mail.

License

Apache License, Version 2.0

Software using PyMC3

• Bambi: BAyesian Model-Building Interface (BAMBI) in Python.
• pymc3_models: Custom PyMC3 models built on top of the scikit-learn API.
• PMProphet: PyMC3 port of Facebook’s Prophet model for timeseries modeling
• webmc3: A web interface for exploring PyMC3 traces
• sampled: Decorator for PyMC3 models.
• NiPyMC: Bayesian mixed-effects modeling of fMRI data in Python.
• beat: Bayesian Earthquake Analysis Tool.
• BayesFit: Bayesian Psychometric Curve Fitting Tool.

Please contact us if your software is not listed here.

Papers citing PyMC3

See Google Scholar for a continuously updated list.

Contributors

See the GitHub contributor page

Support

PyMC3 is a non-profit project under NumFOCUS umbrella. If you want to support PyMC3 financially, you can donate here.

Sponsors

Contents:

• Introduction
  • Purpose
  • Features
  • What’s new in version 3
  • History
  • Usage Overview
• Getting started
  • Getting started with PyMC3
    • Abstract
    • Introduction
    • Installation
    • A Motivating Example: Linear Regression
    • Case study 1: Stochastic volatility
    • Case study 2: Coal mining disasters
    • Arbitrary deterministics
    • Arbitrary distributions
    • Generalized Linear Models
    • Discussion
    • References
  • API quickstart
    • 1. Model creation
    • 2. Probability Distributions
    • 3. Inference
    • 4. Posterior Predictive Sampling
    • 4.1 Predicting on hold-out data
  • Variational API quickstart
    • Basic setup
    • Checking convergence
    • Tracking parameters
    • Multilabel logistic regression
    • Minibatches
  • Gaussian Processes
    • GP Basics
    • Mean and covariance functions
    • GP Implementations
    • Additive GPs
  • PyMC3 and Theano
    • What is Theano
    • How PyMC3 uses Theano
  • Advanced usage of Theano in PyMC3
    • Using shared variables
    • Writing custom Theano Ops
• Probability Distributions
  • Custom distributions
  • Using PyMC distributions without a Model
  • Auto-transformation
• Examples
  • Howto
    • Sampler statistics
    • Diagnosing Biased Inference with Divergences
    • Posterior Predictive Checks
    • Model comparison
    • Model averaging
    • Bayes Factors and Marginal Likelihood
    • How to debug a model
    • PyMC3 Modeling tips and heuristic
    • LKJ Cholesky Covariance Priors for Multivariate Normal Models
    • Live sample plots
    • Compound Steps in Sampling
    • Using a “black box” likelihood function
  • Applied
    • Bayesian Estimation Supersedes the T-Test
    • A Primer on Bayesian Methods for Multilevel Modeling
    • Stochastic Volatility model
    • A Hierarchical model for Rugby prediction
    • The Dawid-Skene model with priors
  • GLM
    • GLM: Linear regression
    • GLM: Robust Linear Regression
    • GLM: Robust Regression with Outlier Detection
    • GLM: Model Selection
    • Rolling Regression
    • GLM: Hierarchical Linear Regression
    • GLM: Poisson Regression
    • Hierarchical Partial Pooling
    • GLM: Negative Binomial Regression
    • Hierarchical Binominal Model: Rat Tumor Example
  • Survival Analysis
    • Bayesian Survival Analysis
    • Bayesian Parametric Survival Analysis with PyMC3
    • Reparameterizing the Weibull Accelerated Failure Time Model
    • Censored Data Models
  • Gaussian Processes
    • Mean and Covariance Functions
    • Marginal Likelihood Implementation
    • Latent Variable Implementation
    • Sparse Approximations
    • Kronecker Structured Covariances
    • Student-t Process
    • Example: CO2 at Mauna Loa
    • Gaussian Process Regression and Classification with Elliptical Slice Sampling
    • Gaussian Process (GP) smoothing
  • Mixture Models
    • Gaussian Mixture Model
    • Marginalized Gaussian Mixture Model
    • Gaussian Mixture Model with ADVI
    • Dirichlet process mixtures for density estimation
    • Dependent density regression
  • Variational Inference
    • GLM: Mini-batch ADVI on hierarchical regression model
    • Automatic autoencoding variational Bayes for latent dirichlet allocation with PyMC3
    • Variational Inference: Bayesian Neural Networks
    • Convolutional variational autoencoder with PyMC3 and Keras
    • Empirical Approximation overview
    • Normalizing Flows Overview
  • Stochastic Gradient
    • Constant Stochastic Gradient
    • Stochastic Gradient Fisher Scoring
    • Bayesian Neural Networks in PyMC3 with Stochastic Gradient Algorithms
• API Reference
  • Distributions
    • Continuous
    • Discrete
    • Multivariate
    • Mixture
    • Timeseries
  • Bounded Variables
    • Usage
    • Caveats
    • API
  • Inference
    • Sampling
    • Step-methods
    • Variational
  • Generalized Linear Models
  • Gaussian Processes
    • Implementations
    • Mean Functions
    • Covariance Functions
  • Plots
  • Stats
  • Diagnostics
  • Backends
    • Available backends
    • Selecting a backend
    • Selecting values from a backend
    • Loading a saved backend
    • Writing custom backends
    • ndarray
    • sqlite
    • text
    • tracetab
  • Math
  • Data
  • Model

Indices and tables

• Index
• Module Index
• Search Page