STATS/DATASCI 451 Fall 2022
Bayesian Data Analysis
Overview
The course is an introduction to both principles and practice of Bayesian inference for data analysis. We will focus on building probabilistic models, algorithms for approximate Bayesian inference, and methods for checking, criticizing, and revising models. Some of the models we will study include classic Bayesian mixture and regression models, hierarchical models, factor models, topic models, and deep generative models. Alongside these models we will study algorithms for approximate Bayesian inference including Markov Chain Monte Carlo and variational inference algorithms. Finally, we will discuss methods for checking, criticizing, and revising models in an iterative manner, completing a virtuous cycle of applied Bayesian statistics.
At the end of this course students will be familiar with the Bayesian paradigm, and will be able to analyze different classes of statistical models. The course gives an introduction to the computational tools needed for Bayesian data analysis and develops statistical modeling skills through a hands-on data analysis approach.
Syllabus
For course policies, course requirements, and grading policies, please see the syllabus [link].
Piazza
Students should sign up Piazza [link] to join course discussions.
All communications with the teaching team (the instructor and the GSIs) should be conducted over Piazza; please do not email. If you'd like to reach the instructor or the GSIs for private questions, please post a private note on Piazza that is only visible to the instructor and the GSIs. See here for detailed instructions. The GSIs and the instructor will be monitoring piazza, endorsing correct student answers, and answering questions that remain after a discussion.
As a bonus, up to 3 percentage points will be added to your final course grade based on piazza participation. You will get (3x/100) bonus percentage points if the number of your total Piazza contributions lie in the top x-th quantile among all students. The number of Piazza contributions will be determined by Piazza class statistics.
Teaching Team and Office Hours
- Instructor: Yixin Wang
- Office Hour: Weekly Office Hours.
- GSI: Unique Subedi
- Office Hour: Mondays 2-3:30 pm (Angel Hall G219) and Fridays 3-4:30 pm (zoom)
- Lecture: Tue/Thur 11:30am-12:50pm
- Location: Weiser 260
- Google Calendar: The Google Calendar below ideally contains all events and deadlines for student's convenience. Please feel free to add this calendar to your Google Calendar by clicking on the plus (+) button on the bottom right corner of the calendar below. Any adhoc changes to the schedule will be visible on the calendar first.
Course Calendar
Lecture Schedule
The Schedule is subject to change.
By each date, please read about the topic at hand; please choose one reading from the list for the topic.
BDA = Bayesian Data Analysis by Gelman [link] PML = Probabilistic Machine Learning: Advanced Topics by Murphy [link] PRML = Pattern Recognition and Machine Learning by Bishop [link] SR = Statistical Rethinking by McElreath [link]| Date | Topic | Readings | ||
|---|---|---|---|---|
Lecture 1 |
08/30 |
Introduction and the "Box's Loop" |
"Build, Compute, Critique, Repeat: Data Analysis with Latent Variable Models" (Blei, 2014) Python Tutorial |
|
Lecture 2 |
09/01 |
Probability: A Review of Basic Concepts and Bayes’ Theorem |
"Review of Probability" (Blei, 2016) BDA, Chap. 1 SR, Sec. 2.1-2 |
|
Lecture 3 |
09/06 |
The Ingredients of Probabilistic Models I |
"The Basics of Graphical Models" (Blei, 2016) "Statistical Concepts" (Blei, 2016) BDA, Sec. 2.1-3 SR, Sec. 2.3-5 and Chap. 3 PRML, Sec. 1.2-3 "Model-based Machine Learning" (Bishop, 2013) |
|
Lecture 4 |
09/08 |
The Ingredients of Probabilistic Models II |
'' |
|
Lecture 5 |
09/13 |
The Exchangeable Data Model and Conjugate Priors I |
Sec. 4 of "The Exponential Family" (Blei, 2016) PML, Sec 3.1-5 BDA, Sec. 2.4-9 |
|
Lecture 6 |
09/15 |
The Exchangeable Data Model and Conjugate Priors II |
P'' |
|
Lecture 7 |
09/20 |
Evaluating Probabilistic Models I |
"Posterior Predictive Checks" (Blei, 2011) BDA, Sec. 6.1-5 PML, Sec. 3.9 |
|
Lecture 8 |
09/22 |
Evaluating Probabilistic Models II |
'' |
|
Lecture 9 |
09/27 |
Going through the Box's Loop I |
BDA, Sec 6.2 |
|
Lecture 10 |
09/29 |
Going through the Box's Loop II |
'' | |
Lecture 11 |
10/04 |
Going through the Box's Loop III |
'' |
|
Lecture 12 |
10/06 |
Going through the Box's Loop IV | '' |
|
Lecture 13 |
10/11 |
Conditional Models: Linear and Logistic Regression I | Sec. 1-2 of "Linear regression, Logistic regression, and Generalized Linear Models" (Blei, 2014) BDA, Chap. 14 PML, Sec. 15.1-2 SR, Chap. 4 |
|
Lecture 14 |
10/13 |
Conditional Models: Linear and Logistic Regression II |
'' |
|
Fall break |
10/18 |
------------ |
------------ |
|
Lecture 15 |
10/20 |
Conditional Models: Linear and Logistic Regression III |
'' |
|
Lecture 16 |
10/25 |
Conditional Models: Linear and Logistic Regression IV |
'' |
|
Lecture 17 |
10/27 |
Conditional Models: Linear and Logistic Regression V | '' |
|
Lecture 18 |
11/01 |
Bayesian Mixture Models and an Introduction to Markov Chain Monte Carlo I |
"Bayesian Mixture Models and the Gibbs Sampler" (Blei, 2016) BDA, Chap. 22 PML, Sec. 12.1-3 SR, Chap. 9 PRML, Chap. 11 "Identifying Bayesian Mixture Models" (Betancourt, 2018) |
|
Lecture 19 |
11/03 |
Bayesian Mixture Models and an Introduction to Markov Chain Monte Carlo II |
'' |
|
Lecture 20 |
11/08 |
Bayesian Mixture Models and an Introduction to Markov Chain Monte Carlo III |
'' |
|
Lecture 21 |
11/10 |
Bayesian Mixture Models and an Introduction to Markov Chain Monte Carlo IV |
'' |
|
Lecture 22 |
11/15 |
Bayesian Mixture Models and an Introduction to Markov Chain Monte Carlo V |
'' |
|
Lecture 23 |
11/17 |
Bayesian Mixture Models and an Introduction to Markov Chain Monte Carlo VI |
'' |
|
Lecture 24 |
11/22 |
Bayesian Mixture Models and an Introduction to Markov Chain Monte Carlo VII |
'' |
|
Thanksgiving break |
11/24 |
------------ |
------------ |
|
Lecture 25 |
11/29 |
Mixed-Membership Models and an Introduction to Variational Inference I |
"Mixed-membership Models (and an Introduction to Variational Inference)" (Blei, 2016)
[Video Tutorial] Variational Inference: Foundations and Innovations (Blei, 2019) (Part 1) [slides]
PML, Sec. 28.5 |
|
Lecture 26 |
12/01 |
Mixed-Membership Models and an Introduction to Variational Inference II |
'' |
|
Lecture 27 |
12/06 |
Mixed-Membership Models and an Introduction to Variational Inference III |
'' |
|
Lecture 28 |
12/08 |
Summary (and wiggle room) |
------------ |
Final Project
The final project is an individual project. For requirements of the final project, please see the final project guidelines. The LaTeX template for the project report is here.