STATS/DATASCI 451 Fall 2023
Bayesian Data Analysis
Overview
The course is an introduction to both principles and practice of Bayesian inference for data analysis. We will focus on building probabilistic models, algorithms for approximate Bayesian inference, and methods for checking, criticizing, and revising models. Some of the models we will study include classic Bayesian mixture and regression models, hierarchical models, factor models, topic models, and deep generative models. Alongside these models we will study algorithms for approximate Bayesian inference including Markov Chain Monte Carlo and variational inference algorithms. Finally, we will discuss methods for checking, criticizing, and revising models in an iterative manner, completing a virtuous cycle of applied Bayesian statistics.
At the end of this course students will be familiar with the Bayesian paradigm, and will be able to analyze different classes of statistical models. The course gives an introduction to the computational tools needed for Bayesian data analysis and develops statistical modeling skills through a hands-on data analysis approach.
Syllabus
For course policies, course requirements, and grading policies, please see the syllabus [link].
Piazza
Students should sign up Piazza [link] to join course discussions.
All communications with the teaching team (the instructor and the GSIs) should be conducted over Piazza; please do not email. If you'd like to reach the instructor or the GSIs for private questions, please post a private note on Piazza that is only visible to the instructor and the GSIs. See here for detailed instructions. The GSIs and the instructor will be monitoring piazza, endorsing correct student answers, and answering questions that remain after a discussion.
As a bonus, up to 3 percentage points will be added to your final course grade based on piazza participation. You will get (3x/100) bonus percentage points if the number of your total Piazza contributions is (x * 100)% of the maximum number of contributions among all students. The number of Piazza contributions will be determined by Piazza class statistics.
Teaching Team and Office Hours
- Instructor: Yixin Wang
- Office Hour: Weekly Office Hours.
- GSI:
- Zhiwei Xu (Office Hour: Monday 4:30-5:30pm virtual, Wednesday 2:20-3:50pm in person, Friday 9:00-9:30am virtual)
- Akhil Goel (Office Hour: Tuesdays: 4:30-6:00pm in person, Thursdays: 9-10:30am virtual)
- Lecture: Tue/Thur 11:30am-12:50pm
- Location: Auditorium D Angel Hall
- Google Calendar: The Google Calendar below ideally contains all events and deadlines for student's convenience. Please feel free to add this calendar to your Google Calendar by clicking on the plus (+) button on the bottom right corner of the calendar below. Any adhoc changes to the schedule will be visible on the calendar first.
Course Calendar
Lecture Schedule
The Schedule is subject to change.
By each date, please read about the topic at hand; please choose one reading from the list for the topic.
Kruschke = Doing Bayesian Data Analysis [link] BDA = Bayesian Data Analysis by Gelman [link] PML = Probabilistic Machine Learning: Advanced Topics by Murphy [link] PRML = Pattern Recognition and Machine Learning by Bishop [link] SR = Statistical Rethinking by McElreath [link]| Date | Topic | Readings | ||
|---|---|---|---|---|
Lecture 1 |
08/29 |
Introduction I |
Kruschke ch. 2 "Bayesian data analysis for newcomers" (Kruschke and Liddel, 2018) |
|
Lecture 2 |
08/31 |
Probability: A Review of Basic Concepts and Bayes’ Theorem I |
Kruschke ch. 4 | |
Lecture 3 |
09/05 |
Probability: A Review of Basic Concepts and Bayes’ Theorem II |
'' | |
Lecture 4 |
09/07 |
The Basics of Bayesian Statistics I |
Kruschke ch. 5 | |
Lecture 5 |
09/12 |
The Basics of Bayesian Statistics II |
'' |
|
Lecture 6 |
09/14 |
The Basics of Bayesian Statistics III |
'' |
|
Lecture 7 |
09/19 |
The Beta-Binomial Model and the Bayesian Workflow I |
Kruschke ch. 6 "Posterior Predictive Checks" (Blei, 2011) | |
Lecture 8 |
09/21 |
The Beta-Binomial Model and the Bayesian Workflow II |
'' |
|
Lecture 9 |
09/26 |
The Exchangeable Data Model and Conjugate Priors I |
BDA, Sec. 2.4-9 |
|
Lecture 10 |
09/28 |
The Exchangeable Data Model and Conjugate Priors II |
'' | |
Lecture 11 |
10/03 |
The Exchangeable Data Model and Conjugate Priors III |
'' |
|
Lecture 12 |
10/05 |
The Exchangeable Data Model and Conjugate Priors IV | '' |
|
Lecture 13 |
10/10 |
Bayesian Computation and an Introduction to Stan I |
Kruschke ch. 7, 14 |
|
Lecture 14 |
10/12 |
Midterm Review |
'' |
|
Fall break |
10/17 |
------------ |
------------ |
|
Lecture 15 |
10/19 |
Midterm Exam |
------------ |
|
Lecture 16 |
10/24 |
Bayesian Computation and an Introduction to Stan II |
'' |
|
Lecture 17 |
10/26 |
Bayesian Computation and an Introduction to Stan III | '' |
|
Lecture 18 |
10/31 |
Grouped Data and Hierarchical Models I |
Kruschke ch. 9 Kruschke ch. 12.1, 16 |
|
Lecture 19 |
11/02 |
Grouped Data and Hierarchical Models II |
'' |
|
Lecture 20 |
11/07 |
Group Comparisons |
'' |
|
Lecture 21 |
11/09 |
Conditional Models: Linear and Logistic Regression I |
Kruschke ch. 15, 17 |
|
Lecture 22 |
11/14 |
Conditional Models: Linear and Logistic Regression II |
'' | |
Lecture 23 |
11/16 |
Conditional Models: Linear and Logistic Regression III; Model Comparisons |
McElreath ch. 7 |
|
Lecture 24 |
11/21 |
Introduction to Markov Chain Monte Carlo I |
BDA, Chap. 22 "Bayesian Mixture Models and the Gibbs Sampler" (Blei, 2016) Kruschke ch. 7, 14 |
|
Thanksgiving break |
11/23 |
------------ |
------------ |
|
Lecture 25 |
11/28 |
Introduction to Markov Chain Monte Carlo II |
'' |
|
Lecture 26 |
11/30 |
Introduction to Markov Chain Monte Carlo III |
'' |
|
Lecture 27 |
12/05 |
Variational Inference; Summary (and wiggle room) |
[Video Tutorial] Variational Inference: Foundations and Innovations (Blei, 2019) (Part 1) [slides] |