CS281a - Statistical Learning Theory

Statistical Learning Theory
CS281A/STAT241A

Instructor: Ben Recht
Time: TuTh 12:30-2:00 PM
Location: 277 Cory Hall

Office Hours: M 1:30-2:30, T 2:00-3:00.
Location: 726 Sutardja Dai Hall

GSIs:

Nick Boyd, 305 Evans, Office Hours W 10-11, Th 2:30-3:30
Orianna Demasi, 562 ``Spruce,'' Aspire Lab, Soda Hall, T 10-11, Th 3:30-4:30
Hongwei Li, 444 Evans, Office Hours T 3:30-4:30, F 1-2
Recitations, Tu 5-6:30, Soda 306

Description: This course is a 3-unit course that provides an introduction to statistical inference. Key issues to be addressed are how we reason about probabilistic models and the computational considerations of probabilistic inference. The primary focus of the course is on theoretical and methodological aspects of probabilistic decision making. We will draw connections to application areas including statistical machine learning, signal processing, computer vision, natural language processing, neuroscience, communication theory, and computational biology. Topics will include

Detection Theory, likelihood ratio tests, operating characteristics
Estimation Theory, maximum likelihood, Bayesean
Classification and Regression
Nonlinear optimization for inference, batch and incremental methods
Filtering and smoothing
Hidden Markov Models
Graphical Models, the junction tree framework
exponential families
generalized linear models
factor models
model selection
regularization

Required background: The prerequisites are previous coursework in linear algebra, multivariate calculus, basic probability and statistics. Some degree of mathematical maturity is also required. Coursework or background in control theory, optimization theory, and harmonic analysis is relevant, and could be helpful but is not required. Familiarity with a numerically oriented programming language (e.g., Python, Julia, MATLAB, etc.) will be helpful.

Grading: Students will be evaluated based on a combination of regular homework assignments (60%), a take-home midterm exam (20%), and a final course project (20%). Homework will be assigned bi-weekly and will be due the following week.

Homework: Homework assignments will be distributed on the bCourses site. Although it is acceptable for students to discuss the homework assignments with one another, each student must write up his/her homework on an individual basis. Each student must indicate with whom (if anyone) they discussed the homework problems. Homeworks must be turned in at the beginning of class on the due date. Hardcopies must be turned in. Do not submit the homework on bCourses. Late homeworks will not be accepted.

Midterm: The midterm will be handed out at 5PM on October 29 and due at 5PM on October 30. Students must work on this midterm alone.

Course project: The course project will involve independent work on a topic of the student's own choosing. Course projects will be presented in an informal poster session at the end of semester, and the work will be summarized in a write-up. The poster presentations will be during the final week of class (either on the 3rd, 4th or 5th, depending on room availability).

Course Reading:

Course reading will be made available on the bCourses site for this class. This will include chapters from An Introduction to Probabilistic Graphical Models by Michael Jordan.

Lectutre 1 (8/29): Introduction and math review.

Homeworks

Problem Set 1. Due in class on September 4.

Problem Set 2. Due in class on September 18.