Syllabus---CS 281A / Stat 241A (Fall, 2002)


Course Description:

This course will provide a thorough grounding in probabilistic and computational methods for the statistical modeling of complex, multivariate data. The emphasis will be on the unifying framework provided by graphical models, a formalism that merges aspects of graph theory and probability theory.

Outline:

  • Basics on graphical models, Markov properties, recursive decomposability, elimination algorithms
  • Sum-product algorithm, factor graphs, semi-rings
  • Frequentist and Bayesian methods
  • Bayesian classification, linear models and generalized linear (GLIM) models, on-line methods
  • Exponential family, sufficiency, conjugacy
  • Density estimation, kernel methods, mixture models
  • The EM algorithm
  • Conditional mixture models, hierarchical mixture models
  • Hidden Markov models (HMM)
  • Factor analysis, principal component analysis (PCA), canonical correlation analysis (CCA), independent component analysis (ICA)
  • Kalman filtering and Rauch-Tung-Striebel smoothing
  • Markov properties of graphical models
  • Junction tree algorithm
  • Chains, trees, factorial models, coupled models, layered models
  • Importance sampling, Markov chain Monte Carlo (MCMC), simulated annealing
  • Free energy, convexity-based variational algorithms, belief propagation
  • Dynamic graphical models
  • Model selection, marginal likelihood, AIC, BIC and MDL
  • Decision networks, Markov decision processes and reinforcement learning
  • Applications to error-control coding, bioinformatics, speech and language, vision