CS 294 / Statistics 242B

CS 294 / Statistics 242
Prof. Michael I. Jordan
Office hours: T 1-2 (401 Evans); F 2-3 (739 Soda)

This course, a continuation of this fall's CS 294/Stat 242A, will focus on nonparametric methods. Examples that will be treated in detail include wavelets, splines, neural networks and support vector machines. We will also discuss theoretical concepts to support such architectures, including regularization, VC dimension and minimum description length, as well as tools for model selection and model averaging.

The prerequisites for this course include previous coursework in linear algebra, multivariate calculus, probability and statistics. Students will need to be familiar with Matlab, SPlus or a related matrix-oriented programming language.

The course will be a lecture course. The outline of topics is as follows:

  • Examples of nonparametric learners
    kernel methods
    wavelets
    splines
    neural networks
    decision trees
    nearest neighbor
    support vector machines
  • Regularization
  • Capacity concepts
    annealed entropy
    growth functions and VC dimension
  • Structural risk minimization
  • Information theoretic approaches
    AIC, TIC, etc.
    minimum description length
  • Resampling methods
    bagging and boosting

    Students will be required to complete bi-weekly homework assignments. These must be turned in on time to receive credit. Also, each student will carry out an independent project in an area subject to the instructor's approval.

    Prof. Michael I. Jordan
    jordan@{cs,stat}.berkeley.edu