CS 294-10, Fall 2005, Reading List
Week 1 (9/2): Transfer Learning
Required reading
- Effective Bayesian Transfer Learning, Berkeley/Stanford/MIT/Oregon State DARPA Proposal, 2005 (hardcopy).
Ideas for how to achieve transfer learning; evaluation methods; application domains.
Week 2 (9/9): Hierarchical Bayes
Required reading
- Christian Robert, The Bayesian Choice, 2nd Edition, Chapter 10, Springer, 2001 (hardcopy).
Basics of hierarchical Bayesian models.
Supplemental reading
- Brani Vidakovic, lecture handouts 1-8.
[available here]
Handouts 1-7 provide basic preparation. Handout 8 deals specifically with hierarchical Bayes.
- William DuMouchel, Hierarchical Bayes Linear Models for Meta-Analysis.
Technical Report Number 27, National Institute of Statistical Sciences, 1994.
[pdf]
- Andrew Gelman, ``Multilevel (hierarchical) modeling: what it can and can't do.''
Technometrics, to appear, 2005
[pdf]
Week 3 (9/16): Variable selection in hierarchical Bayesian models
Required reading
- Ed George and Robert McCulloch (1993).
"Variable selection by Gibbs sampling."
JASA, 88, 881-889.
[pdf]
- Xinlei Wang and Edward I. George (2004).
A Hierarchical Bayes Approach to Variable Selection for Generalized Linear
Models.
Technical report SMU-TR-321, Department of Statistics, Southern Methodist
University.
[pdf]
- J.E. Griffin and P.J. Brown (2005).
Alternative prior distributions for variable selection with very many more
variables than observations.
Technical report, Dept. of Statistics, University of Warwick.
[pdf]
Week 4 (9/23): Nonparametric Bayesian approaches
Required reading
- Y. W. Teh, M. I. Jordan, M. J. Beal and D. M. Blei. (2005).
Hierarchical Dirichlet processes. In Advances in Neural Information Processing Systems (NIPS) 17, 2005.
[pdf]
This is the short version a longer technical report. You may find it necessary to consult the
longer paper for background and explanations.
- MacEachern, S.N. (1999). Dependent Nonparametric Processes. In ASA Proceedings of the
Section on Bayesian Statistical Science, Alexandria, VA: American Statistical Association,
pp. 50-55.
[pdf]
Supplemental reading
- MacEachern, S., Kottas, A., and Gelfand, A. (2001).
Spatial Nonparametric Bayesian Models.
Technical Report 01-10, Institute of Statistics and Decision Sciences, Duke University.
[pdf]
Week 5 (9/30): Statistical learning theory for transfer learning
Required reading
- Rie K. Ando and Tong Zhang (2004). A Framework for Learning Predictive
Structures from Multiple Tasks and Unlabeled Data. Technical Report RC23462,
IBM T.J. Watson Research Center.
[pdf]
Week 6 (10/7): Learning theory contd.; transfer learning algorithms
Required reading
- Jonathan Baxter (2000). A model of inductive bias learning. JAIR,
12, 149-198.
[pdf]
- Rich Caruana (1997). Multitask learning. Machine Learning, 28, 41-75.
[pdf]
- Sebastian Thrun (1996). Is learning the nth thing any easier than learning the
first? In Advances in NIPS, 640-646.
[pdf]
- Nathan Intrator and Shimon Edelman (1998).
Making a low-dimensional representation
suitable for diverse tasks. In Learning to learn. Kluwer.
[pdf]
Week 7 (10/14):
Required reading
- L. Getoor, N. Friedman, D. Koller, and A. Pfeffer, ``Learning Probabilistic Relational Models.''
In S. Dzeroski and N. Lavrac, Eds., Relational Data Mining, Springer-Verlag, 2001.
[pdf]
Probably the most readable introduction to
RPMs appears in Section 1.4 of this paper.
- N. Friedman, L. Getoor, D. Koller and A. Pfeffer, ``Learning Probabilistic Relational Models.''
In Proc. IJCAI-99, 1999.
[pdf]
Section 4.1 of this paper includes the
general analysis of acyclicity conditions for RPMs, missing from the
book chapter version above.
- Brian Milch, Bhaskara Marthi, David Sontag, Stuart Russell, Daniel L. Ong and
Andrey Kolobov, ``BLOG: Probabilistic Models with Unknown Objects.''
In Proc. IJCAI-05, 2005.
[pdf]
Eliminates the unique-names and
domain-closure assumptions of RPMs so that domains with unknown
objects and identity uncertainty can be handled.
- Peter Carbonetto, Jacek Kisynski, Nando de Freitas and David
Poole. Nonparametric Bayesian Logic . UAI 2005.
[pdf]
Combines BLOG with
nonparametric Bayesian modelling.
Week 8 (10/21):
No meeting
Background reading
- Chapters 17 and 21 of Artificial Intelligence: A Modern
Approach, second edition. Stuart Russell and Peter Norvig,
Prentice Hall, 2003.
In S. Dzeroski and N. Lavrac, Eds., Relational Data Mining, Springer-Verlag, 2001.
[pdf]
Week 9 (10/28):
Required reading
- Prasad Tadepalli, Robert Givan, and Kurt Driessens (2004).
Relational Reinforcement Learning: An Overview.
In Proc. ICML-04 Workshop on Relational Reinforcement Learning.
[pdf]
Brief historical survey of RRL
motivation and methods.
- S. Dzeroski, L. De Raedt, and K. Driessens (2001).
Relational reinforcement learning.
Machine Learning, 43, 7-52.
[pdf]
The first substantial paper to
investigate the use of relational representations for
value functions and policies.
- Craig Boutilier, Ray Reiter and Bob Price (2001).
Symbolic Dynamic Programming for First-order MDPs.
In Proc. Seventeenth International Joint Conference on Artificial Intelligence (IJCAI-01), Seattle, pp.690--697.
[pdf]
Shows how to apply value and policy
iteration directly to relational representations of values and
policies, given a relational transition model.
Week 10 (11/4):
Required reading
- Dietterich, T. (2000). Hierarchical reinforcement learning with the MAXQ
value function decomposition. JAIR, 13, 227-303.
[pdf]
Introduces the idea of temporal
decomposition of value functions into additive components
associated with separate activities, and the idea of "safe state
abstraction" for representing value function components in
low-dimensional subspaces of the full state--action space.
- Bhaskara Marthi, Stuart Russell, David Latham, and Carlos Guestrin,
``Concurrent hierarchical reinforcement learning.''
In Proc. IJCAI-05, Edinburgh, Scotland, 2005.
[pdf]
Discusses single-threaded and
multi-threaded partial programming languages for imposing
hierarchical constraints on policies; and introduces functional
deocmposition of value functions across threads. Draws on ideas
from the Guestrin et al. and Russell and Zimdars papers
given below.
Additional background reading
- Stuart Russell and Andrew Zimdars,
``Q-Decomposition for Reinforcement Learning Agents.''
In Proc. ICML-03, Washington, DC, 2003.
[pdf]
A very simple paper showing that
additive structure in the reward function allows for a
distributed but globally optimal learning process and a
"functionally" decomposed Q-function.
- Carlos Guestrin, Michail Lagoudakis and Ronald Parr (2002).
Coordinated Reinforcement Learning.
In Proc. Nineteenth International Conference on Machine Learning (ICML
2002), pp. 227 - 234, Sydney, Australia.
[pdf]
The relevant idea here is the use of an
efficient dynamic programming algorithm to select optimal actions
from very large action spaces, given a factored Q-function.
- Carlos Guestrin, Daphne Koller, Chris Gearhart and Neal Kanodia
(2003).
Generalizing Plans to New Environments in Relational MDPs.
In Proc. International Joint Conference on Artificial Intelligence,
Acapulco, Mexico.
[pdf]
Demonstrates transfer learning using
"object-oriented" representation of Q-function.
Week 11 (11/11):
No meeting (Veterans' Day)
Week 12 (11/18):
Required reading
- Theodoros Evgeniou, Charles Micchelli, and Massimiliano Pontil (2005).
Learning multiple tasks with kernel methods.
J. Machine Learning Research, 6: 615--637.
[pdf]
- Andreas Maurer (2005). Bounds for linear multi-task learning.
Technical report.
[pdf]
Additional background reading
- Ben-David, S. and Schuller, R. (2003). Exploiting task
relatedness for multiple task learning. In Proc COLT, pages 567-580.
[pdf]
- Theodoros Evgeniou and Massimiliano Pontil (2004).
Regularized multi--task learning.
In Proc. 17th SIGKDD Conf. on Knowledge Discovery and Data Mining.
[pdf]
A briefer presentation of some of the ideas in
the 2005 JMLR paper.
- Charles Micchelli and Massimiliano Pontil (2004).
Kernels for multi--task learning.
Proc.NIPS 18.
[pdf]
A briefer presentation of some of the
other ideas in the 2005 JMLR paper.
Week 13 (11/25):
No meeting (Thanksgiving)
Week 14 (12/2):
Required reading, all from NIPS 2005 Workshop on Transfer Learning
- Rajat Raina, Andrew Y. Ng, and Daphne Koller (2005).
Transfer Learning by constructing informative priors.
[pdf]
- Zvika Marx, Michael T. Rosenstein, and Leslie Pack Kaelbling (2005).
Transfer leraning with an ensemble of background tasks.
[pdf]
- Geremy Heitz, Gall Elidan, and Daphne Koller (2005).
Transfer Learning of Object Classes: From Cartoons to Photgraphs.
[pdf]
- Kai Yu and Volker Tresp (2005).
Learning to Learn and Collaborative Filtering.
[pdf]
- Neville Mehta, S.Natarajan, Prasad Tadepalli, and Alan Fern (2005).
Transfer in Variable-Reward Hierarchical Reinforcement Learning.
[pdf]
- Alex Niculescu-Mizil and Rich Caruana (2005).
Learning the Structure of Related Tasks.
[ps]