[Lecture slides]

References

• Isabelle Guyon, Andre Elisseeff. An Introduction to Variable and Feature Selection. 2003. Provides a general overview of feature selection.
• Tim Hesterberg, Nam Hee Choi, Lukas Meier, and Chris Fraley. Least angle and L1 penalized regression: A review. Statist. Surv. 2008.
• Galen Andrew , Jianfeng Gao. Scalable training of L1-regularized log-linear models. ICML 2007. Open source implementation here (see edu.berkeley.nlp.math.OW_LBFGSMinimizer).
• Isabelle Guyon, Constantin Aliferis, Andre Elisseeff. Causal Feature Selection. 2007.
• Jianfeng Gao, Galen Andrew, Mark Johnson, Kristina Toutanova. Comparative Study of Parameter Estimation Methods for Statistical Natural Language Processing. ACL 2007. Compares many different methods including ordinary logistic regression, logistic regression with L1 regularization (to get sparsity in features), and boosting (another way to get sparsity).
• Bradley Efron, Trevor Hastie, Iain Johnstone and Robert Tibshirani. Least Angle Regression. 2004.
• Robert Tibshirani. Regression Shrinkage and Selection via the Lasso. 1996.
• Ron Kohavi , George H. John. Wrappers for Feature Subset Selection. 1996.

Case studies and applications

• Slav Petrov and Dan Klein. Sparse Multi-Scale Grammars for Discriminative Latent Variable Parsing. EMNLP 2008.
• Ben Blum, Michael Jordan, David E. Kim, Rhiju Das, Philip Bradley, David Baker. Feature Selection Methods for Improving Protein Structure Prediction with Rosetta. NIPS 2007.
• Alice X. Zheng. Statistical Software Debugging. 2005.
• NIPS Feature Selection Challenge from 2003.
• Forman. An Extensive Empirical Study of Feature Selection Metrics for Text Categorization. 2003.
• E.P. Xing, M.I. Jordan and R.M. Karp. Feature selection for high-dimensional genomic microarray data. ICML 2001.
• Y. Yang, J. Pederson. A comparative study on feature selection in text categorization. 1997. Applies filtering.

Papers not directly related to feature selection but referenced in the lecture

• Using features to do domain adaptation or multi-task learning: Hal Daume Frustratingly Easy Domain Adaptation (2006), Jenny Rose Finkel and Christopher D. Manning Hierarchical Bayesian Domain Adaptation (2009).
• Kilian Weinberger, Anirban Dasgupta. Alex Smola, Martin Zinkevich. Feature Hashing for Large Scale Multitask Learning. ICML 2009.
• Foo, Chuan-Sheng and Do, Chuong and Ng, Andrew. A majorization-minimization algorithm for (multiple) hyperparameter learning. ICML 2009.