CS 294-5: Statistical Natural Language Processing, Fall 2005
|Instructor: Dan Klein|
|Lecture: Mondays and Wednesdays, 1:00-2:30pm, 310 Soda Hall|
|Office Hours: Mondays and Wednesdays 2:30-3:30pm in 775 Soda Hall, or by appointment|
11/4/05: Homework 5
10/18/05: Homework 4
10/18/05: Section on 10/21 in Soda, 1-2pm, on word alignment
10/15/05: Extension: Homework 3 due on Wednesday 10/19
10/4/05: Homework 3
10/1/05: Update: No class on 10/3 or 10/5 (HW2 still late if timestamped after 10/3)
9/26/05: No class on 10/5
9/26/05: Invite: StatNLP lunch, Tuesdays 12:30 in Soda 373 [topic]
9/26/05: Final project guidelines
9/19/05: Homework 2
9/19/05: Reminder: my office hours are cancelled on Tuesday, but I'll be back on Wednesday.
9/14/05: Update: Aria's office hours will be extended to F 12-3 in Soda 493, at least this week.
9/12/05: Aria's office hours will be F 12-1 in Soda 493
9/11/05: My office hours have moved, by popular demand, to T 11-12, W 2:30-3:30
9/02/05: Want a Millennium account? Fill out the form by Tuesday morning if you want it soon.
8/31/05: Problems with the newsgroup? Check here.
8/31/05: Homework 1
8/31/05: Accounts and access
8/29/05: Class policies
8/29/05: Class questionnaire
8/16/05: The course newsgroup is ucb.class.cs294-5. If you use it, I'll use it!
8/16/05: The previous website has been archived.
This course will explore current statistical techniques for the automatic
analysis of natural (human) language data. The dominant modeling paradigm is
corpus-driven supervised learning, but unsupervised methods and even hand-coded
rule-based systems will be mentioned when appropriate.
In the first part of the course, we will examine the core tasks in natural language processing, including language modeling, word-sense disambiguation, morphological analysis, part-of-speech tagging, syntactic parsing, semantic interpretation, coreference resolution, and discourse analysis. In each case, we will discuss which linguistic features are relevant to the task, how to design efficient models which can accommodate those features, and how to estimate parameters for such models in data-sparse contexts. In the second part of the course, we will explore how these core techniques can be applied to user applications such as information extraction, question answering, speech recognition, machine translation, and interactive dialog systems.
Course assignments will highlight several core NLP tasks. For each task, we will construct a basic system, then improve it through a cycle of linguistic error analysis and model redesign. There will also be a final project, which will investigate a single topic or application in greater depth. This course assumes a familiarity with basic probability and the ability to program in Java. Prior experience with linguistics or natural languages is helpful, but not required.
The texts for this course are:
The former is loosely required (i.e. you'll want access to a copy) while the latter is recommended as supplementary reading. Both are on reserve in the Engineering library.
|Week||Date||Topics||Techniques||Readings||Assignments (Out)||Assignments (Due)|
|1||Aug 29||Course Introduction||M+S 1-3|
|Aug 31||Language Models||Multinomial Smoothing||M+S 6, J+M 6,
Chen & Goodman
|HW1: Language Models|
|2||Sep 5||NO CLASS|
|Sep 7||Language Models||EM, More Smoothing|
|3||Sep 12||Text Categorization||Naive-Bayes||M+S 7, Event Models|
|Sep 14||Word-Sense Disambiguation||Maximum-Entropy||Berger's tutorial|
|4||Sep 19||Text Clustering||EM||HW2: PNP Classification||HW1|
|Sep 21||Part-of-Speech Tagging||HMMs||M+S 9-10, J+M 7.1-7.4|
|5||Sep 26||Part-of-Speech Tagging||MEMMs / CRFs||
|Sep 28||Word Class Induction||Distributional Models||HMM Learning, Distributional Clustering|
|HW3: POS Tagging||HW2|
|7||Oct 10||Machine Translation (Transfer)||IBM Models||M+S 13, J+M 21, IBM Models.|
|Oct 12||Machine Translation (Transfer+Decoding)||IBM Models||HMM, Decoders|
|8||Oct 17||Machine Translation||Phrase-Based Models||Phrase-Based|
|Oct 19||Syntactic Parsing / Ambiguities||M+S 3.2, 12.1, J+M 12||HW4: Machine Translation||HW3|
|9||Oct 24||Unlexicalized PCFGs||Splitting Methods||Unlexicalized, M+S 12.1|
|Oct 26||Parsing Algorithms||CKY||M+S 11|
|10||Oct 31||Lexicalized Parsing||M+S 12.2, J+M 12.3-4, Best-First, A*, Collins, Charniak and Johnson|
|Nov 2||Semantic Interpretation||Compositional semantics, J+M 14,15||HW5: Parsing / Grammars||HW4|
|11||Nov 7||Semantic Roles, Coreference||Semantic Role Labeling, Empty Reconstruction, Coreference|
|Nov 9||Grammar Induction||Model Merging, Distributional, Constituency/Dependency, Translingual Constraint|
|12||Nov 14||Question Answering|
|Nov 16||Information Extraction||HW5|
|13||Nov 21||Syntactic Translation|
|Nov 23||The Speech Signal|
|14||Nov 28||Speech Recognition|
|Nov 30||Final Projects (NIPS groups)|
|15||Dec 5||Speech Synthesis|
|Dec 7||Final Project Presentations||FP|