CS 2731 / ISSP 2230 / INFSCI 2420: Introduction to Natural Language Processing (Fall 2019)
Dr. Diane Litman (5105 Sensq)
Ravneet Singh (5501 Sensq)
When & Where Tuesday and Thursdays 1:00-2:15, SENSQ 5313
Office Hours Litman: Tu 4:30-5:30, Th 2:15-3:30, by appointment; Singh: M-Th 9-10:30
Description This course provides an introduction to the field of natural language processing - the creation of programs that can understand, generate, and learn languages used by humans. It will expose students to applications such as chatbots by means of computational techniques including search algorithms, dynamic programming, hidden markov models, probalistic context free grammars, and machine learning algorithms.
Prerequisites: CS 1501 (algorithms) OR consent of the instructor
Textbook: Speech and Language Processing (3rd edition online draft - free!)
Required Work (tentative) Homeworks (35%): written and programming
Exams (35%): midterm and final
Group Course Project (25%): presentation and written report
Supplemental Research Papers (5%): leading discussion and class participation
Late Penalty: For assignments that may be accepted late, the penalty is 10% per day up to 5 days including Saturday, Sunday, and holidays. Assignments are due by 11:59pm.
Textbook Readings

Assignments and Other Readings

August 27
Introduction (pdf)
Ch 1 xkcd
Humor (credits to E. Riloff for this and links below)
I. Words
August 29
Text Normalization (pdf)
Ch 2 (2.1-2.4) Unix for Poets, pages 1-19
Humor: plurals, sentence tokenization
September 3, 5, 10
Language Modeling with N-Grams (pdf)
Ch 3 (3.1-3.4) Humor
Do at home: come prepared to discuss 9/5
HW1: assigned 9/5, due 9/24
9/4 NY Times: A breakthrough for AI (really NLP) technology
September 10, 12, 17
Part-of-Speech Tagging (pdf)
Ch 8 (8.1-8.4.6) Humor
Schoolhouse Rock for Conjunctions
Do at home: come prepared to decode "Fish sleep" using Viterbi on 9/17
II. Syntax (NOTE: The syntax chapter numbers are all off by one as they changed in the Octobe 2 textbook update)
September 17, 19
Constituency Grammars (pdf)
Ch 11 (through 11.5) Humor
September 19, 24
Constituency Parsing (pdf)
Ch 12 Humor
Do at home: come prepared to show the result of parsing "The old dog the footsteps of the young" using the provided grammar on 9/24
September 24, 26
Statistical Constituency Parsing (pdf)
Ch 13 (13.1-13.5, 13.8) HW2: assigned 9/26, due 10/10
Do at home: come prepared to derive parse trees for the provided sentence and grammar using CKY
III. Machine Learning
October 1, 3
Naive Bayes Classification (and Sentiment) (pdf)
Ch 4 (through 4.8) Humor
Do at home: come prepared to answer
October 3, 8
Logistic Regression (pdf)
Ch 5 (5.1-5.2; concepts in 5.3-5.7) 9/26 NY Times: At Tech's Leading Edge, Worry about a Concentration of Power (evaluation of ML)
October 8, 10, 15
Representation Learning (and Vector Semantics) (sparse vectors) (pdf)
Ch 6 10/14 New Yorker: The Next Word: Where will predictive text take us?
Midterm Review (pdf)
October 17 Midterm Exam (closed book) Through 9/26 class (statistical parsing). No makeups.
October 22
Representation Learning (and Vector Semantics) (dense vectors) (pdf)
Ch 6 HW3: assigned 10/22, due 11/5
Word2Vec Tutorial
October 24
Neural Nets (and Language Models)
Guest Lecturer: Ravneet Singh
Ch 7 Introduction to Story Cloze Project (Corpus Paper, Shared Task Paper)
IV. Semantics
October 29, 31
Information Extraction (pdf)
Ch 18 (18.1-2) From word2vec to BERT (Bidirectional Encoder Representations from Transformers)
DRAFT project description
October 31, November 5
Word Senses and WordNet (pdf)
Ch 19 humor1, humor2
November 7, 12
Word Sense Disambiguation (pdf)
Ch 19 Midterm discussion
Contextual Word Representations: A Contextual Introduction
November 14, 19
Semantic Role Labeling (pdf)
Ch 20 11/11 NY Times: A.I. Systems Echo Biases They're Fed, Putting Scientists on Guard
V. Discourse and Applications
November 19, 21
Coreference Resolution (pdf)
Ch 22
December 3
Dialog Systems and Chatbots (pdf)
Ch 26 Final Exam Review (pdf)
December 5 Final Exam (closed book) Everything since midterm (i.e., not cumulative). No makeups.
December 10
cancelled due to double class on December 12
December 12 (from 1-3:50!!) Project Presentations Attendance required.