CS 421: Natural Language Processing
Fall 2024
Contact Information
| Professor: | Natalie Parde (parde@uic.edu) | |
| Office Hours: | Tuesday 3:00 - 5:00 p.m. CST | |
| Teaching Assistants: | Abari Bhattacharya (Office Hours: Wednesday 1:30 - 3:30 p.m. CST in SELW 1228) | |
| Meghan Guo (Office Hours: Thursday 3:15 - 5:15 p.m. CST in SELW 1228) | ||
| Piazza: | https://piazza.com/uic/fall2024/cs421 |
What is this class about?
Natural language processing (NLP) is the subfield of artificial intelligence that focuses on automatically understanding and generating natural language (e.g., languages that we use for day-to-day communication, such as English, Arabic, or Navajo). It is pervasive in modern technology; popular examples include online search and chatbot applications. This class will provide an introduction to the foundations and most popular tasks performed using natural language processing, through a combination of readings, lectures, short assignments, and projects. Topics covered will include text preprocessing, part-of-speech tagging, language modeling, language representations, text classification, and dialogue systems, among others.Textbooks
Readings, learning content, and (some) assignments for this class will be drawn from the following source:- Daniel Jurafsky and James H Martin. Speech and Language Processing (3rd Edition). Draft, 2024.
This textbook is still being written; its current draft can be freely accessed at the link above.
Deliverables
This is a 400-level course, designed for both graduate students and advanced undergraduates. Depending on your classification, you may have enrolled in either the four-hour version (grad students) or the three-hour version (undergrad students). There are slightly different requirements for the two versions of the course, with the biggest difference being that students in the four-hour version will be required to complete a semester-long research study. Undergrads may opt to complete this component as well if they would like, in which case their final course grade will be determined according to the same breakdown as that used for graduate students; however, doing this extra work is certainly not a requirement. Some further details about the work you will be expected to complete for this course are provided below:- Python Bootcamp (Assignment 0): One introductory coding "bootcamp" assignment will be due before submitting the first standard deliverable, to ensure necessary Python proficiency.
- Assignments: Four assignments will be due over the course of the semester (due dates are indicated on the course calendar). These assignments will contain a mix of theoretical and coding questions. Code should be written in Python.
- Project: All students will complete a semester-long project, divided into three deliverables (due dates are indicated on the course calendar). Code, when applicable, should be written in Python.
- Research Study: Graduate students (and any undergraduates who choose to do so) will complete a semester-long study pertaining to research reproducibility and evaluation, due the week before finals week. For the study, students will analyze the reproducibility of an existing NLP research paper. These studies can be completed individually or in pairs; if done in pairs, the submission must be accompanied by a statement detailing which component(s) each student worked on, signed by both students.
Grading rubrics will be posted in the deliverables' descriptions. Final course grades will be determined according to the following breakdowns:
- Undergraduate Students:
- Python Bootcamp (Assignment 0): 5%
- Project: 39% (13% for each deliverable)
- Assignments: 56% (14% for each assignment)
- Graduate Students:
- Python Bootcamp (Assignment 0): 4%
- Project: 30% (10% for each deliverable)
- Assignments: 48% (12% for each assignment)
- Research Study: 18% (4% for the presentation, 10% for the report, and 4% for the source code)
Schedule
Below is a list of course topics, readings, deadlines, and slides by week. The version of the schedule here is subject to change. All deliverables are due by 12:00 p.m. (noon) CST on the specified due date.| Week | Topic | Readings | Deliverables | Slides |
|---|---|---|---|---|
| 8/26-8/30 | Introduction and Dialogue Systems and Chatbots | Chapter 15 | — | Download |
| 9/2-9/6 | Text Preprocessing and Edit Distance | Chapter 2 | — | Download |
| 9/9-9/13 | N-Gram Language Models and Hidden Markov Models | Chapter 3 and Appendix A |
Assignment 0 (9/13; Recommended much sooner!)
Assignment 1 (9/13) |
Download |
| 9/16-9/20 | Text Classification | Chapters 4 and 5 | — | Download |
| 9/23-9/27 | Vector Semantics | Chapter 6 | Assignment 2 (9/27) | — |
| 9/30-10/4 | Deep Learning for NLP | Chapters 7-9 (just skim!) | — | — |
| 10/7-10/11 | Generative AI and Practical Guidelines for Data-Driven NLP | Chapters 10-12 (just skim!) | Project Part 1 (10/11) | — |
| 10/14-10/18 | Syntactic Parsing | Chapter 18 and Appendix C | — | — |
| 10/21-10/25 | Semantic Parsing | Chapter 19 | Assignment 3 (10/25) | — |
| 10/28-11/1 | Temporality and Affect | Chapters 20 and 22 | Project Part 2 (11/1) | — |
| 11/4-11/8 | Word Sense Disambiguation and Coreference Resolution | Appendix G and Chapter 22 | — | — |
| 11/11-11/15 | Discourse Coherence and Question Answering | Chapters 24 and 14 | Assignment 4 (11/15) | — |
| 11/18-11/22 | Automated Speech Recognition and Text-to-Speech Synthesis | Chapters 16 | — | — |
| 11/25-11/27 | Co-Working Day | — | Project Part 3 (11/27) | — |
| 12/2-12/6 | Research Study Presentations | — | Videos (12/2) or Presentations (Tuesday/Thursday); Source Code and Report (12/6) | — |
| 12/9-12/13 | — | — | — | — |
Final Notes
This website is provided partially for student convenience, partially for my own record-keeping purposes, and partially for the benefit of others who are not able to enroll in the course but who may find the content interesting for one reason or another. It is not a substitute for the course pages on Blackboard and Gradescope, or the course discussion board on Piazza! Please refer to those sources for copies of the full syllabus, assignments, grading rubrics, submission links, and other useful information. If you are not enrolled in the course but would like to request access to those materials, please send me an email introducing yourself and explaining why you would like to have access to them. If you use these materials for your own work, please cite the publication here.Happy studying!