Foundations of NLP
Fall 2024 · Mahindra University
Fall 2024Course CS3126ELT 1
| Lecture | Topic | Resources |
|---|---|---|
| Lecture 0 | Introduction to NLP and its applications | |
| Lecture 1 | Regular Expressions |
|
| Lecture 2 | Text Normalization and Tokenization |
|
| Lecture 3 | Stop Words, Bag-of-Words, TF-IDF, POS Tagging, NER | Lecture SlidesTF-IDF (Python documentation)Textual Data(Implementation/Code)Stemming/Lemmatization (Textbook Chapter)Words and Vectors (Textbook Chapter)Text Preprocessing (Implementation)NLP DatasetsSpacy Library for NLP
|
| Lecture 4 | Semantic, Distributed representations, Vector Embeddings and Word2vec | Lecture SlidesVector Semantics (Textbook Chapter)Logistic regression (Optional)GloVe: Global Vectors for Word Representation (Implementation/Code)fast text: Library for efficient text classification and representation learning (Implementation/Code)
|
| Lecture 5 | Language Models-Probabilistic Language Modeling |
|
| Lecture 6 | Recurrent Neural Networks, LSTM | Lecture SlidesRNNs and LSTMs (Textbook Chapter)Language Models, RNN and LSTM [CS224n Lecture Notes]RNN PyTorch Slides & ImplementationOn the difficulty of training Recurrent Neural Networks [Research Paper][Additional]Vanishing and Exploding gradients (Colab Notebook)Recurrent Neural Network [Implementation/Code]Recurrent Neural Network [Blog]
|
| Lecture 7 | Attention | Lecture SlidesTransformers (Textbook Chapter)Attention Visualization [Implementation]Attention is all you need [Research Paper]Visual Attention paper [Research Paper]WMT: The Conference on Machine Translation [Conference]
|
| Lecture 8 | Transformers |
|
| Lecture 9, Lecture 11 | BleU Score, Contextual Embeddings, BERT, MLM | Lecture SlidesLecture SlidesMasked Language Models (Textbook Chapter)BERT [Implementation]Fine-tuned BERT with NER [Implementation]
|
| Lecture 12 | Overview of Large Language Models | Lecture SlidesLarge Language Models (Textbook Chapter)Unsloth - Finetune for Free (Additional Link)A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT [Research Paper]A Survey of Large Language Models [Research Paper]
|
| Lecture 13 | Generative Modeling, Naive Bayes |
|
| Lecture 14 | POS, Markov Models, HMM, Viterbi |
|