VLaNC Lab

Natural Language Processing

Spring 2024 (Jan–Apr) · Mahindra University

Spring 2024Course CS3126ELT 1
WeekTopicResources
Week 1Introduction to NLP and its applications
Lecture Slides
Week 2Regular Expressions, Text Normalization
Lecture Slides
  • In Class given Assignment 1
Week 3Text pre-processing (Stop Words, Bag-of-Words, TF-IDF, POS Tagging, NER)
Lecture SlidesN-gram Language Models (Textbook Chapter)
  • Suggested Readings and Hands-on:
  • N-gram Language Models (Textbook Chapter)
Week 4Distributional semantics, Vector Embeddings
Lecture SlidesVector Semantics and Embeddings (Textbook Chapter)
  • Assignment 2 Released!
  • Suggested Readings and Hands-on:
  • Vector Semantics and Embeddings (Textbook Chapter)
Week 5Language Models-Probabilistic Language Modeling
Lecture SlidesN-gram Language models (Textbook Chapter)Pytorch Basics
  • Suggested Readings and Hands-on:
  • N-gram Language models (Textbook Chapter)
  • Pytorch Basics
Week 6Neural Networks and Neural Language Modeling
Lecture SlidesNeural Networks and Neural Language Models (Textbook Chapter)PyTorch Hands-on TutorialsNeural Networks, Backpropagation [Notes-CS224n]
  • Suggested Readings and Hands-on:
  • Neural Networks and Neural Language Models (Textbook Chapter)
  • PyTorch Hands-on Tutorials
  • Neural Networks, Backpropagation [Notes-CS224n]
Week 7Recurrent Neural Networks, LSTM
Lecture SlidesRNN SlidesRNNs and LSTMs (Textbook Chapter)Language Models, RNN and LSTM [CS224n Lecture Notes]RNN PyTorch Slides & ImplementationOn the difficulty of training Recurrent Neural Networks [Research Paper][Additional]Vanishing and Exploding gradients (Colab Notebook)
  • Suggested Readings and Hands-on:
  • RNNs and LSTMs (Textbook Chapter)
  • Language Models, RNN and LSTM [CS224n Lecture Notes]
  • RNN PyTorch Slides & Implementation
  • On the difficulty of training Recurrent Neural Networks [Research Paper][Additional]
  • Vanishing and Exploding gradients (Colab Notebook)
Week 8Advanced topics - Encoder Decoder Models, Attention
Lecture SlidesTransformers (Textbook Chapter)
  • Suggested Readings and Hands-on:
  • Transformers (Textbook Chapter)
Week 9Advanced topics - Multi-head attention, Transformers
Lecture SlidesBERT Visualization
  • Suggested Readings and Hands-on:
  • BERT Visualization
Week 10Advanced topics - BLEU Score, Introduction to LLM's Pretraining, Fine-tuning
Lecture SlidesThe Illustrated Transformer by Jay AlammarAttention is all you Need [Research Paper]Attention Visualization
  • Advanced topics - Suggested Readings and Hands-on:
  • The Illustrated Transformer by Jay Alammar
  • Attention is all you Need [Research Paper]
  • Attention Visualization
Week 11Advanced topics - MLM, BERT and NLP Applications
Lecture SlidesThe Illustrated BERT, ELMo, and co. by Jay AlammarBERT [Research Paper]Hugging Face - BERT
  • Suggested Readings and Hands-on:
  • The Illustrated BERT, ELMo, and co. by Jay Alammar
  • BERT [Research Paper]
  • Hugging Face - BERT