Pre-requisites | Evaluation | Logistics | Schedule | Quizzes/Assignments | What Next?
Note: The video lectures for this course are now available on youtube

Pre-requisites


There are no official pre-requisites for this course but it would help if you have done the following courses (preferably in the order mentioned below) : If you can solve most of this assignment then you are ready for this course!

Logistics


Name Lab Email Office hours Days
Anushka Singh AI4Bharat cs22s015@smail.iitm.ac.in 3-4 pm Tuesday, Thursday
Oikantik Nath AI4Bharat cs22s013@smail.iitm.ac.in 12-1 pm Tuesday, Thursday
Bibhuti Majhi AI4Bharat cs22m031@smail.iitm.ac.in 2-4 pm Wednesday
Sarthak Naithani AI4Bharat cs22m078@smail.iitm.ac.in 3-4 pm Tuesday, Wednesday
Putta Sai Sree Ram AI4Bharat cs22m076@smail.iitm.ac.in 3-4 pm Monday, Friday
Ravi Prakash Singh AI4Bharat cs22m069@smail.iitm.ac.in 1-2 pm Tuesday, Friday
Guddeppagari Shathish Kumar Reddy AI4Bharat cs22m080@smail.iitm.ac.in 3-4 pm Monday, Friday
Poorbi Mukesh Dalal - cs22m064@smail.iitm.ac.in 1-3 pm Wednesday
Ashok R - cs22m023@smail.iitm.ac.in 2-4 pm Wednesday
Amit Kumar - cs22m010@smail.iitm.ac.in 2-4 pm Wednesday

Evaluation


70% Assignments (equal weightage for each assignment) | 30% Endsem

Reference Textbooks


  1. Ian Goodfellow and Yoshua Bengio and Aaron Courville. Deep Learning. An MIT Press book. 2016.
  2. Charu C. Aggarwal. Neural Networks and Deep Learning: A Textbook. Springer. 2019.
  3. Dive into Deep Learning
/

Schedule


*M = Module (each lecture is broken down into smaller modules)
Lecture# Contents Lecture pdf Lecture Videos Extra Reading Material
Lecture 1 (Partial) History of Deep Learning, Deep Learning Success Stories Slides M1 | M2| M3 | M4 | M5 Juergen Schmidhuber, Deep Learning in Neural Networks: An Overview. 2014
Lecture 2 McCulloch Pitts Neuron, Thresholding Logic, Perceptrons, Perceptron Learning Algorithm and Convergence, Multilayer Perceptrons (MLPs), Representation Power of MLPs Slides M1 | M2| M3 | M4 | M5| M6 | M7 | M8 Chapters 1,2,3,4 from Neural Networks by Rojas
Lecture 3 Sigmoid Neurons, Gradient Descent, Feedforward Neural Networks, Representation Power of Feedforward Neural Networks Slides M1 | M2| M3 | M4 | M5| M6 | M7| M8 http://neuralnetworksanddeeplearning.com/chap4.html
Lecture 4 Feedforward Neural Networks, Backpropagation Slides M1 | M2 | M3 | M4 | M5 | M6 | M7 | M8 See Lecture 2 [Training Neural Networks] by Hugo Larochelle
Lecture 5 Gradient Descent (GD), Momentum Based GD, Nesterov Accelerated GD, Stochastic GD, AdaGrad, RMSProp, Adam Slides-1 | Slides-2 M1 | M2 | M3 | M4 | M5 | M6 | M7 | M8 | M9 | M10 | M11 | M12 | M13 | M14| M15
Lecture 6 Bias Variance Tradeoff, L2 regularization, Early stopping, Dataset augmentation, Parameter sharing and tying, Injecting noise at input, Ensemble methods, Dropout Slides M1| M2 | M3 | M4| M5 | M6 | M7 | M8 | M9 | M10| M11|M12
Lecture 7 Greedy Layerwise Pre-training, Better activation functions, Better weight initialization methods, Batch Normalization Slides M1 | M2| M3 | M4 | M5| M6
Lecture 8 Convolutional Neural Networks, LeNet, AlexNet, ZF-Net, VGGNet, GoogLeNet, ResNet Slides M1 | M2| M3 | M4 | M5| M6 | M7| M8
Lecture 9 Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks Slides M1 | M2| M3 | M4 | M5| M6 | M7 | M8 | M9 | M10
Lecture 10 Learning Vectorial Representations Of Words Slides M1 | M2| M3 | M4 | M5| M6 | M7 | M8 | M9 | M8 | M9 | M10| M11
Lecture 11 Recurrent Neural Networks, Backpropagation Through Time (BPTT), Vanishing and Exploding Gradients, Truncated BPTT Slides M1 | M2| M3 | M4 | M5
Lecture 12 Gated Recurrent Units (GRUs), Long Short Term Memory (LSTM) Cells, Solving the vanidhing gradient problem with LSTMs Slides M1 | M2| M3 | M4
Lecture 13 Encoder Decoder Models, Attention Mechanism, Attention over images, Hierarchical Attention Slides M1 | M2| M3
Lecture 14 Transformers: Multi-headed Self Attention, Cross Attention Slides M1 | M2| M3 | M4 | M5

Quizzes/Assigments


Topics Resources Release Date Submission Date
Assignment 1 (Graded) Feedforward Neural Networks Link 19-Feb-2024 10-Mar-2024
Assignment 2 ( Ungraded ) Convolutional Neural Networks Link 03-Mar-2024 03-Apr-2024
Assignment 3 ( Ungraded ) Recurrent Neural Networks Link 03-Mar-2024 03-Apr-2024
Assignment 4 ( Ungraded ) RBMs and GANs Link 03-Mar-2024 03-Mar-2024
Assignment 5 ( Graded ) Transformers Link 22-Mar-2024 22-Apr-2024
Endsem 11-May-2024 (Saturday) --

Tutorials


Topics Resources Release Date Submission Date
Tutorial 1 Calculus pdf
Tutorial 2 Linear Algebra pdf
Tutorial 3 MP Neurons, Perceptrons pdf
Tutorial 4 Sigmoid Neurons, Gradient Descent pdf
Tutorial 5 Feedforward Neural Networks, Backpropagation pdf

What Next?


You may find the following courses to be useful: