Calculus [Online course from MIT]
Linear Algebra [CS6015 or equivalent]  [Online course from MIT]
Probability Theory [CS6015 or equivalent]  [Online course from MIT]
Nonlinear Optimization [CS5020 or equivalent]  [First Course in Optimization by Prof. Soman (IITB) available on CDEEP]
Pattern Recognition and Machine Learning [CS5691 or equivalent]  [Andrew Ng's ML course]
Instructor: Mitesh M. Khapra
When: JanMay 2019
Lectures: Slot C
Where: CS24, CS Building, First Floor
Teaching Assistants
Name  Lab  Working hours  Days  

Shweta Bhardwaj  RBCDSAI  shwetabhardwaj44@gmail.com  24 pm  Wed,Fri 
Ananya Sai B  RBCDSAI  cs16m037@smail.iitm.ac.in  24 pm  Tue, Fri 
Shubham Patel  RBCDSAI  cs17m051@smail.iitm.ac.in  45 pm  Thu,Fri 
Jaya Ingle  RBCDSAI  cs17m060@smail.iitm.ac.in  23 pm  Mon, Tue 
Vamsi Dikkala  CV Lab  dikkalavamsi@gmail.com  1012 am  Tue, Wed, Thu 
Golla Satish Kumar Yadav  CSE Library  satti20417608@gmail.com  24 pm  Wed, Thu 
Harsh Kumar Rai  AIDB Lab  cs17m015@smail.iitm.ac.in  46 pm  Tue, Wed, Thu 
Lecture#  Contents  Lecture Slides  Lecture Videos  Extra Reading Material 

Lecture 0  Jargon Busting  Slides     
Lecture 1  (Partial) History of Deep Learning, Deep Learning Success Stories  T  H  M1  M2 M3  M4  M5 M6  M7  M8 M9  Juergen Schmidhuber, Deep Learning in Neural Networks: An Overview. 2014 
Lecture 2  McCulloch Pitts Neuron, Thresholding Logic, Perceptrons, Perceptron Learning Algorithm and Convergence, Multilayer Perceptrons (MLPs), Representation Power of MLPs  T  H  M1  M2 M3  M4  M5 M6  M7  M8  Chapters 1,2,3,4 from Neural Networks by Rojas 
Lecture 3  Sigmoid Neurons, Gradient Descent, Feedforward Neural Networks, Representation Power of Feedforward Neural Networks  T  H  M1  M2 M3  M4  M5  http://neuralnetworksanddeeplearning.com/chap4.html 
Lecture 4  Feedforward Neural Networks, Backpropagation  T  H  M1  M2  M3  M4  M5  M6  M7  M8  M9  M10  See Lecture 2 [Training Neural Networks] by Hugo Larochelle 
Lecture 5  Gradient Descent (GD), Momentum Based GD, Nesterov Accelerated GD, Stochastic GD, AdaGrad, RMSProp, Adam  T  H  M1 and M2  M3  M4  M5  M6  M7  M8  M9  M9 part 2  
Lecture 6  Eigenvalues and eigenvectors, Eigenvalue Decomposition, Basis, Principal Component Analysis and its interpretations, Singular Value Decomposition  T  H  M1  M2 M3  M4  M5 M6  M6 part 2  M7 M8  
Lecture 7  Autoencoders and relation to PCA, Regularization in autoencoders, Denoising autoencoders, Sparse autoencoders, Contractive autoencoders  T  H  M1  M2 M3  M4  M5 M6  
Lecture 8  Bias Variance Tradeoff, L2 regularization, Early stopping, Dataset augmentation, Parameter sharing and tying, Injecting noise at input, Ensemble methods, Dropout  T  H  M1  M2 M2 part 2  M3  M4 M5  M6  M7  M8  M9  M10 M11  
Lecture 9  Greedy Layerwise Pretraining, Better activation functions, Better weight initialization methods, Batch Normalization  T  H  M1  M2 M3  M4  M5  
Lecture 10  Learning Vectorial Representations Of Words  T  H  M1  M2 M3  M3 part 2  M4 M5  M5 part 2  M6  M7  M8  M9  M10  
Lecture 11  Convolutional Neural Networks, LeNet, AlexNet, ZFNet, VGGNet, GoogLeNet, ResNet  T  H  M1  M2 M3  M3 part 2  M4 M4 part 2  M5  
Lecture 12  Object Detection, RCNN, Fast RCNN, Faster RCNN, YOLO  T  H  Will be available soon  
Lecture 13  Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks  T  H  M1  M2 M3  M4  M5 M6  M7  M8  M9  M10 

Lecture 14  Recurrent Neural Networks, Backpropagation Through Time (BPTT), Vanishing and Exploding Gradients, Truncated BPTT  T  H  M1  M2 M3  M4  M5  
Lecture 15  Gated Recurrent Units (GRUs), Long Short Term Memory (LSTM) Cells, Solving the vanidhing gradient problem with LSTMs  T  H  M1  M2 M3  M3 part 2  
Lecture 16  Encoder Decoder Models, Attention Mechanism, Attention over images, Hierarchical Attention  T  H  M1  M2 M3  M3 part 2  M4 M5  
Lecture 17  Directed Graphical Models  T  H  Will be available soon  
Lecture 18  Markov Networks  T  H  Will be available soon  
Lecture 19  Using joint distributions for classification and sampling, Latent Variables, Restricted Boltzmann Machines, Unsupervised Learning, Motivation for Sampling  T  H  Will be available soon  
Lecture 20  Markov Chains, Gibbs Sampling for training RBMs, Contrastive Divergence for training RBMs  T  H  Will be available soon  
Lecture 21  Variational autoencoders  T  H  Will be available soon  
Lecture 22  Autoregressive Models: NADE, MADE, PixelRNN  T  H  Will be available soon  
Lecture 23  Generative Adversarial Networks (GANs)  T  H  Will be available soon 
Topics  Resources  Release Date  Submission Date  

Assignment 1  Derivatives + Probability [Theory]  PDF  Source Code  21Jan2019  27Jan2019 
Assignment 2  Linear Algebra [Theory]  PDF  Source Code  24Jan2019  31Jan2019 
Assignment 3  Backpropagation [Programming]    31Jan2019  12Feb2019 
Quiz 1  Lectures 17    20Feb2019  
Assignment 4  Convolutional Neural Networks [Programming]    28Feb2019  15Mar2019 
Quiz II  Lectures 815    27Apr2019  
Assignment 5  Recurrent Neural Networks [Programming]    15Mar2019  10Apr2019 
Assignment 6  Probability Refresher [Theory]    10Apr2019  15Apr2019 
Assignment 7  Variational Autoencoders [Programming]    10Apr2019  26Apr2019 
End Sem  Lectures 123    01May2019 
Deep Learning for Computer Vision [from Stanford]
Deep Learning for NLP [from Stanford]