CEng 783 - Deep Learning - Fall 2019
- Instructor: Asst. Prof. Dr. Emre Akbas; e-mail: emre at ceng dot metu.edu.tr; Office: B-208; Office hours by apppointment
- Teaching Assistant: Nermin Samet and Merve Tapli are kindly volunteering. Thank you, Nermin and Merve!
- Lectures: Tuesdays 13:40-16:30 at
BMB-2 BMB-5
- Online communication: (e-mail list, forum, homework submissions) https://odtuclass.metu.edu.tr/
- Syllabus: pdf
Announcements
- Oct 16, 2019 - Homework assignment #2 is announced.
- Oct 11, 2019 - Nermin Samet and Merve Tapli will be voluntary TAs of the course. Thank you, Nermin and Merve!
- Sep 24, 2019 - Students with these IDs can add the class during the add/drop period next week.
- Sep 11, 2019 - I am receiving many e-mails about taking this course as a special student, as an undergrad, from other departments/universities, etc. I am not able to respond each of them. If you want to take the course, please make sure you come to the first lecture.
Late submission policy
Any work, e.g. an assignment solution, that is submitted late past its deadline will receive -10 points per day delayed. For example, if the deadline is Oct 14, 23:55 and a student submits his/her work anytime on Oct 15, that work will be evaluated over 90 instead 100.
Detailed syllabus:
An important note: Lecture slides provided below are by no means a “complete” resource for studying. I frequently use the board in class to supplement the material in the slides.
NOTE: Below, links to slides and homework material are broken because I removed the files. Recent versions of these files can be found in the Fall 2020 version of the course.
Week 1
Week 2
Week 3
- Lecture topics: Biological neuron, artificial neuron, Perceptron, Multilayer Perceptrons, Artificial Neural Networks, Backpropagation, Activation Functions, Stochastic Gradient Descent, Momentum (slides)
- Adding regularization to the hinge loss classifier: Colab notebook
- Building, training and evaluating a basic MLP: Colab notebook
- Reading assignment for next week: “Chapter 8: Optimization for trainig deep models” from the book “Deep Learning.”
Week 4
- Lecture topics: Convolutional neural networks, convolution, connectivity types, pooling, AlexNet, data augmentation, dropout. (slides)
- Building, training and evaluating a basic CNN: Colab notebook
- Reading assignment for next week: LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.
- Assignment 2 is due Oct 29 via ODTUClass. The rest of the files can be found at ODTUClass.
Week 5
- Lecture topics: Convolutional neural networks continued, multiclass hinge loss, derivation of cross-entropy loss, notes on initializing neural networks, implementing backpropagation in a modular way, variants of stochastic gradient methods, adaptive learning rate methods. (slides)
- Reading assignment for next week:
- He, K., Zhang, X., Ren, S. and Sun, J. Deep residual learning for image recognition. In CVPR 2016.
- Lin, T.Y., Goyal, P., Girshick, R., He, K. and Dollár, P. Focal Loss for Dense Object Detection. In ICCV 2017.
Week 6
- No lecture due to national Republic Day. Lecture topics of this week will be distributed to other weeks.
Week 7
- Midterm exam held in class.
Week 8
Week 9
Week 10
- Lecture topics: Some applications of RNNs (character-level language modeling, word-embeddings, image captioning, machine translation, attention mechanism, image generation, external memory models) (slides)
- Building a basic sequence-to-sequence translation model (for simple arithmetic operations): Jupyter notebook
Week 11
- Lecture topics: Deep generative modeling (slides)
Week 12
- Project progress demos/presentations in class.
Week 13
- Lecture topics: A brief intro to deep reinforcement learning (slides)
- Lecture topics: Some notes on deep hierarchies in human/biological vision. (slides)
- Misc topics: neural architecture search, importance of sampling during training, self-supervision, transformers.
Content will be added here as we go…