STAT3007/STAT7007 Deep Learning

Semester 1 2022

Lecturer: Nan Ye

Course Description: Deep learning has become a much sought-after game-changing technology that has enabled breakthroughs in applications such as intelligent virtual assistants, medical diagnosis, recommender systems, and autonomous driving. This course provides a comprehensive and rigorous coverage of deep learning from both applied and theoretical perspectives. Students taking this course will understand how, why and when the algorithms work, and be able to effectively apply deep learning methods to practical problems. This course begins with the basics of machine learning, followed by a broad coverage of deep neural networks, including some major deep neural network architectures, optimization of network parameters, and applications in classification, regression and reinforcement learning. This course is suitable for both students who want to build data-driven enabling applications with deep learning, and students who want to develop a solid foundation for doing research in deep learning in particular, and machine learning or artificial intelligence more broadly. To maximise the learning outcomes, students are expected to have a solid foundation in statistics, calculus, linear algebra, and programming. Python will be used for this course.

This course is for advanced undergraduate and graduate students.


Schedule

Event Date Description Materials
Week 1
Lecture 1 22 Jan 2022 Introduction slides
Lecture 2 24 Jan 2022 Regression slides
Lecture 3 25 Jan 2022 Classification slides
Tutorial 1 questions
Prac 1 questions
Week 2
Lecture 4 08 Mar 2022 Principal Component Analysis slides
Lecture 5 10 Mar 2022 Statistical Learning Theory slides
Lecture 6 11 Mar 2022 Model Selection slides
Tutorial 2 questions
Prac 2 questions
Week 3
Lecture 7 15 Mar 2022 Perceptrons slides
Lecture 8 17 Mar 2022 Adaline slides
Lecture 9 18 Mar 2022 Hopfield Networks slides
Tutorial 3 questions
Prac 3 questions
Week 4
Lecture 10 22 Mar 2022 Gradient-based Learning slides
Lecture 11 24 Mar 2022 Multilayer Perceptron slides
Lecture 12 25 Mar 2022 Deep Learning Software slides
Tutorial 4 questions
Prac 4 questions
Week 5
Lecture 13 29 Mar 2022 Convolutional Neural Nets slides
Lecture 14 31 Mar 2022 Convolutional Neural Nets (cont.) slides
Lecture 15 1 Apr 2022 Convolutional Neural Nets (cont.) slides
Tutorial 5 questions
Prac 5 questions
Week 6
Lecture 16 5 Apr 2022 Recurrent Neural Nets slides
Lecture 17 7 Apr 2022 Recurrent Neural Nets (cont.) slides
Lecture 18 8 Apr 2022 Recurrent Neural Nets (cont.) slides
Tutorial 6 questions
Prac 6 questions
Week 7
Lecture 19 12 Apr 2022 Numerical Optimization slides
Lecture 20 14 Apr 2022 Initialization and Input Transformation slides
Tutorial 7 questions
Prac 7 questions
Week 8
Lecture 21 26 Apr 2022 Normalization slides
Lecture 22 28 Apr 2022 Adaptive Learning Rates slides
Lecture 23 29 Apr 2022 Improving Generalization slides
Tutorial 8 questions
Prac 8 questions
Week 9
Lecture 24 5 May 2022 Adversarial Learning slides
Lecture 25 6 May 2022 Activation functions slides
Tutorial 9 questions
Prac 9 questions
Week 10
Lecture 26 10 May 2022 Residual Networks slides
Lecture 27 12 May 2022 Attention slides
Lecture 28 13 May 2022 Autoencoders slides
Tutorial 10 questions
Prac 10 questions
Week 11
Lecture 29 17 May 2022 Variational Autoencoders slides
Lecture 30 19 May 2022 Generative Adversarial Networks slides
Lecture 31 20 May 2022 Reinforcement Learning slides
Tutorial 11 questions
Prac 11 questions
Week 12
Lecture 32 24 May 2022 Reinforcement Learning (cont.) slides
Lecture 33 26 May 2022 Reinforcement Learning (cont.) slides
Lecture 34 27 May 2022 Review slides
Tutorial 12 questions
Prac 12 questions
Week 13
Project seminars 31 May 2022
Project seminars 02 Jun 2022
Project seminars 03 Jun 2022