Deep Learning (CAS machine intelligence, 2023)

This course in deep learning focuses on practical aspects of deep learning. We therefore provide jupyter notebooks (complete overview of all notebooks used in the course).

For doing the hands-on part we recommend to use colab (you might need a google account) an internet connections is also needed. If you want to do it without internet connection on your own computer you can install anaconda (details and installation instruction). Please note that we are not experts in anaconda and thus can only give limited support.

To easily follow the course please make sure that you are familiar with the some basic math and python skills.

Info for the projects

You can join together in small groups and choose a topic for your DL project. You should prepare a poster and a spotlight talk (5 minutes) which you will present on the last course day. To get some hints how to create a good poster you can check out the links that are provided in poster_guidelines.pdf

If you need free GPU resources, we might want to follow the instructions how to use google colab.

Examples for projects from previous versions the DL course: 2018, 2019 2020 2021 2023

Fill in the Title and the Topic of your Projects until End of Week 5 here

Other resources

We took inspiration (and sometimes slides / figures) from the following resources.

Probabilistic Deep Learning (DL-Book) Probabilistic Deep Learning. This book is written by us the tensorchiefs and covers the increasingly popular probabilistic approach to deep learning.
Deep Learning Book (DL-Book) http://www.deeplearningbook.org/. This is a quite comprehensive book which goes far beyond the scope of this course.
Convolutional Neural Networks for Visual Recognition http://cs231n.stanford.edu/, has additional material and youtube videos of the lectures. While the focus is on computer vision, it also treats other topics such as optimization, backpropagation and RNNs. Lecture notes can be found at http://cs231n.github.io/.
More TensorFlow examples can be found at dl_tutorial
Another applied course in DL: TensorFlow and Deep Learning without a PhD

Dates

The course is split in 8 sessions, each 4 lectures long.

Day	Date	Time
1	21.02.2023	13:30-17:00
2	28.02.2023	13:30-17:00
3	07.03.2023	13:30-17:00
4	14.03.2023	13:30-17:00
5	21.03.2023	13:30-17:00
6	28.03.2023	13:30-17:00
7	04.04.2023	13:30-17:00
8	11.04.2023	13:30-17:00

Syllabus (might change during course)

Day 1
- Topics: Introduction, Fully Connected Networks (fcNN)
- Slides: 01_Introduction
- Additional Material:
  - Network Playground
- Exercises and Homework:
  - 01_simple_forward_pass, colab
- Solutions to Exercises:
  - 01_sol, 01_sol_colab
Day 2
- Topics: Looking back at fcNN, DL framework Keras, convolutional neural networks (CNN)
- Slides: 02_fcNN_CNN
- Additional Material:
  - Understanding convolution
- Exercises and Homework:
- Solutions to Exercises:
  - 02_sol, 02_sol_colab
  - 03_sol, 03_sol_colab
  - 04_sol, 04_sol_colab
  - 05_sol, 05_sol_colab
Day 3
- Topics: Convolutional neural networks (CNN)
- Slides: 03_CNN
- Exercises and Homework:
- Solutions to Exercises:
  - 06_sol colab
  - 08_sol colab
Day 4
- Topics: Details Backpropagation in DL, MaxLike-Principle
- Slides: 04_Training_Details
- Exercises and Homework:
  - Backprop. linear regression eager 10_linreg_tensorflow colab
  - Optional Backprop static graph11_backpropagation colab
  - Simple Max Like 12_maxlike colab
  - Max Like MNIST 12b_mnist_loglike colab
Day 5
- Topics: Probabilistic Prediction Models
- Slides: 05_Probabilistic_Models
- Exercises and Homework:
  - Linear Regression with Tensorflow Probability 13_linreg_with_tfp colab
  - Poisson Regression with Tensorflow Probability 14_poisreg_with_tfp colab
- Solutions to Exercises:
  - 13_Sol colab
  - 14_Sol colab
Day 6
- Topics: Flexible CPDs
- Slides: 06_flexible_CPDs
- Exercises and Homework:
  - Regression with Tensorflow Probability on Images 15_faces_regression colab
  - Visualization of Network Decisions with GradCam 16_elephant_in_the_room colab
- Solutions to Exercises:
  - 15_Sol colab
  - 16_Sol colab
Day 7
- Topics: Ensembling and Bayes
- Slides: 07_ensembling_bayes
- Exercises and Homework:
  - Classification with Ensembles and Bayes 17_cifar10_ensemble_bayes colab
  - Bayesian Model for Coin Toss 18_bayesian_coin_toss colab
Day 8
- Topics: Bayes (continued), Projects
- Slides: 08_bayes_2023