Deep Learning (CAS machine intelligence, 2019)

This course in deep learning focuses on practical aspects of deep learning. We therefore provide jupyter notebooks (complete list of notebooks used in the course).

For doing the hands-on part on your own computer you can either install anaconda (details and installation instruction) or use the provided a docker container (details and installation instruction).

You can also execute the notebooks for the hands-on part on the cloud using binder Binder or open them in colab.

To easily follow the course please make sure that you are familiar with the some basic math and python skills.

Info for the projects

You can join together in small groups and choose a topic for your DL project. You should prepare a poster and a spotlight talk (5 minutes) which you will present on the last course day. To get some hints how to create a good poster you can check out the links that are provided in poster_guidelines.pdf

If you need free GPU resources, we might want to follow the instructions how to use google colab. Help for preparing a hdf5 file from your images you can be found in the example Notebook.

Examples for projects from the DL course 2018 and 2019 can be found here.

Other resources

We took inspiration (and sometimes slides / figures) from the following resources.


The course is split in 8 sessions, each 4 lectures long.

Day Date Time
1 Tue Feb 19 13:30-17:00
2 Tue Feb 26 13:30-17:00
3 Tue March 05 13:30-17:00
4 Tue March 12 09:00-12:30
5 Tue March 19 09:00-12:30
6 Tue March 26 09:00-12:30
7 Tue April 02 09:00-12:30
8 Tue April 09 09:00-12:30


Syllabus (might change during course)

Day Topic and slides Additional Material Exercises and homework
1 Deep learning basics slides_day1
  • Overview of deep learning
  • Computational graphs, feeding and fetching
  • Loss function (RMS)
  • Gradient descent and generalizations
  • Example: linear regression
2 Gradient Descent and loss functions for classification slides_day2
  • Gradient Descent
  • Logistic regression
  • Multinomial Logistic Regression
3 Going Deeper / Tricks of the trade slides_day2 slides_day3</a>
  • Fully connected network
  • Backpropagation and Gradient Flow
  • ReLU
  • Regularization:
    • Early stopping
    • Dropout
4 Convolutional Neural Networks I slides_day4(pdf) slides_day4(ppt)
  • Why going beyond fully connected NN?
  • What is convolution?
  • Building a CNN
5 Convolutional Neural Networks II slides_day5
  • What to do in case of limited data?
  • What is a CNN looking at?
  • Checking out challange-winning CNN architectures
  • 1D CNNs for time-ordered data
6 Modern CNN Architectures and Recurent Neural Networks slides_day6
  • Working with text: bag of words, embeddingss
  • Sentiment analysis with conv1D
  • Recurrent Neural Networks
  • Topic I: RNN continued, GRU, LSTM and Quantifying prediction uncertainties slides_day7
    • Using GRU
    • Using LSTMs
    • Colah (August 2015) Understanding LSTM Networks (blog post)
    8 Uncertainties in DL slides_day8
    • Recap: Modeling uncertainties in simple linear regression
    • MC Dropout to quantify uncertainties in DL models

    Tensorchiefs are Oliver Dürr, Beate Sick and Elvis Murina.