Deep Learning (CAS machine intelligence)

This course in deep learning focuses on practical aspects of deep learning.

For the hands-on part we provide a docker container (details and installation instruction).

Other resources

We took inspiration (and sometimes slides / figures) from the following resources.


The course is split in 8 sessions, each 4 hours long.

Tensorchiefs are Oliver Dürr, Beate Sick and Elvis Murina.
Day Topic and slides Additional Material Exercises and homework
1 Deep learning basics slides
  • Overview of deep learning
  • Computational graphs, feeding and fetching
  • Loss function (crossentropy)
  • Gradient descent and generalizations
  • Example: linear regression
2 Multinomial Logistic Regression slides
  • Logistic regression
  • Multinomial Logistic Regression
DL-book chapter 6
3 Going Deeper / Tricks of the trade slides
  • Fully connected network
  • Backpropagation and Gradient Flow
  • ReLU
  • Regularization:
    • Early stopping
    • L2 (Weight Decay)
    • Dropout
4 Convolutional Neural Networks I slides
  • Batch-Normalization
  • Why going beyond fully connected NN?
  • What is convolution?
  • Building a CNN
  • Simlarities between a CNN and the brain
5 Convolutional Neural Networks II slides
  • Typical CNN architectures
  • Transfer learning: Use pretrained nets for fine-tuning or feature generator
  • Feature/activation maps in detail
  • Understand CNN features
6 Recurent Neural Networks slides
  • Recurrent Neural Networks
  • Vanishing Gradient Problem
  • LSTMs
7 Un- and Semi-supervised Learning I slides
  • How to do unsupervised feature learning or representation learning?
  • Autoencoder and denoising AE
  • Use unsupervised learned features for pattern recognition
  • Use unsupervised learned features for classifcation with few labeled instances
8 Un- and Semi-supervised Learning II slides
  • Classical and denoising AE
  • Classical and denoising variational AE
  • Ladder network for un- and semi-supervised learning
  • Spotlight talks
  • Poster presentations