Deep Learning (CAS machine intelligence, 2019)

This course in deep learning focuses on practical aspects of deep learning. We therefore provide jupyter notebooks (complete list of notebooks used in the course).

For doing the hands-on part on your own computer you can either install anaconda (details and installation instruction) or use the provided a docker container (details and installation instruction).

You can also execute the notebooks for the hands-on part on the cloud using binder or open them in colab.

To easily follow the course please make sure that you are familiar with the some basic math and python skills.

Info for the projects

You can join together in small groups and choose a topic for your DL project. You should prepare a poster and a spotlight talk (5 minutes) which you will present on the last course day. To get some hints how to create a good poster you can check out the links that are provided in poster_guidelines.pdf

If you need free GPU resources, we might want to follow the instructions how to use google colab. Help for preparing a hdf5 file from your images you can be found in the example Notebook.

Examples for projects from the DL course 2018 and 2019 can be found here.

Other resources

We took inspiration (and sometimes slides / figures) from the following resources.

Deep Learning Book (DL-Book) http://www.deeplearningbook.org/. This is a quite comprehensive book which goes far beyond the scope of this course.
Convolutional Neural Networks for Visual Recognition http://cs231n.stanford.edu/, has additional material and youtube videos of the lectures. While the focus is on computer vision, it also treats other topics such as optimization, backpropagation and RNNs. Lecture notes can be found at http://cs231n.github.io/.
More TensorFlow examples can be found at dl_tutorial
Another applied course in DL: TensorFlow and Deep Learning without a PhD

Dates

The course is split in 8 sessions, each 4 lectures long.

Day	Date	Time
1	Tue Feb 19	13:30-17:00
2	Tue Feb 26	13:30-17:00
3	Tue March 05	13:30-17:00
4	Tue March 12	09:00-12:30
5	Tue March 19	09:00-12:30
6	Tue March 26	09:00-12:30
7	Tue April 02	09:00-12:30
8	Tue April 09	09:00-12:30

Syllabus (might change during course)

Day	Topic and slides	Additional Material	Exercises and homework
1	Deep learning basics slides_day1 Overview of deep learning Computational graphs, feeding and fetching Loss function (RMS) Gradient descent and generalizations Example: linear regression	Nature review article DL-book chapter 6 TensorFlow Intro Intro Lecture on TensorFlow form Stanford CS224d	01 Matrix Multiplication Exercises \| Notebook \| Solution 02 Linreg (with_slider) Exercises \| Notebook \| Solution
2	Gradient Descent and loss functions for classification slides_day2 Gradient Descent Logistic regression Multinomial Logistic Regression	DL-book chapter 6 DL-book chapter 3 Cross Entropy Loss	log. reg. in TF-playground 03 Logistic Regression (Challenger) Exercises \| Notebook \| Solution
3	Going Deeper / Tricks of the trade slides_day2 slides_day3</a> Fully connected network Backpropagation and Gradient Flow ReLU Regularization: Early stopping Dropout	Backpropagation DL-book chapter 7 (for regularization)	Multinomial Logistic Regression (forward pass, numpy, see slides) Solution 04 Multinomial Logistic Regression Exercises \| Notebook \| Solution 05 Fully Connected Network MNIST Exercises \| Notebook \| Solution 06 Fully Connected network (Tricks of the Trade) Exercises \| Notebook \| Solution Keras intro
4	Convolutional Neural Networks I slides_day4(pdf) slides_day4(ppt) Why going beyond fully connected NN? What is convolution? Building a CNN	Convolution arithmetic demo CNN and activation maps	Notebook Artlover \| Homework: 07 and 08 CNN Exercises \| CNN1 Notebook \| CNN1 Solution \| CNN2 Notebook \| CNN2 Solution
5	Convolutional Neural Networks II slides_day5 What to do in case of limited data? What is a CNN looking at? Checking out challange-winning CNN architectures 1D CNNs for time-ordered data	live CNN in browser Understanding CNNs through visualization cs231 on visualization CNNs Dilated 1D convolution	8 Faces Overview and FCN baseline Data overview\| Exercise \| FC Solution possible CNN Solution 11 transfer learning Exercises \| Notebook \| Solution
6	Modern CNN Architectures and Recurent Neural Networks slides_day6 Working with text: bag of words, embeddingss Sentiment analysis with conv1D Recurrent Neural Networks	Karpathy (May 2015) The unreasonable effectiveness of Recurrent Neural Networks (blog post) DL-book chapter 10	Sentiment analysis Exercise \| Notebook \| Solution
7	Topic I: RNN continued, GRU, LSTM and Quantifying prediction uncertainties slides_day7 Using GRU Using LSTMs	Colah (August 2015) Understanding LSTM Networks (blog post)	Prediction with time series using Conv1D, RNN and LSTM Exercise \| Notebook \| Solution
8	Uncertainties in DL slides_day8 Recap: Modeling uncertainties in simple linear regression MC Dropout to quantify uncertainties in DL models	What My Deep Model Doesn't Know	MC-Dropout on Cifar10 Notebook \| Solution Linear Regression with TF Probability Notebook

Tensorchiefs are Oliver Dürr, Beate Sick and Elvis Murina.

dl course 2019

the 2019 version of the dl course

Deep Learning (CAS machine intelligence, 2019)

Info for the projects

Other resources

Dates

Syllabus (might change during course)