Module 1 Deep learning

The idea is that the material in Section 1.1 and 1.2 are covered during the first day of the module (and in the corresponding material to read up on at home). After that, we cover convolutional neural networks during the second day.

1.1 Fundamental knowledge about machine learning, Python and the components in a neural network

• A crash course in Python. The course contains videos and assignments
  that you can do at home. These assignments are not mandatory but useful
  for anyone who has limited experienced in Python programming.
• Overview of dierent types of machine learning: reinforcement learning
  (RL) vs supervised learning (SL) vs unsupervised learning (UL).
• Basic principles behind supervised learning, including empirical risk and
  cross-entropy.
• Basics of feedforward networks, common activation functions and when to
  use them.

1.2 Basics of how to train a neural network

• Computational graphs, backprop and stochastic gradient descent (SGD).
• Train/val/test sets.
• Basic recipe for machine learning.
• Regularization.

1.3 Convolutional neural networks

• An introduction to the basic components (convolutions, strides, pooling
  layers, etc).
• An overview of common architectures (AlexNet, VGG, GoogLeNet, ResNet,
  etc)
• Relation to feed-forward networks (shared weights, number of parameters)
• Transfer learning
• Adversarial examples and the idea of maximizing variables with respect
  to input data (easy to understand if we cover backprop in detail)
• CNNs for different applications such as
  • Semantic segmentation: shared calculations and transpose convolutions.
  • Object detection: single shot detectors, their loss functions and basic
    principles.

Anyone who has been admitted to WASP-AI should be able to attend this module. However, students with a strong background in Python, machine learning, neural networks and convolutional neural networks are likely to find the module very easy, whereas other students will have a harder time, but hopefully also learn a lot from the experience.

The module comprises:

• a crash course in Python,
• video lectures,
• two days of classes that contain a combination of:
  • short lectures with the objective to refresh the material taught in the
    videos,
  • practice sessions where students work on the material in groups of 4
    students.
• two quizzes,
• two home assignments using Jupyter notebooks and Google cloud (one on
  Python and one on CNNs).

We use Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, 2016, which is available online http://www.deeplearningbook.org, and supporting documents provided in the course (e.g., lecture slides and Jupyter notebooks).

This year’s course page is found here:

https://kth.instructure.com/courses/29062

We expect everyone to watch the video lectures and fill out a short survey before coming to class. The objective with the survey is to use the result to form groups that contain a mix of students with different backgrounds.