Deep Learning from Scratch

Monday 4 - 5:45 pm // Write your own neural network from scratch and learn how it works!


Für diesen Kurs kann man keine Teilnahmenbescheinigung erhalten.

Bei ausreichender Teilnahme und Erbringung eines der angebotenen Leistungsnachweise kannst Du ein Leistungszertifikat erwerben.



Our courses are open to all and free of charge for students, pupils, trainees, founders, scientific staff, job seekers, refugees, and pensioners. For employees and self-employed persons, a fee of 200 euros is charged when acquiring a certificate of achievement. Employees and self-employed persons can apply for a scholarship via

What you get

This course is based on the first three courses of the online Coursera Specialization "Deep Learning" provided by The instructor is Andrew Ng, co-founder of Coursera, founding lead of the Google Brain Team, and professor in computer science at Stanford University.

The course will provide you with an introduction to machine learning and deep learning, teach you the theoretical foundation behind neural network and how to implement them. We will discuss and implement step by step the basic blocks of deep learning: forward and backpropagation, neuron and layers, activation functions, optimizer, parameters and more. You will then apply the newly acquired knowledge to implement your own deep learning project in a team which you will present at the end of the course.

This course is part of the Machine Learning Degree. Participants of the program for the Machine Learning Degree get preferred access to this course. Find more information on the Machine Learning Degree page.

All needed software and online course content is free. As Germany's first Coursera for Campus partner, will in particular provide you with full access to the Coursera online learning platform.

What you should bring

You have some basic knowledge about programming and linear algebra (working with vector and matrices) and are interested in getting in-depth knowledge on how to implement machine learning algorithms.
If you are interested in machine learning more from a practical perspective, check out also our Machine Learning with Tensorflow course. Also, you should calculate sufficient time. During the week you will be expected to work through the assigned online course content and complete home assignments, for which you should calculate 4 to 8 hours each week. With the start of your project (after four weeks), you will in addition need some hours more to work on that (the assignment will decrease in the last weeks to give you more time to work on the project).

In order to receive a certificate of achievement ("Leistungszertifikat"/ ECTS) for this course, active participation is expected, no more than two classes may be missed, and you have to conduct a practice project in a team of 2 to 4 persons. At the end of the course the project has to be presented and a well documented project source code has to be submitted.

Before the first meeting there will be a Semester Kick-Off Event on October 21.

In the Kick-Off you will meet your course instructor, can ask questions about the participation in the course, and get helpful information to prepare for the start of the course. The attendance at the Kick-Off is not mandatory but recommended for all participants. During the current Coronavirus outbreak, the Kick-Off as well as the course will be streamed online via Zoom. Consultation hours will be organized to allow also personal meeting and an easier communication.


Deep Learning from Scratch
Remote via Zoom
Wintersemester 2021/2022
16:00 - 17:45 Uhr


Luca Palmieri
Start: 25.10.2021
Ende: 24.01.2022
Short introduction about the course, the teacher, how it will work and what you can expect from it. A round of introduction from all participants to get to know each others.

Luca Palmieri

First 2 weeks of the online course. Introduction about tools and support for set up. Discussion about python environment, dot product against element wise multiplication, and first exercise session training a small neural network recognizing cats!

Luca Palmieri

Backpropagation: some math to understand the basics behind the magic. Weights initialization in the training and notebook example of planar data classification changing the number of hidden unit in a shallow network - only 1 hidden layer.

Luca Palmieri

Deep neural network. First example of generalizing a neural network with L layers. Discussion and choice about the projects. Quick overview of different network architectures and when they may be used. End of the first Coursera course.

Luca Palmieri

Beginning of the secound Coursera course. Practical hints about initialization and regularization techniques to avoid overfitting and improving the training of a neural network.

Luca Palmieri

Mini-batches, Momentum, RMSProp and AdamOptimizer: an overview of optimization algorithm to train faster neural networks.

Luca Palmieri

Quick overview of the projects, discussion about batch normalization and hyperparameters search, first assignment using Tensorflow to create a small neural network. Small discussion about Tensorflow ideas and modalities and difference between 1.0 and 2.0 versions.

Luca Palmieri

Overview of the projects, discussion about how to structure the training, test and validation set, and more general on how to structure the whole project. Guidelines for the projects presentations, suggestions on what to put on them, baseline and human performances. What about using transfer learning, end-to-end approaches, divide the problem into smaller subproblems, or using multi-tasking? Sometimes the problem can be seen from another perspective. Checking out a LSTM tutorial on time series.

Luca Palmieri

Additional material about different neural network architectures, project checkpoint with peer review process. The homework consists in preparing a small presentation for the other groups.

Luca Palmieri

We discuss at the beginning of the course and agree during the course on a topic we want to explore further, for example: * data preparation * deployment * convolutional neural network (CNN) * long short term memory (LSTM) * transformers And we have a discussion with a practical example about that.

Luca Palmieri

Each Project will have time to make a presentation. After each presentation a round of questions will follow. The exact schedule and time will be decided based on the number of projects.

Luca Palmieri

If many groups are presenting the projects, the final event may be distributed in two sessions.

Luca Palmieri