Solving Inverse Problems with Deep Learning

Module Number: EI71068

Duration: one Semester

Ocurrence: Summer Semester

Language: English

Number of ECTS: 6



Professor in charge: Reinhard Heckel

Amount of Work

Contact hours: 60

Self-study hours: 120

Total: 180

Description of Achievement and Assessment Methods

Students take a written exam of two hours duration. The exam consists of questions on the theory and algorithms of signal recovery with deep neural networks. The exam tests whether students can analyse, evaluate, and design solvers for inverse problems with deep networks. The exam is open-book, thus lecture notes and any other notes are permitted, but no computational devices are needed or are allowed.

Besides the written exam, 20% of the grade will be either through a presentation of a paper or through homework submission. In either case we will test whether students can analyse, evaluate, and design solvers for inverse problems with deep networks.
Specifically, if sufficiently few students are enrolled, each student has to present or defend a paper in the discussion session, and the evaluation of the paper, as well as the quality of presenting the arguments will count 20% towards the final grade.
If more students are enrolled than enabling each student to present a paper, then students will not present papers. In this case the submission of the homeworks which will contain evaluation and design of solvers for inverse problems will count 20% towards the final grade. The homework will contain analysis, evaluation, and design problems on solving inverse problems with deep networks.
The mode will be determined before the first lecture, and be communicated during the first lecture.

Prerequisites (recommended)

Analysis, an introduction to probability and statistics, and linear algebra. An introduction to machine learning is very helpful, but not necessary.

Intended Learning Outcomes

Upon successful completion of the module, students will be able to i) apply learning based methods to inverse problems such as signal and image recovery from few and noisy images, ii) evaluate their theoretical foundations and limits, and iii) critically evaluate papers and methods in that area, and iv) design variations of existing methods


There is a long history of algorithmic development for solving inverse problems arising in sensing and imaging systems and beyond. Examples include medical and computational imaging, compressive sensing, as well as community detection in networks. Until recently, most algorithms for solving inverse problems in the imaging and beyond were based on static signal models derived from physics or intuition, such as wavelets or sparse representations.

Today, the best performing approaches for the aforementioned image reconstruction and sensing problems are based on deep learning, which learn various elements of the method including i) signal representations, ii) stepsizes and parameters of iterative algorithms, iii) regularizers, and iv) entire inverse functions. Motivated by those success stories, researchers are redesigning traditional imaging and sensing systems, and deep learning based signal reconstruction methods are starting to be used in important imaging technologies, for example in GEs newest computational tomography scanners and in the newest generation of the iPhone.

This course gives a graduate/master level introduction to deep learning based imaging. The course first introduces classical approaches to solving inverse problems and then aims to explain the recent advances of deep neural network based approaches for solving inverse problems in the imaging sciences.

Topics include classical sparse models, optimization for fitting classical methods and for training deep networks, unrolled algorithms, convolutional neural networks for image recovery and generation, generative models for image recovery, un-trained neural networks for signal recovery. The course ends with a brief outlook on how to apply those methods beyond image recovery for the recovery of a variety of other signals.

Teaching and Learning Methods

The course will take place in two sessions on the same day: In the first session the foundations of a topic are explained during lectures, with a focus on methods, ideas, and the theory behind the methods.
In the second session, we will discuss the method in more detail, go through a problem, or discuss a recent paper on that topic. If we discuss a paper, all students will need to read that paper before class so that we can have a meaningful discussion.


The lecture will be given via zoom. Lecture notes and exercises will be distributed.

Reading List

Lecture notes will be distributed.