Module Number: EI70350

Duration: 1 Semester

Occurence: Winter Semester

Language: English

Number of ECTS: 5

Staff

Professor in charge: Gerhard Kramer

Amount of work

Contact hours: 75

Self-study hours: 75

Total: 150

Description of achievement and assessment methods

Examination with the following elements: -Written examination

Exam type: written

Exam duration (min.): 90 Minutes

Possibility of retaking: In the next semester: Yes; at the end of the semester: No

Homework: No

Lecture: No

Conversation: No

Written paper: No

Recommended requirements

Basic principles of probability theory an statistics

Contents

Review of probability theory. Uncertainty, mutual information, informational divergence, Fano's inequality, convexity. Source coding: Discrete Memoryless Sources, PRefix-Free codes, Shannon-Fano codes. huffman codes, Tunstall codes, Discrete Stationary Sources, Elias-WIllems universal souce coding. Typical sequences and sets. Channel coding: Discrete Memoryless Channels, Capacity. Relative entropy and additive white Gaussian noise channels.

The course covers the basic concepts of information theory, including entropy and mutual information, lossless data compression for memoryless and stationary sources, and reliable communication over memoryless channels.

Study goals

After completion of the module the student is able to understand, apply analyse, and evaluate information theoretic principles for source and channel coding. He or she will be able to create source and channel codes, and formulas, for specific communication problems.

Teaching and learning methods

• Learning method: Lectures, tutorials, exercises, individual study
• Teaching method: The students are instructed in an explorative teaching style

Media formats

• Lecture notes
• Problem sheets
• Presentation slides

Literature

Further reading:

• Cover, T., Thomas, J.: Elements of Information Theory, Wiley-Interscience; 2nd edition