Information Theory
Module Number: EI70350
Duration: 1 Semester
Occurence: Winter Semester
Language: English
Number of ECTS: 5
Staff
Professor in charge: Gerhard Kramer
Amount of work
Contact hours: 75
Self-study hours: 75
Total: 150
Description of achievement and assessment methods
Examination with the following elements: -Written examination
Exam type: written
Exam duration (min.): 90 Minutes
Possibility of retaking: In the next semester: Yes; at the end of the semester: No
Homework: No
Lecture: No
Conversation: No
Written paper: No
Recommended requirements
Basic principles of probability theory an statistics
Contents
Review of probability theory. Uncertainty, mutual information, informational divergence, Fano's inequality, convexity. Source coding: Discrete Memoryless Sources, PRefix-Free codes, Shannon-Fano codes. huffman codes, Tunstall codes, Discrete Stationary Sources, Elias-WIllems universal souce coding. Typical sequences and sets. Channel coding: Discrete Memoryless Channels, Capacity. Relative entropy and additive white Gaussian noise channels.
The course covers the basic concepts of information theory, including entropy and mutual information, lossless data compression for memoryless and stationary sources, and reliable communication over memoryless channels.
Study goals
After completion of the module the student is able to understand, apply analyse, and evaluate information theoretic principles for source and channel coding. He or she will be able to create source and channel codes, and formulas, for specific communication problems.
Teaching and learning methods
- Learning method: Lectures, tutorials, exercises, individual study
- Teaching method: The students are instructed in an explorative teaching style
Media formats
- Lecture notes
- Problem sheets
- Presentation slides
Literature
Further reading:
- Cover, T., Thomas, J.: Elements of Information Theory, Wiley-Interscience; 2nd edition