Module Number: EI70350
Duration: 1 Semester
Occurence: Winter Semester
Number of ECTS: 5
Professor in charge: Gerhard Kramer
Contact hours: 75
Self-study hours: 75
Examination with the following elements: -Written examination
Exam type: written
Exam duration (min.): 90 Minutes
Possibility of retaking: In the next semester: Yes; at the end of the semester: No
Written paper: No
Basic principles of probability theory an statistics
Review of probability theory. Uncertainty, mutual information, informational divergence, Fano's inequality, convexity. Source coding: Discrete Memoryless Sources, PRefix-Free codes, Shannon-Fano codes. huffman codes, Tunstall codes, Discrete Stationary Sources, Elias-WIllems universal souce coding. Typical sequences and sets. Channel coding: Discrete Memoryless Channels, Capacity. Relative entropy and additive white Gaussian noise channels.
The course covers the basic concepts of information theory, including entropy and mutual information, lossless data compression for memoryless and stationary sources, and reliable communication over memoryless channels.
After completion of the module the student is able to understand, apply analyse, and evaluate information theoretic principles for source and channel coding. He or she will be able to create source and channel codes, and formulas, for specific communication problems.
- Learning method: Lectures, tutorials, exercises, individual study
- Teaching method: The students are instructed in an explorative teaching style
- Lecture notes
- Problem sheets
- Presentation slides
- Cover, T., Thomas, J.: Elements of Information Theory, Wiley-Interscience; 2nd edition