Course notes for a single-semester university course on applied information theory, split into six chapter PDFs.
Authors: István Vassányi, Benedek Szakonyi Institution: University of Pannonia (Pannon Egyetem), Veszprém, Hungary Year: 2019
This text attempts to present an application-oriented introduction to information theory, with fewer theorems and proofs than usual, and more engineering considerations. The approach and notation are very similar to those of the excellent textbook [1], with some simplifications and extensions that seemed necessary for a single-semester university course.
| # | Chapter | Pages | |
|---|---|---|---|
| 1 | Entropy and Mutual Information | 8 | 01_entropy_and_mutual_information.pdf |
| 2 | Source Coding | 10 | 02_source_coding.pdf |
| # | Chapter | Pages | |
|---|---|---|---|
| 3 | Channel Coding | 12 | 03_channel_coding.pdf |
| 4 | Binary Linear Block Codes | 7 | 04_binary_linear_block_codes.pdf |
| 5 | Cyclic Codes | 9 | 05_cyclic_codes.pdf |
| 6 | Convolutional Codes | 9 | 06_convolutional_codes.pdf |
1. Entropy and Mutual Information 1.1. The Concept of Information (Everyday, Philosophical, Engineering) 1.2. Characterising Discrete Sources, Entropy 1.3. Conditional Entropy and Mutual Information
2. Source Coding 2.1. The Basics 2.2. Huffman Coding 2.3. Source Coding Theorems 2.4. Lempel-Ziv Coding 2.5. Arithmetic Coding 2.6. Evaluating Source Coding Methods
3. Channel Coding 3.1. The Basics 3.2. Characterizing Channels 3.3. The Binary Communication Problem 3.4. The Channel-Coding Theorem 3.5. Error Detection and Correction 3.6. Filling the Code Space
4. Binary Linear Block Codes 4.1. Code Words as Vectors 4.2. Properties of Linear Block Codes 4.3. The Parity-Check Theorem 4.4. The Hamming Code
5. Cyclic Codes 5.1. Construction of Cyclic Codes 5.2. Reed-Solomon Codes 5.3. Interleaving 5.4. Error Rates of Block Codes
6. Convolutional Codes 6.1. Generating Convolutional Codes 6.2. The State Machine Model 6.3. Decoding and Correcting in One Step: the Viterbi Algorithm 6.4. Improvements on Convolutional Codes
- Richard B. Wells: Applied Coding and Information Theory for Engineers, Prentice Hall, 1999.
- Steven Roman: Introduction to Coding and Information Theory, Springer, 1997.
- R. B. Ash: Information Theory, Dover Publications, 1990.
Originally published as a single volume by the University of Pannonia within the project EFOP-3.4.3-16-2016-00009 — "A felsőfokú oktatás minőségének és hozzáférhetőségének együttes javítása a Pannon Egyetemen". The six chapter PDFs here are splits of that volume, one per chapter.