Skip to content
csd81Public

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Applied Information Theory — Course Notes

Course notes for a single-semester university course on applied information theory, split into six chapter PDFs.

Authors: István Vassányi, Benedek Szakonyi Institution: University of Pannonia (Pannon Egyetem), Veszprém, Hungary Year: 2019

This text attempts to present an application-oriented introduction to information theory, with fewer theorems and proofs than usual, and more engineering considerations. The approach and notation are very similar to those of the excellent textbook [1], with some simplifications and extensions that seemed necessary for a single-semester university course.

Chapters

Part I — Source Coding

# Chapter Pages PDF
1 Entropy and Mutual Information 8 01_entropy_and_mutual_information.pdf
2 Source Coding 10 02_source_coding.pdf

Part II — Channel Coding

# Chapter Pages PDF
3 Channel Coding 12 03_channel_coding.pdf
4 Binary Linear Block Codes 7 04_binary_linear_block_codes.pdf
5 Cyclic Codes 9 05_cyclic_codes.pdf
6 Convolutional Codes 9 06_convolutional_codes.pdf

Detailed contents

1. Entropy and Mutual Information 1.1. The Concept of Information (Everyday, Philosophical, Engineering) 1.2. Characterising Discrete Sources, Entropy 1.3. Conditional Entropy and Mutual Information

2. Source Coding 2.1. The Basics 2.2. Huffman Coding 2.3. Source Coding Theorems 2.4. Lempel-Ziv Coding 2.5. Arithmetic Coding 2.6. Evaluating Source Coding Methods

3. Channel Coding 3.1. The Basics 3.2. Characterizing Channels 3.3. The Binary Communication Problem 3.4. The Channel-Coding Theorem 3.5. Error Detection and Correction 3.6. Filling the Code Space

4. Binary Linear Block Codes 4.1. Code Words as Vectors 4.2. Properties of Linear Block Codes 4.3. The Parity-Check Theorem 4.4. The Hamming Code

5. Cyclic Codes 5.1. Construction of Cyclic Codes 5.2. Reed-Solomon Codes 5.3. Interleaving 5.4. Error Rates of Block Codes

6. Convolutional Codes 6.1. Generating Convolutional Codes 6.2. The State Machine Model 6.3. Decoding and Correcting in One Step: the Viterbi Algorithm 6.4. Improvements on Convolutional Codes

Further reading

  1. Richard B. Wells: Applied Coding and Information Theory for Engineers, Prentice Hall, 1999.
  2. Steven Roman: Introduction to Coding and Information Theory, Springer, 1997.
  3. R. B. Ash: Information Theory, Dover Publications, 1990.

Provenance

Originally published as a single volume by the University of Pannonia within the project EFOP-3.4.3-16-2016-00009 — "A felsőfokú oktatás minőségének és hozzáférhetőségének együttes javítása a Pannon Egyetemen". The six chapter PDFs here are splits of that volume, one per chapter.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors