Esisar rubrique Formation 2022

Information theory - 3AMMA331

  • Number of hours

    • Lectures 12.0
    • Projects -
    • Tutorials 10.5
    • Internship -
    • Laboratory works 3.0
    • Written tests -


    ECTS 3.0


At the end of the course, the student should be knowledgeable in the fundamentals of information theory, source coding (data compression) and channel coding (error detection and correction), in order to be able later to grasp the more recent techniques in the field.


Nicolas BARBOT


  • Information theory
    • Information measure
    • Sources, stateful and stateless sources, Markov sources, source extention, source rate
    • Entropy, mutual entropy, conditional entropy
    • Communication canals, transition matrices
    • Lossless canals, deterministic, noiseless, binary, symetric
    • Mutual information, canal capacity
    • White noise additive canals, capacity
    • Second Shannon theorem - canal coding
  • Introduction to source coding
    • Kraft inequality, Mac-Millan theorem
    • Mean length, efficiency and redundancy of a source code
    • First Shannon theorem
    • Shannon-Fano and Huffman codes
    • Lempel-Ziv codes
  • Linear block codes
    • Generation matrix, control matrix
    • Hamming distance and weight
    • Repetition code, parity code, Hamming code, extended Hamming code
    • Syndrom, decoding
  • Cyclical codes
    • Polynomial representation
    • Generating polynomial, control polynomial, generating matrix
    • Coder and decoder principlse, Meggit decoder
    • Code examples: BCH code, Golay code, Reed-Salomon code


Basic mathematics. Combinatorics. Probability theory. Signal processing.



The course exists in the following branches:

see the course schedule for 2022-2023

Additional Information

Course ID : 3AMMA331
Course language(s): FR

The course is attached to the following structures:

You can find this course among all other courses.


  • "Théorie de l'information : application aux techniques de communication", Battail, G., Masson, 1997
  • "Introduction aux codes correcteurs", Csillag, P., Ellipses, 1990
  • "Communications analogiques et numériques", Hwei Hsu, P., Mc Graw Hill, 2002
  • "Introduction aux communications numériques", Khireddine, A., Dunod, 2006
  • "Digital Communications", Proakis, J.G., 4ème éd., Mc Graw Hill, 2003