Note! This is the page for course in Fall 2007. The up-to-date course page is located in Noppa.

In 1948 Claude Shannon published his paper "A mathematical theory of communication" -- and information theory was born. The results took the scientific community by surprise. It was generally believed that increasing the transmission rate of information over a communication channel would always increase the probability of error. But Shannon proved that this probability can always be kept arbitrarily small as long as the rate is below channel capacity.

This course gives an introduction to information theory and its most important applications in communications. The topic is mathematically oriented. The basic concepts of entropy, relative entropy, and mutual information are defined, and their connections to channel capacity, coding, and data compression are presented. In addition to limits for error-free communication, information theory also presents limits for data compression. Whereas coding methods for error control are discussed in the spring course S-72.3410 Coding Methods, data compression methods are here discussed in more detail, including Huffman, Lempel-Ziv, and Shannon coding. The course book provides a variety of interesting areas of application outside communications, including gambling and investment (stock market).

The teaching language is English. (P = may be included in postgraduate studies.)

Learning objectives:

• Upon completion of the course, the students will be able to define and apply the basic concepts of information theory (entropy, etc.);

• differentiate between lossy and lossless data compression methods, and describe the most common such methods;

• design an efficient data compression scheme for a given information source;

• calculate the capacity of communication channels;

• sketch Shannon's proof regarding the limits of error-free communication; and

• explain the impact of feedback and/or many senders or receivers on the communication problem.

Prerequisites: Basic courses in mathematics.
Literature: T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley, New York, 1991.
Lectures: Mondays 9-12, room I346 and Wednesdays 9-12, room E110, first lecture: 5.11.2007. WWW page for lectures
Teacher: Prof. Patric Östergård
Tutorials: Tuesdays 14-16, hall S3 and Fridays 12-14, hall S2, first tutorial: 6.11.2007. WWW page for tutorials and home assignments.
Assistant: Vesa Vaskelainen, M.Sc. (Tech.), room I436, tel. 451 2401, e-mail: vesa.vaskelainen@tkk.fi
Registration: In WWWTopi.
To pass the course: Project (P), five home assignments (A), and exam (E).
Mark: max(E,P-1,A-2). All parts must be passed.