An Introduction to Information Theory

by Fazlollah M. Reza

Written for an engineering audience, this book has a threefold (1) to present elements of modern probability theory — discrete, continuous, and stochastic; (2) to present elements of information theory with emphasis on its basic roots in probability theory; and (3) to present elements of coding theory.The emphasis throughout the book is on such basic concepts as sets, the probability measure associated with sets, sample space, random variables, information measure, and capacity. These concepts proceed from set theory to probability theory and then to information and coding theories. No formal prerequisites are required other than the usual undergraduate mathematics included in an engineering or science program. However, since these programs may not include a course in probability, the author presents an introductory treatment of probability for those who wish to pursue the general study of statistical theory of communications. The book is divided into four memoryless discrete themes, memoryless continuum, schemes with memory, and an outline of some recent developments. An appendix contains notes to help familiarize the reader with the literature in the field, while the inclusion of many reference tables and an extensive bibliography with some 200 entries makes this an excellent resource for any student in the field.

Genres: Mathematics, Science, Nonfiction, Technology, Reference

528 pages, Kindle Edition

First published January 1, 2012

Reviews

[Aryan Prasad · Rating 2 out of 5]

The book was written merely 13 years after Shannon's paper, and the field was well in its infancy so a lot of things were are not settled down in the book. That and my edition being badly printed, with many a + - missing makes me think that the book can only serve a historic purpose now.

[Tom Schulte · Rating 4 out of 5]

Originally published in 1961, this work on the surface appears to be a graduate-level text for computer engineering students. As such, it may seem to have only historical interest. It would be tempting to skip this over for more up to date works. However, besides being a light in the constellation of seminal works on information theory, this is still very illustrative and enlightening, especially for someone mathematically sophisticated enough merely for elementary probability and set theory with an interest in the development of information theory. The first four chapters take the reader from elementary probability to information theory basics. Additionally, classroom or independent study can be augmented with a final chapter on group codes. This excellent introduction to the coding theory topic in chapter thirteen does not require the higher mathematics of earlier chapters.

Merged review:

Originally published in 1961, this work on the surface appears to be a graduate-level text for computer engineering students. As such, it may seem to have only historical interest. It would be tempting to skip this over for more up to date works. However, besides being a light in the constellation of seminal works on information theory, this is still very illustrative and enlightening, especially for someone mathematically sophisticated enough merely for elementary probability and set theory with an interest in the development of information theory. The first four chapters take the reader from elementary probability to information theory basics. Additionally, classroom or independent study can be augmented with a final chapter on group codes. This excellent introduction to the coding theory topic in chapter thirteen does not require the higher mathematics of earlier chapters.

[William Schram · Rating 4 out of 5]

Information Theory is a fascinating topic of inquiry. An Introduction to Information Theory is by Fazlollah M Reza and published in 1961. Considering the speed at which computers and other pieces of technology evolve, this is a red flag for a student. If I recall, Gordon Moore put forth his observation in 1965, so this book predates integrated circuits.

I am not a practitioner in the field, and I don't have any newer books to reference, so I don't know what it doesn't know. I assume they updated the Coding techniques used, and some other various improvements furthered efficiency. Some of the pages and tables are redundant because of the march of technology.

On the other hand, as the book states, you don't need prerequisite knowledge to read this book. It thoroughly explains the symbols used and the equations employed. That alone garners it four stars. In addition, it contains example problems, and we all know that I love those.

It is easy to breeze through the basics, but you have to remember that they provide a foundation. It's all well and good to be an electrical engineer, but when you need to examine a circuit, you still rely on Ohm's Law.

Given the period and what Reza knew, An Introduction to Information Theory is a superb book. The author balances the words and equations well. Thanks for reading my review, and see you next time.