Coding and Information Theory

by Richard Hamming

Chpt 1-Intro, Chpt 2-Error-Detecting Codes, Chpt 3- Error Correcting Codes, Chpt 4-Variable-Length Huffman, Chpt 5-Miscellaneous Codes, Chpt 6-Entropy and Shannon's first Theorem, Chpt 7- Channel and Mutual Information, Chpt 8- Channel Capacity, Chpt 9- Some Mathematical Preliminaries, Chpt 10- Shannon's Main Theorem, Chpt 11- Algebraic Coding, Appendix Bandwidth and the Sample Theorem, Appendix Some tables for Entropy Calculations.

Genres: Mathematics, Computer Science

259 pages, Hardcover

First published February 1, 1980

About the author

Professor Richard Wesley Hamming, Ph.D. (University of Illinois at Urbana-Champaign, 1942; M.S., University of Nebraska, 1939; B.S., University of Chicago in 1937), was a mathematician whose work had many implications for computer science and telecommunications. His contributions include the Hamming code (which makes use of a Hamming matrix), the Hamming window (described in Section 5.8 of his book Digital Filters), Hamming numbers, sphere-packing (or hamming bound) and the Hamming distance.

Hamming was a professor at the University of Louisville during World War II, and left to work on the Manhattan Project in 1945, programming one of the earliest electronic digital computers to calculate the solution to equations provided by the project's physicists. The objective of the program was to discover if the detonation of an atomic bomb would ignite the atmosphere. The result of the computation was that this would not occur, and so the United States used the bomb, first in a test in New Mexico, and then twice against Japan. Later, from 1946 to 1976, he worked at the Bell Telephone Laboratories, where he collaborated with Claude Shannon. During this period, he was an Adjunct Professor at the City College of New York, School of Engineering. On July 23, 1976 he moved to the Naval Postgraduate School, where he worked as an Adjunct Professor until 1997, when he became Professor Emeritus. He died a year later in 1998.

He was a founder and president of the Association for Computing Machinery. His philosophy on scientific computing appears as preface to his 1962 book on numerical methods: The purpose of computing is insight, not numbers.

Awards:
Turing Award, Association for Computing Machinery, 1968.
Fellow of the IEEE, 1968.
IEEE Emanuel R. Piore Award, 1979.
Member of the National Academy of Engineering, 1980.
Harold Pender Award, University of Pennsylvania, 1981.
IEEE Richard W. Hamming Medal, 1988.
Fellow of the Association for Computing Machinery, 1994.
Basic Research Award, Eduard Rhein Foundation, 1996.
Certificate of Merit, Franklin Institute, 1996

The IEEE Richard W. Hamming Medal, named after him, is an award given annually by Institute of Electrical and Electronics Engineers (IEEE), for "exceptional contributions to information sciences, systems and technology", and he was the first recipient of this medal.

Hamming discusses the use and potential of computers in the 1965 film Logic By Machine.

Reviews

[Matthew MacLennan · Rating 4 out of 5]

Written by the great Hamming, this book is a perfect balance of information theory and coding theory.

The coding theory examples begin from easy-to-grasp concepts that you could definitely do in your head, or at least visualize them.

The information theory concepts are the fundamentals and they are presented using intuitive notions and applied to other areas of science (e.g., biological evolution) and communication.

The book was written to be a educational text on the subject, and it is written well for that purpose: You can get a good feel of what information theory and coding theory are all about and subjects are supplemented with accessible mathematical examples.

I would say this book is a great companion for a cursory, but comprehensive look at information theory and coding theory and would be a good choice for an introductory text on the topic.

[Luciano Sanzaro · Rating 5 out of 5]

I already read books and followed university courses about these topics, but in this book information theory is explained as if its ideas and insights come out naturally from the very first assumptions. This is probably due to the fact that Hamming spent a lot of time with Claude Shannon, Brockway McMillan, Robert Fano and other pioneers of information theory.
Concepts are solidly explained with a lot of examples, some theorems are explained with multiple proofs in a sort of combinatorial style.

Little space is left for the error correcting codes part, but the essential ideas are presented very well without the need to digress on abstract algebra; indeed, the general concepts behind CRCs, explained as the polynomial version of the Hamming codes or even-weight Hamming codes, are introduced together with some ideas behind correction of more than one error.

The book is full of practical “computer” examples that are not usually part of information theory. This addition may be misleading to the novice but it’s certainly welcome to the reader with some training.
I’m quite sure Hamming loved the whole field of information theory and would have spent more time on it, his enthusiasm is transferred to the reader during the discussion.

[Bria]

Hamming is obviously not a teacher, as he'll start off with walking the reader through painfully simple examples, only to jump about six levels up without any explanation and leave it to us to figure out how on earth something follows from anything else. A good introduction to a lot of the ideas of coding and information theory, though, and presented in a different manner than every other text on the subject, which, at least from my persuals, seem to have come to settle on fairly similar general methods of teaching it. Not recommended if you want to actual do any coding or anything though, but a good launching point.

[Ivana · Rating 4 out of 5]

very helpfull for the lecture I had to write :) and if you are interested in Information theory or codes.