An unbelievably insightful and tantalising look at the similarities and differences between computers and the human nervous system. At just over a hundred pages it can easily be read in a day, though some background knowledge of computing and neuroscience would help. Although von Neumann was writing in 1958, his thinking is just as relevant today as it was then. By today's standards the section on digital computers may seem quaint, but it is a succinct and interesting summary of the state of affairs then, and the explanations are easily extended to today's technology. The section on analog computers, by contrast, is fascinating, because it's easy to forget that they were such an important predecessor to digital machines.
The fundamental question of the paper is: "To what extent are we digital, and to what extent analog, and what might our understanding of computers tell us about the way our brains work?" The "winner-takes-all" nature of neurons firing and the binary nature of genetics imply that a large part must be digital, and yet the way neurons use time summation/frequency and several other mysteries about the statistical nature of our nervous systems mean we must be at least in part analog.
The massive increase in computing power in the meantime makes updating von Neumann's figures as an implicit exercise for the reader. Obviously computers have come a long way since then, while the brain hasn't, and yet virtually everything von Neumann writes still applies. The brain has far more neurons than any single computer has transistors, and neurons interact with so many other neurons in such complex ways (not just electrically but chemically, mechanically, spatially, temporally) that to treat them as equivalent to any single "active organ" (i.e. transistor) is a mistake. The questions of how memory works in the nervous system and the limitations of mathematical precision in the nervous system capable are equally fascinating, as is the question of parallel versus serial processing and their respective memory requirements.
In some ways, this book is interesting precisely because of its simplicity in comparing the two systems. Today, most people would dismiss the question by saying that the brain is too complex to compare to a computer. But why and how is this the case? That his writing is cogent and coherent for non-experts also makes this a great read. Highly recommended.