The Art of Multiprocessor Programming

by Maurice Herlihy, Nir Shavit

The Art of Multiprocessor Programming promises to be the first comprehensive presentation of the principles and tools available for programming multiprocessor machines. As the computer industry changes from single-processor to multiprocessor architectures, this revolution requires a fundamental change in how programs are written. To leverage the performance and power of multiprocessor programming, also known as multicore programming, programmers need to learn the new principles, algorithms, and tools. The book will be of immediate use to programmers working with the new architectures. For example, the next generation of computer game consoles will all be multiprocessor-based, and the game industry is currently struggling to understand how to address the programming challenges presented by these machines. This change in the industry is so fundamental that it is certain to require a significant response by universities, and courses on multicore programming will become a staple of computer science curriculums. This book includes fully-developed Java examples detailing data structures, synchronization techniques, transactional memory, and more. Students in multiprocessor and multicore programming courses and engineers working with multiprocessor and multicore systems will find this book quite useful.

Genres: Programming, Computer Science, Technology, Computers, Nonfiction, Technical, Software

528 pages, Paperback

First published February 29, 2008

Reviews

[Bugzmanov · Rating 5 out of 5]

This book is very uneven. And yet I think this is one of the best (if not THE best) books on concurrency.
It's starts very light with metaphors of parallelism & coordination from real life, but then hits you with formalism & hardcore computer science theory. Some of it is really hard to follow and despite the authors of the book being the authors of original discoveries (things like impossibility of using simple registers for consensus and coordination) i found it easier to look for alternative explanations (https://www.youtube.com/watch?v=3ZR-R...).

Theoretical part is a hump you need to slug through to get to the real gems: chapters 7 - 10 are amazingly good. Chapter 7 alone makes this book worth reading.

Surprisingly I find the chapters that present body of academic work done by the authors (chapters 5,6, 12) the most hard to understand. And it's relatively hard to find alternative explanation for dummies. Most of my google search lead to white papers created by same people and laying out the info in the same manner.

Books that I recommend reading along this one:
* A Primer On Memory Consistency And Cache Coherence - gives perspective on the topic from hardware perspective. There is good overlap on locks & data structures. But it worth looking at it from different perspectives.
* Is Parallel Programming Hard, And, If So, What Can You Do About It? - gives perspective that GC-based runtimes have it easy, and concurrency in native environments is even more challenging.
* Java Concurrency in Practice - it's a better book to understand what are the fundamentals of JVM platform when it comes to concurrency. Surprisingly "The Art of Multiprocessor Programming" have couple of bugs that will lead to data races. So "the art ..." is not the book to learn about jvm & concurrency primitives in java language

[Phin Jensen]

I read this over the course of a few months for Phil Eaton's Software Internals Book Club, and very quickly fell behind on the readings. The theory was and still is difficult for me to understand, but there was a sweet spot about in the beginning of the application part where I felt like I was understanding things. It didn't last.

I can't say that it's not a good book, but I had a hard time with it. To be clear, this is an academic book that is geared towards university classroom use (and I wouldn't be surprised if it typically gets used in graduate courses). I may have learned a lot more from it if I had a professor and TAs to ask clarifying questions from. I could have asked questions in the book club, but I guess the real issue is that I didn't have anything forcing me to understand this stuff in depth, and I was juggling a lot of other things.

But I decided to finish the book because the moments of understanding were satisfying, and I believe there's value in knowing about things, even if you don't really know them.

[r · Rating 4 out of 5]

The book is split up into a Principles and a Practice section. The former is especially nice. In particular, the result that wait-free objects with consensus number greater than 1 cannot be implemented using atomic registers is now one of my favourites in computer science. It implies that we really need do other hardware primitives than plain old read & writes for most concurrent algorithms, and this conclusion about hardware requirements is reached through a very transparent, abstract argument.

The main lesson, in my opinion, is that concurrent algorithms are extremely delicate given the presently available language constructs. Your compiler will screw you over, wrongly ordered statements will screw you over, memory management will screw you over, forgetting to use a synchronization primitive somewhere will screw you over, and so on. This book teaches you to be mindful about all of these issues, which I think makes it more valuable than the average algorithm cookbook.

[Nikos Koukis · Rating 3 out of 5]

A bit too much Java for my taste. If you 're into threads and parallel programming though, this book goes fairly deep in terms of computer science, math as well as actual code implementations in Java.

[Kirill · Rating 5 out of 5]

The excellent book to understand deeply of underneath concurrent primitives. The book has a diverse range of examples and exercises. If you are not going to write your own concurrent library then you may not read it. However it's good to have an overview of tools you are daily using.

[Madhura Parikh · Rating 5 out of 5]

Fantastic book, I didn't read this exhaustively but it has a really great balance of theory vs practice, managing to dive deep in each case, and yet remaining highly accessible and engaging.

[Nilendu Misra · Rating 5 out of 5]

Worth reading for Chapter 7 alone. That chapter presents nuances of modern hardware architecture with clear code and its effects.

[Tim Tulsky · Rating 4 out of 5]

These books are great. Really easy to read and understand, but not too boring. A great foundation and/or a refresher.

[Ushan · Rating 4 out of 5]

In my day, synchronization was just one topic out of many in the undergraduate operating systems course, alongside process scheduling and virtual memory. Nowadays, however, with multicore processors appearing in $500 personal computers, this topic has become more important. The first author teaches a whole undergraduate course on it at Brown University using this book as a textbook, and the second author does so at Tel Aviv University. It has a long discussion of locks (spinlocks, reader-writer queue-based locks, and combinations thereof), lock-free processor instructions (the IBM/360 had one theoretically less powerful than the Intel x86 and newer processors), data structures based on lock-free processor instructions (lists, stacks, queues, hash tables, skip lists). and software transactional memory, which is supposed to be the future of multiprocessor programming.

[Grzegorz Sikora · Rating 4 out of 5]

This is book is not for pragmatic people. Mostly it is about theory behind multithread programming, but there are few practical chapters. I think it will be very hard to read for the beginner and most probably reader will no go and program good multithread programs after reading it.

The book the worst part are terrible code examples - not well chosen and they just are erroneous.

Polish translation of this book is not more than just acceptable. There is lack of unified translation of terms, sentences sound very unnatural.


Overall quite good book, academic-like but not too much.

[Mark · Rating 3 out of 5]

Not terribly clear and the code is riddled with errors, but as far as I can tell this is still the definitive text.
--------
note: the code here is in Java. For the purposes of theory, that's not a big deal, but I suspect practical translation of some of the algorithms to a different language with a different memory model may prove tricky if not impossible. The algorithms that assume the presence of a garbage collector look especially tricky; I don't know if it's possible to implement a lock-free concurrent skiplist in a language like C++, but if it is, the details are elsewhere.

[Scott · Rating 5 out of 5]

The days of Moore's law are moving to multiprocessors. Manufacturers are having difficulty increasing speed on single processors. thus, multiprocessors are necessary to continue improving software performance. The first third of the book evolves the terms and understanding of multiprocessor programming and the rest looks at the practice. This is written by two very outstanding professors who really understand the topic and provide some great content.

[DJ]

the most theory-driven intro to parallel computing I could find

[David Robins · Rating 3 out of 5]

Seems quite comprehensive, but the number of careless errors and a few confusing/incomplete explanations is off-putting, even though there are errata online.

[Suliman · Rating 3 out of 5]

على حد علمي الأفضل في فنه

[Erik Neel · Rating 3 out of 5]

Not always realistic. Not always usable

[Bob Hancock · Rating 4 out of 5]

Great exposition of a complex subject.