What do you think?
Rate this book
Deep Learning: Foundations and Concepts
This book offers a comprehensive introduction to the central ideas that underpin deep learning. It is intended both for newcomers to machine learning and for those already experienced in the field. Covering key concepts relating to contemporary architectures and techniques, this essential book equips readers with a robust foundation for potential future specialization. The field of deep learning is undergoing rapid evolution, and therefore this book focusses on ideas that are likely to endure the test of time. The book is organized into numerous bite-sized chapters, each exploring a distinct topic, and the narrative follows a linear progression, with each chapter building upon content from its predecessors. This structure is well-suited to teaching a two-semester undergraduate or postgraduate machine learning course, while remaining equally relevant to those engaged in active research or in self-study.
A full understanding of machine learning requires some mathematical background and so the book includes a self-contained introduction to probability theory. However, the focus of the book is on conveying a clear understanding of ideas, with emphasis on the real-world practical value of techniques rather than on abstract theory. Complex concepts are therefore presented from multiple complementary perspectives including textual descriptions, diagrams, mathematical formulae, and pseudo-code. Chris Bishop is a Technical Fellow at Microsoft and is the Director of Microsoft Research AI4Science. He is a Fellow of Darwin College Cambridge, a Fellow of the Royal Academy of Engineering, and a Fellow of the Royal Society. Hugh Bishop is an Applied Scientist at Wayve, a deep learning autonomous driving company in London, where he designs and trains deep neural networks. He completed his MPhil in Machine Learning and Machine Intelligence at Cambridge University. “Chris Bishop wrote a terrific textbook on neural networks in 1995 and has a deep knowledge of the field and its core ideas. His many years of experience in explaining neural networks have made him extremely skillful at presenting complicated ideas in the simplest possible way and it is a delight to see these skills applied to the revolutionary new developments in the field.” -- Geoffrey Hinton " With the recent explosion of deep learning and AI as a research topic, and the quickly growing importance of AI applications, a modern textbook on the topic was badly needed. The "New Bishop" masterfully fills the gap, covering algorithms for supervised and unsupervised learning, modern deep learning architecture families, as well as how to apply all of this to various application areas." – Yann LeCun
“This excellent and very educational book will bring the reader up to date with the main concepts and advances in deep learning with a solid anchoring in probability. These concepts are powering current industrial AI systems and are likely to form the basis of further advances towards artificial general intelligence.” -- Yoshua Bengio
A full understanding of machine learning requires some mathematical background and so the book includes a self-contained introduction to probability theory. However, the focus of the book is on conveying a clear understanding of ideas, with emphasis on the real-world practical value of techniques rather than on abstract theory. Complex concepts are therefore presented from multiple complementary perspectives including textual descriptions, diagrams, mathematical formulae, and pseudo-code. Chris Bishop is a Technical Fellow at Microsoft and is the Director of Microsoft Research AI4Science. He is a Fellow of Darwin College Cambridge, a Fellow of the Royal Academy of Engineering, and a Fellow of the Royal Society. Hugh Bishop is an Applied Scientist at Wayve, a deep learning autonomous driving company in London, where he designs and trains deep neural networks. He completed his MPhil in Machine Learning and Machine Intelligence at Cambridge University. “Chris Bishop wrote a terrific textbook on neural networks in 1995 and has a deep knowledge of the field and its core ideas. His many years of experience in explaining neural networks have made him extremely skillful at presenting complicated ideas in the simplest possible way and it is a delight to see these skills applied to the revolutionary new developments in the field.” -- Geoffrey Hinton " With the recent explosion of deep learning and AI as a research topic, and the quickly growing importance of AI applications, a modern textbook on the topic was badly needed. The "New Bishop" masterfully fills the gap, covering algorithms for supervised and unsupervised learning, modern deep learning architecture families, as well as how to apply all of this to various application areas." – Yann LeCun
“This excellent and very educational book will bring the reader up to date with the main concepts and advances in deep learning with a solid anchoring in probability. These concepts are powering current industrial AI systems and are likely to form the basis of further advances towards artificial general intelligence.” -- Yoshua Bengio
669 pages, Hardcover
Published November 2, 2023
47 people are currently reading
332 people want to read
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 8 of 8 reviews
March 29, 2024
The PRML book was a good part of my ML education, and I was looking forward to reading this new book. It has all the qualities of PRML: good organization of the material, great figures, and everything connected to probabilistic models is some of the best material I have found around. The first part of the book is a bit weird, because you can feel it's a patch-up from older material (mostly from PRML), and in some cases it's not really clear why some sections were kept (including some in-depth discussions about maximum likelihood on certain distributions, or K-means clustering and EM). The middle part covers basic NN layers (convolutional, recurrent, etc.). It's good but it's a quick exposition compared to other books. The last chapters cover generative models and they are incredibly good (as expected). Overall it's a fantastic addition to any library but (like PMLR) it's a more advanced book than most other introductions.
February 17, 2025
A very modern and readable introduction to the basics of machine learning. Compared to its predecessor, it feels less Bayesian in spirit but still incorporates more of that perspective than the ML course I took.
What I appreciated most was how well the book motivates the key decisions in modern ML. This makes studying it not just informative but genuinely engaging.
What I appreciated most was how well the book motivates the key decisions in modern ML. This makes studying it not just informative but genuinely engaging.
February 18, 2025
Used in the ML class at Berkeley. From a theoretical perspective, it is a wonderful introduction. Most books either lack mathematical rigor or lose you in the dense math without any grounds for intuition. This book is the best of both worlds: intuition through lots of graphs and pictures described with precise math.
Probably won’t help too much if you’re just training some models at a high level. Most frameworks already implement them and abstract away the math and performance optimization. Great if you’re just curious or want to explore research in this area or push the boundary of the field or just train better models.
*I was able to correct some minor mistakes in the textbook since it is only recently published (AI is a pretty new field compared to physics). I’ve been in light contact with Chris Bishop and he is very nice!
Probably won’t help too much if you’re just training some models at a high level. Most frameworks already implement them and abstract away the math and performance optimization. Great if you’re just curious or want to explore research in this area or push the boundary of the field or just train better models.
*I was able to correct some minor mistakes in the textbook since it is only recently published (AI is a pretty new field compared to physics). I’ve been in light contact with Chris Bishop and he is very nice!
April 28, 2024
This excellent textbook serves as a sequel to the book "Pattern Recognition and Machine Learning", with several already covered chapters in the latter. It provides comprehensive coverage of the foundational principles and concepts in the field of Deep Learning, presented in a clear and cohesive manner. However, it is worth noting that there are several typos scattered throughout the text.
August 18, 2024
Review based on the first 11 chapters as I'm still reading, but I'm not entirely convinced so far.
A lot of the mathematical background seems pretty irrelevant as presented. For example, design matrices and the Moore-Penrose pseudo inverse are introduced on page 116, then never mentioned again (I've searched the whole book for both terms). Maybe they are still relevant, but if so, the book never explains why.
At several stages, terms are introduced in a way that is either unclear or inaccurate. For example, page 172 says that networks having more than one layer of learnable parameters are known as feed-forward networks or multilayer perceptrons, but "feed-forward" should be reserved for acyclic networks, and MLPs would additionally be fully connected. Page 347 says that "a conditional independence property that is helpful when discussing more complex directed graphs is called the Markov blanket or Markov boundary", but these aren't interchangeable terms and neither of them are clearly defined in the text, which talks about "the" Markov blanket rather than "a" Markov blanket even though Markov blankets aren't usually unique. A Markov boundary specifically refers to a minimal Markov blanket. Without explicit and accurate definitions, the text is harder to follow than it would be otherwise.
I'm a bit concerned about attention to detail in general. The entirety of page 7 is devoted to an example output from GPT-4 from the prompt "Write a proof of the fact that there are infinitely many primes; do it in the style of a Shakespeare play through a dialogue between two parties arguing over the proof", but the book never points out that the proof as presented is wrong (Q is coprime to the other primes, but not necessarily prime itself).
An additional very minor quibble is the use of "error function" rather than "loss function". While not incorrect, this feels non-standard and results in E(x) representing both loss functions and energy functions (and expectations if you include \mathbb{E}). The book also discusses Erf(x), so "error function" means different things depending on what section you're in.
On the plus side, the book gives an up-to-date treatment of a good range of topics. A lot of the illustrations are very helpful, and it doesn't shy away from mathematical detail. I'm likely to persevere with the rest of the book, which will hopefully stick to the point a lot more, but so far I much prefer Understanding Deep Learning (Prince).
A lot of the mathematical background seems pretty irrelevant as presented. For example, design matrices and the Moore-Penrose pseudo inverse are introduced on page 116, then never mentioned again (I've searched the whole book for both terms). Maybe they are still relevant, but if so, the book never explains why.
At several stages, terms are introduced in a way that is either unclear or inaccurate. For example, page 172 says that networks having more than one layer of learnable parameters are known as feed-forward networks or multilayer perceptrons, but "feed-forward" should be reserved for acyclic networks, and MLPs would additionally be fully connected. Page 347 says that "a conditional independence property that is helpful when discussing more complex directed graphs is called the Markov blanket or Markov boundary", but these aren't interchangeable terms and neither of them are clearly defined in the text, which talks about "the" Markov blanket rather than "a" Markov blanket even though Markov blankets aren't usually unique. A Markov boundary specifically refers to a minimal Markov blanket. Without explicit and accurate definitions, the text is harder to follow than it would be otherwise.
I'm a bit concerned about attention to detail in general. The entirety of page 7 is devoted to an example output from GPT-4 from the prompt "Write a proof of the fact that there are infinitely many primes; do it in the style of a Shakespeare play through a dialogue between two parties arguing over the proof", but the book never points out that the proof as presented is wrong (Q is coprime to the other primes, but not necessarily prime itself).
An additional very minor quibble is the use of "error function" rather than "loss function". While not incorrect, this feels non-standard and results in E(x) representing both loss functions and energy functions (and expectations if you include \mathbb{E}). The book also discusses Erf(x), so "error function" means different things depending on what section you're in.
On the plus side, the book gives an up-to-date treatment of a good range of topics. A lot of the illustrations are very helpful, and it doesn't shy away from mathematical detail. I'm likely to persevere with the rest of the book, which will hopefully stick to the point a lot more, but so far I much prefer Understanding Deep Learning (Prince).
February 20, 2024
The book is a clear, thorough guide to deep learning that's great for beginners and experts alike. It's well-organized, making complex ideas easy to understand while studying independently. With the foundational math provided, it is a must-read for anyone diving into deep learning. I am a first-year PhD Comp Sci student and found it very helpful.
September 2, 2024
reuse เนื้อหาจาก Pattern Recognition and Machine Learning พอสมควร
June 30, 2025
all heuristics. latest trends in what seems to work and guesses for what's happening. i hate living in a world with this at its center.
Displaying 1 - 8 of 8 reviews