The Deep Learning Revolution

by Terrence J. Sejnowski

How deep learning—from Google Translate to driverless cars to personal cognitive assistants—is changing our lives and transforming every sector of the economy.

The deep learning revolution has brought us driverless cars, the greatly improved Google Translate, fluent conversations with Siri and Alexa, and enormous profits from automated trading on the New York Stock Exchange. Deep learning networks can play poker better than professional poker players and defeat a world champion at Go. In this book, Terry Sejnowski explains how deep learning went from being an arcane academic field to a disruptive technology in the information economy.

Sejnowski played an important role in the founding of deep learning, as one of a small group of researchers in the 1980s who challenged the prevailing logic-and-symbol based version of AI. The new version of AI Sejnowski and others developed, which became deep learning, is fueled instead by data. Deep networks learn from data in the same way that babies experience the world, starting with fresh eyes and gradually acquiring the skills needed to navigate novel environments. Learning algorithms extract information from raw data; information can be used to create knowledge; knowledge underlies understanding; understanding leads to wisdom. Someday a driverless car will know the road better than you do and drive with more skill; a deep learning network will diagnose your illness; a personal cognitive assistant will augment your puny human brain. It took nature many millions of years to evolve human intelligence; AI is on a trajectory measured in decades. Sejnowski prepares us for a deep learning future.

Genres: Science, Artificial Intelligence, Nonfiction, Technology, Computer Science, History, Neuroscience

461 pages, Kindle Edition

First published October 1, 2018

About the author

Terrence Joseph Sejnowski is an Investigator at the Howard Hughes Medical Institute and is the Francis Crick Professor at The Salk Institute for Biological Studies where he directs the Computational Neurobiology Laboratory. His research in neural networks and computational neuroscience has been pioneering.
Sejnowski is also Professor of Biological Sciences and Adjunct Professor in the Departments of Neurosciences, Psychology, Cognitive Science, and Computer Science and Engineering at the University of California, San Diego, where he is Director of the Institute for Neural Computation. In 2004 he was named the Francis Crick Professor and the Director of the Crick-Jacobs Center for Theoretical and Computational Biology at the Salk Institute.

Reviews

[Randy · Rating 1 out of 5]

This book is NOT about deep learning. Despite its grand title, this book is strictly a memoir.

Since about 1980 Professor Terrence Sejnowski has worked with biologically inspired neural nets. However, his work is NOT notable for the theory behind or the development of deep neural nets (AKA deep learning). As such, this book consists of a very lightweight historical introduction to some of the people who were active in the early days of NNs, and Dr Sejnowski's kinship with them, but little more.

The text explains none of the technical details of Sejnowski's work, nor the work of other NN pioneers, nor does the author offer any insights into any future work he anticipates taking place in NNs or in deep learning.

Frankly, titled as it is, this book is a HUGE disappointment. It simply doesn't describe the past, present, or future of deeplearning nor any other facet of artificial intelligence. Read it only if you want a quick tour through the chapters of Dr. Sejnowski's career.

[Matthew · Rating 4 out of 5]

I disagree with the other reviews that state this is purely a 'memoir.' It is more than that.
Terry describes the evolution of neural networks and the personalities that helped the field along the way. He was a part of it, hence the 'memior' theme, but goes beyond his own contributions and neatly outlines the evolution of the idea from both personnel and theoretical perspectives. This won't teach you how to perform deep learning, but will help you understand how we got to where we are today.

[William Boyle · Rating 3 out of 5]

This book is not about deep learning. You'll learn more about how the eye works than anything else.. was interesting, just not really even remotely what I was expecting given the title...

[Carsten · Rating 4 out of 5]

4.5/5. Good book with many historical details. The author is clearly at the forefront of research and I liked that he gives references (up to 2017). But I wish there would be more details in places. I am aware that this book is not a textbook but I wish the Sejnowski would provide more technical details. As a bird eyes view however, I highly recommend the book to everyone who is interested in neural networks and the deep learning revolution. It whet my appetite for more (I think I will pick up his book with Churchland (The Computational Brain)).

[Bouga Bougi · Rating 1 out of 5]

Half of this book consists of Sejnowski flattering himself, the rest of the content is poorly explained. I learned nothing from this book that I did not already learn from much easier readings. This book is nothing but a waist of paper.

[PetitShya™ · Rating 4 out of 5]

这大佬前面也只是顺带介绍深度学习领域的基础算法和脉络就已经很深奥了，差点就读不下去了（当然有人认为科普著作写这些也没有必要），但是中间有介绍很多你之前从未了解过的人（包括很多女性，例如天才物理学少女 Misha Mahowald，她19岁就创造了世界上第一个硅视网膜。）这些人极具智识，你可能大致会感觉到世界很复杂，但真正看到复杂系统的诞生的时候，你仍然会惊异这种顶级智慧高度集中的领域，并且感叹智力上限（和下限一样）是没有边界的。

[Allan Olley · Rating 5 out of 5]

This is a review of the developments in machine learning based around neural networks mostly since the 1980s, but with some look back at earlier developments. Even within that domain I expect he focuses on particular subsections. The account starts with some recent achievements of machine learning such as the world beating Go playing of the AlphaGo and AlphaZero systems and proceeds through a mix of chapters some proceeding diachronically (according to a time), but often by theme switching between themes like the development in vision systems and then going back to discuss various learning algorithms.

The book is aimed at a lay audience but contains the broad outlines of some technical issues. It is full of the author's opinions on development and also their speculation on broader issues such as education and the way scientific research operates. It provides an interesting perspective on developments and ways of thinking about them. The author is happy to state his opinions on the virtues and deficiencies he sees in other researchers and their approaches. He is especially critical of symbolic approaches to artificial intelligence, although he does do some work to try and explain and motivate this point of view.

The ebook is pretty good but there are a few problems. For instance I think at one point the exponent of a number is treated as an endonte. Also in some cases the link to a figure is immediately adjacent to a endnote and it is difficult to click on the link to the endnote as a result (instead you tend to actuate the link to the figure).

[Adrian · Rating 5 out of 5]

This is one of the best scientific biographies I have ever read. It is short. It is full of insights. It tells an interesting story. It presents both the man and his friends. It explains what this revolution is all about. The only two problems: (i) the revolution is still ongoing; (ii) sometimes the author skips years making me wonder what happened within those time periods. Even with all these shortcomings it is still in the same league with Eric Kandel and other great scientific writers. Here's hoping a second volume will focus on the last decade.

[Andy Masley · Rating 5 out of 5]

Confused about the bad reviews. Thought this was great all around.

[Tona · Rating 4 out of 5]

Una excelente lectura para empaparse en la historia de las redes neutrales y el deep learning. Un poco inclinado a lo anecdótico pero muy bueno de cualquier manera.

[Jung]

The journey of artificial intelligence, as detailed in "The Deep Learning Revolution" by Terrence J. Sejnowski, unveils how an unconventional idea radically transformed technology and our understanding of intelligence. For years, scientists attempted to teach machines using logic and rules, believing computers needed precise instructions to perform tasks like facial recognition or language translation. But this method, while orderly, lacked flexibility and failed to deliver the depth of understanding that comes naturally to even a young child. A child, after all, doesn’t learn what a face looks like by reading a rulebook; they learn by exposure—seeing thousands of examples until recognition becomes intuitive. A handful of pioneering researchers dared to propose that machines might be able to do the same. Instead of programming intelligence directly, what if they allowed computers to learn from raw experience, just as humans do?

This idea sparked a revolution. Instead of following the rigid logic of symbolic AI—the dominant paradigm in the 1980s that mirrored philosophical reasoning—the rebels of AI, including Sejnowski and Geoffrey Hinton, proposed a model inspired by the human brain. The brain, they noted, is not a rule-following machine. It learns through interactions between billions of neurons, forming and reforming connections based on experience. When you ride a bike, you don’t follow written instructions to stay balanced—you fall, adjust, and eventually master it through neural trial and error. These researchers, scorned at first, introduced the world to connectionism: the idea that intelligence could emerge from networks that learn from data, not rules.

Despite skepticism and a lack of support, these visionaries pressed on, fueled by the belief that biology had already solved the challenges traditional AI couldn’t. If birds could fly, babies could learn to speak, and animals could navigate danger without pre-written instructions, then perhaps machines could emulate the learning systems of living creatures. Their early efforts produced artificial neural networks—simple systems modeled on real neurons, capable of adjusting their connections to reinforce useful patterns and discard noise. These artificial brains didn’t need to be taught the appearance of a cat—they could learn it from seeing thousands of pictures, identifying the subtle traits that separate one image from another.

One of the breakthroughs came when researchers recognized the value of randomness. In biological systems, some neural activity appears chaotic, but that randomness actually allows the brain to break out of bad habits and find better solutions. Inspired by this, Sejnowski and Hinton developed Boltzmann machines, early networks that explored possible answers through randomness and settled on the most efficient ones. But even more transformative was their discovery of backpropagation—a method for networks to learn from their own mistakes. Just as humans improve by reflecting on failure, artificial networks could strengthen the correct paths and weaken the ones that led to error. This insight unlocked the potential to build systems that grow smarter with experience.

At the time, however, they faced a major limitation. These networks were conceptually sound, but computationally weak. They lacked the hardware, data, and algorithms needed to match their ambitions. Their designs were brilliant, but the engines too small. That changed dramatically in the 2000s. Graphics processing units (GPUs), originally built for rendering video games, offered the perfect architecture for neural network computations. Meanwhile, the internet became an endless source of training data—images, texts, sounds, and user behaviors—ready to fuel the new engines of learning. With better algorithms in hand, the AI rebels finally had the ingredients to realize their vision.

What followed was a series of astonishing advances. Image recognition systems trained on vast datasets could now identify objects in photos more accurately than humans. Translation services like Google Translate evolved from robotic syntax to fluent conversation. These systems didn’t memorize translations; they found the hidden mathematical structures that linked different languages. The game of Go, once considered untouchable by AI due to its immense complexity, was conquered by AlphaGo—a deep learning system that not only beat human champions but developed creative strategies never seen before. Self-driving cars began interpreting road signs, pedestrians, and traffic with ease. Voice assistants became conversational partners. Fraud detection systems outpaced human experts. Deep learning wasn’t just theoretical anymore; it had become practical, scalable, and immensely powerful.

Despite all these achievements, there remained a fundamental difference between machines and humans. Current AI, no matter how sophisticated, lacks the kind of embodied experience that shapes human understanding. A child learns what 'hot' means not just by hearing the word, but by touching something warm and forming a memory. AI lacks this physical, sensory learning. It can process millions of examples but doesn’t truly 'feel' or 'experience' the world. Human cognition is grounded in emotion, sensation, and social awareness. We adapt constantly, not just by learning new information, but by integrating it into a lived, physical context.

Even more critically, humans possess something machines still struggle to replicate: common sense. Ask a person why someone might carry an umbrella on a sunny day, and they can infer hidden context like the possibility of rain later. An AI, unless trained on such scenarios, might miss the nuance. Our intelligence is deeply tied to our bodies, environments, and emotional states—something no neural network, no matter how vast, can yet reproduce.

Nonetheless, AI continues to evolve. Researchers are exploring ways to make machines more flexible, capable of continual learning, and aware of context. Medical AI systems diagnose diseases with increasing precision. Climate models predict weather patterns with unprecedented accuracy. Educational platforms adapt in real time to student needs. At the same time, these capabilities bring complex challenges. Students can generate full essays with AI tools, forcing educators to rethink how they teach. Jobs are being replaced faster than workers can be retrained. And perhaps most troubling, deepfakes and AI-generated misinformation threaten our ability to distinguish truth from fiction. A tool that can create knowledge can also manipulate it.

Yet the same technology that creates these risks also offers solutions. AI systems can be trained to detect fabricated content, verify facts quickly, and support responsible journalism. The issue is not whether AI will advance, but how we will guide that advancement. Will we build systems that empower humanity or ones that exploit our weaknesses? The future of AI depends not just on engineers and researchers but on all of us—how we legislate, educate, and participate in this new world.

Ultimately, Sejnowski’s account is not just a technical history of deep learning, but a philosophical reflection on what intelligence really means. The rebels of the 1980s didn’t just build better machines—they questioned long-held assumptions about thought, learning, and the nature of knowledge itself. By studying the brain, they uncovered principles that allowed machines to recognize patterns, adapt to new situations, and even begin to reason. But human intelligence remains a blend of data, context, sensation, and emotion—something AI has yet to fully grasp.

The revolution that began by mimicking the brain has now turned into a broader collaboration between biology and technology. As machines become more capable, the focus turns toward building systems that don’t just replicate intelligence but enhance it. With careful thought and ethical guidance, AI can become not a rival to human minds, but a partner—one that helps us better understand ourselves, solve our greatest challenges, and expand the very definition of what it means to be intelligent.

[Raghu Nathan · Rating 4 out of 5]

Deep learning is a term you hear nowadays, a lot if you are working in the computer industry. Artificial intelligence, Neural networks, Machine learning, Big Data Analytics are some other terms we come across, particularly so if you live in Silicon Valley. For the non-computer person, it can all be confusing to sort out what each of these concepts means and how they impact our lives. I have had a reasonable interest in Artificial Intelligence ever since I saw Stanley Kubrick’s film, ‘2001: A Space Odyssey’, a long while ago. Recalling that film today, I would think the computer in that film, called HAL, was probably using an advanced version of deep learning through neural networks to do what it was doing. The author of this book, Dr. Sejnowski, is a pioneer and founder in the Deep learning field of AI. I started reading this book, hoping I will get some grasp of the principles and practices of deep learning so I will have a good understanding of the state-of-the-art in this field. However, it turned out that this is not primarily a technical book. It is more of a memoir, giving the history of the developments in the field and the key players who contributed significantly to the rise of deep learning. Still, the middle section of the book, comprising six chapters, gives an excellent background to the learning algorithms and other novel approaches such as Generative Adversarial Networks (GAN) and Reinforcement Learning. Along the way, there is much discussion on how the human brain works, how it sees, and how it learns.

The author defines Deep Learning as learning from data, just as how babies learn from the world around them. Babies start with fresh eyes and gradually gain the skills needed to navigate novel environments. With this definition, he takes an approach that places data ahead of theory. When vast amounts of data are available, and the volume of data is increasing all the time, why not start with an empirical approach and eventually let it lead us to a theory? Dr. Sejnowski says, “information comes from raw data and is used to create knowledge; knowledge leads to understanding; and understanding leads to wisdom” So, we do not need an overarching theory about how learning takes place. Instead, we start from a practical way in deep learning.
In the 1980s, the dominant paradigm in AI was to use symbols, logic, and rules to codify intelligent behavior. Deep learning was a radical departure from this approach, using neural networks to do AI. Neural networks have their origin in the work of Minsky and Papert on Perceptrons in the 1950s, which modeled a single biological neuron. However, in the 1950s, the intuitive conjecture was that there is no learning algorithm for multi-layer perceptrons. Hence, research on neural networks stopped until the 1970s. Then, in 1985, the author and Geoffrey Hinton published a learning algorithm for multi-layer networks. Gradually, with the explosion of computing power and storage in the first decade of the 21st century, it has become possible to build neural nets using supercomputers to have the size of a frog brain (about 16 million neurons). It is such networks that power Google Translator, self-driving vehicles, algorithmic stock trading, and Chess playing programs like Deep Blue. Even more, it was a neural network which detected within minutes that Donald Trump was having the highest emotional impact on a focus group in the first Republican primary debate before 2016.

This book presents the subject in three parts. Part 1 provides the background to deep learning and its origins. Part 2 is the technical part of the book which covers the evolution of the various forms of machine learning and neural network systems and their architecture. Part 3 talks about the different application domains of deep learning and how they influence us in today’s world. The book takes an upbeat view of deep learning and its promise for the future. It also eschews hype in its assessments. For example, the author talks at length about the limitations of neural nets.
One of the significant limitations in neural networks today is that they do not explain how they arrive at a decision. When an AI program makes a diagnosis on a patient that she has cancer, it becomes necessary to see the reasoning behind this conclusion. However, the author puts this limitation in perspective. He says that we can make the same argument about the human brain also, that it is a black box. When a doctor draws an inference from his decades of experience, we do not know how his brain arrives at it.
Another aspect of Neural networks is that they use cost functions, which they optimize before arriving at decisions. If we give a network the goal of maximizing profit as the only goal in making a decision, this cost function can cause a bias against specific elements of the problem under solution. If a bank denies an African-American a home mortgage through its neural network, it may have used many items of information that correlate with his minority status and applied them in maximizing profit in its cost function. His zip code and other data like substantial levels of incarcerations of African-American men could have minimized the profit output, which would make the Neural net to decline the mortgage. However, if we include ‘fairness’ as a second goal in the cost function, this bias can be mitigated.

Yet another question that always arises about AI is whether it will get smarter than humans and take over the world. The book does not address this question, but the author provides some comparisons between the human brain and today’s computers to give some balance to this argument. The human brain has 100 billion neurons, each connected to several thousand others, adding up to a thousand trillion synaptic connections. The power needed for this is 20 watts i.e., 20% of the energy necessary to run the entire body. In contrast, a petascale supercomputer, which is not nearly as powerful as the brain, consumes 5 MW or a quarter million times as much power. Nature evolved these technologies over millions of years, signaling and communicating at the molecular level and inter-connecting neurons in three dimensions.
In comparison, transistors on microchips connect in only two. So, a simple answer to computers fully taking over the world is that we still have plenty of catching up to do. The author reiterates this point in the final chapter through another comparison between humans and Chimpanzees. Only 1.4 percent of our three billion DNA base pairs differ from those in chimps. Our brains are remarkably similar to those of chimps; neuroanatomists have identified the same brain areas in both species. But our differences are at a molecular level and are subtle when one looks at the dramatic differences in our behaviors. Dr. Sejnowski concludes, saying, ‘once again, Nature is cleverer than we are.’

I enjoyed reading the book, getting to know about the pioneers in the field, and their contributions. The author’s enthusiasm promises us that machine learning based on big data can lead us to significant discoveries and even new theories on education and how the human brain works.
The holy grail in AI is Artificial General Intelligence (AGI), which we define as a machine capable of understanding the world like any human, and with the same capacity to learn how to carry out a vast range of tasks. AGI does not exist yet, even though we have seen it in science fiction novels and movies for almost a century. The films, The Matrix, and The Terminator depicted AGI as an evil force committed to enslaving and destroying humanity. When we get closer to creating AGI, perhaps artists and film-makers will develop a kinder, gentler view of AI and create a more benevolent supercomputer as its hero or heroine.

[Gregory Witek · Rating 2 out of 5]

I couldn't force myself to finish it. I don't think it's a terrible book, but it's not what I expected, and it's not written in a way that would encourage me to continue listening to it.

The author talks a lot about his own work in artificial intelligence, mentioning the people he met and talked to, like in a memoir. That part's easy to follow, but not that interesting. The other part is deeply technical, author explains various concepts like his own inveition - Boltzmann machine. This is really interesting, but I don't think it's explained well. I'm far from an expert in this area, but also I'm familiar with a lot of technical terms, and I struggled with the explanation.

I put the book on hold a few times and whenever I got back to it, I stopped after less than 30 minutes. Eventually I gave up.

[Bui Thang · Rating 4 out of 5]

Khó đọc. Ko chỉ giành riêng cho dân công nghệ, nó giành cho tất cả mọi người thế kỷ 21. Một cuộc cách mạng thực sự đang diễn ra. Cách mạng 4.0, cách mạng IoT, trong tương lai không xa rất nhiều nghành nghề sẽ đc thay thế bằng AI, bằng robot.

Cuốn sách về lịch sử deep learning. Những tưởng Deep learning mới đc nghiên cứu gần đây nhưng thực tế ko phải vậy. DL ko đơn thuần là lập trình, là coding, ní láuwj kết hợ giữa toán học, vật lý, sinh hịc và cả thân kinh học...

Với dl bạn sẽ hiểu đc tầm quan trong của lập trình đối với công dân thế kỷ 21.

Cuốn sách chỉ giành cho người kiên nhẫn.

[Nicolas · Rating 2 out of 5]

Disappointed after realising that this book was not about deep learning, I still pushed through it.

It is not an easy read, a lot of scientific concepts are not really explained (or at least that I couldn’t understand) and the concepts are convoluted.
It seems like the author expect us to have a good base of scientific understanding and does take for granted a lot of knowledge.

I have the impression that this book was mostly targeting a specific group in the scientific community and was not meant for the broad public.

I personally do not think that I have learned much from the book.

[thanh · Rating 2 out of 5]

this one is my fault for not doing careful research beforehand... it has interesting insights but is very technical and requires a lot of prior knowledge. would be suitable for someone learning about neural networks in-depth, not for beginners or those looking for analysis of real-world applications. the writing is very good though! it's just not for me.

[Sarah Cupitt · Rating 3 out of 5]

notes:
- For decades, computer scientists tried the opposite approach, they wrote endless rules to teach machines what a face looks like – but the results were disappointing. Then a small group of researchers had a radical idea: What if computers could learn like babies do? Instead of programming intelligence, what if we could grow it from data?
- What took nature millions and millions of years to evolve, artificial intelligence achieved in a few decades.
- Brains don’t follow programmed rules – instead, billions of simple neurons connect and reconnect, learning from experience.
- Think about riding a bicycle. You can’t program the rules for balance, yet somehow your brain figures it out through practice. You fall, you adjust, you fall again, you adjust again. Eventually, your neural networks encode the patterns of successful balance without anyone writing a single rule. The AI rebels called this approach, connectionism, and the AI establishment hated it.
- Birds navigate using vision, babies learn language from hearing sounds, and animals recognize threats instantly. No programmer taught them these skills through the rules of logic.
- early AI rebels were convinced that the secret lay not in better programming but in better learning.
- AI rebels scientists like Sejnowski and Hinton discovered that biological learning operates like a vast democracy, where simple neurons vote on what they’re experiencing. No single neuron holds all the answers, but through their interconnection they create intelligence.
- They began building artificial, programmed versions of these biological networks. They created mathematical neurons that could strengthen or weaken their connections based on experience, just like real brain cells. When they fed these networks thousands of examples, something remarkable happened: the artificial neurons organized themselves to recognize patterns without being explicitly programmed.
- Your brain doesn’t follow a checklist of features. It processes the whole picture at once, combining height, walk, and posture into instant recognition. The AI rebels built networks that worked the same way, processing information in layers that gradually built up understanding.
- Randomness helps brains escape bad solutions and find better ones, like shaking a jar of marbles until they settle into the most efficient arrangement.
- Feed a biological brain enough examples, and it learns to see, hear, and understand. Feed an artificial network enough examples, and it could do the same.
- For decades, the AI rebels had the right idea about how to advance machine learning but lacked the raw power to prove it. Their neural networks were like Formula One race cars stuck with bicycle engines.
- when researchers fed millions of labeled images into deep neural networks, magic happened. The networks learned to recognize cats, dogs, cars, and faces with superhuman accuracy. They didn’t just memorize the training images, they extracted the essence of what makes a cat a cat. Show them a cat they’d never seen before, in any pose or lighting, and they’d recognize it instantly
- Gaming provided the most dramatic proof. In 2016, a deep learning system called AlphaGo defeated the world champion at Go, an ancient strategy game more complex than chess. Go has more possible board positions than there are atoms in the observable universe.
- Self-driving cars began navigating real roads, recognizing stop signs, pedestrians, and other vehicles in real-time. Voice assistants started understanding natural speech and responding appropriately. Financial algorithms began spotting fraud patterns that human experts missed.
- Despite all the breathtaking advances in AI, one profound element remains missing. Current systems are like brilliant students who’ve memorized every textbook but have never stepped outside a classroom. They can recognize millions of images, translate dozens of languages, and defeat world champions at complex games, yet they lack something that every toddler possesses: direct sensory experience of the world.
- Ask the same question to an AI system, and it might provide statistically probable answers without truly understanding the concept.
- studying human intelligence wasn’t about copying it, but understanding what makes learning possible in the first place
- The question isn’t whether machines will eventually match human intelligence, but what new forms of intelligence might emerge when silicon and carbon-based learning systems work together.
- Whether it enhances human potential or disrupts society depends largely on the choices we make today.
- In classrooms, students can now generate entire essays with a few keystrokes, forcing educators to rethink how they teach critical thinking and creativity. The technology that makes learning more accessible also makes cheating effortless.
- The job market faces similar disruption. AI systems already handle customer service calls, analyze legal documents, and create marketing content. While new jobs emerge in AI development and oversight, many traditional roles are disappearing faster than people can retrain. The challenge is more than just a technological one, it’s human too: How do we help millions of workers adapt to a rapidly changing economy?

[tricia · Rating 2 out of 5]

Good historical survey in some ways but too much name-dropping of people and where they got their degrees.

[天天 拿大熊 · Rating 1 out of 5]

Do not expect too much from this book.

[Tiep · Rating 2 out of 5]

Very brief info. Almost on biology

[Chris Esposo · Rating 5 out of 5]

This is an excellent personal history of the development of neural networks as a subdiscipline of AI in the 80s, to their current incarnation, as a mainline area of practical and theoretical machine learning (and AI), which in a real way, driving much of the growth-in-value of the field both academically and in real-dollar terms in business, for the past decade or so. The author is a contemporary of Hinton, and has collaborated with him several times in the early days of the field in the 80s and 90s.

The book is divided into 3 parts. In the first segment, the author goes though the history of artificial neural networks (ANN), from the era of McCullough-Pitts, during the era where ANNs were thought to useful model of biological neural networks in the brain, with the graph-like structure of the ANNs being inspired by the axon-dendrite networks of the animal brain. In this era, the work was primarily being done by biologist, who sought to use the ANNs as a simulation of neural processing of living organisms, and thus can be thought of as an era of “simulation” where work was being done to decrease the difference between ANNs and true neural processing. However, the work was necessarily constricted by the lack of computing processing on the hardware side, and clunky language interface on the software side. Yet, this era saw the birth of many concepts and techniques that would end up exploding into practical relevance in the late aughts and 2010s, including computer vision.

The second part of the book is the real gem. Here, the author delves fairly deep (for a layman book) on the nature of instrumenting and solving an ANN. Specifically, he dedicates significant subsections to both the solution of simulated annealing of Boltzman machines, a kind of recurrent neural network, and the backpropagation technique of Hinton. In the former, the technique of learning weights was inspired by the real physio-chemical process of annealing, such as done by craftsman sword smiths, whereas the backpropogation technique of Hinton was more of an abstract solution, which though doesn’t really mimic the biological mechanics of a living neural network, has provided an effective solution to the weight-learning problem with respect to the input-data. This section also provides a great introduction the notion of CNNs and reinforcement learning, as well as a history of the NIPS conference.

The third part of the book is the weakest, as it goes a bit broad (and vague), covering the notions of how algorithms impacts society, and will continue to do so in the future, as well as the business/economics of applying machine learning, mostly from the consumer-side service areas. Honestly, this last part is the most generic of the sections, and the book could have benefited from it being shortened/removed, with a dramatic expansion of sections 2 and a bit of section 1.

That being said, the book overall is still a worthy read, especially to those learning the craft, as it provides a historical connective that helps inform/motivate why it is the tools one is utilizing are the way they are, and can possibly motivate creative thinking on original approaches based on those previous motivations. For the non-technical reader, this book is a good brief history of the field, and can help dispel some of the “magical thinking” that has permeated in the general public on all things AI and machine learning as of late, as well as possibly create a better informed retail investor class on the technology. Enjoyable read and recommended.

[BCS · Rating 4 out of 5]

Terry Sejnowski holds the Francis Crick Chair at the Salk Institute for Biological Studies in La Jolla, California. He has been at the forefront of neuroscience, the understanding of neural networks and their application in the development of artificial intelligence for the past 30 years. His book is both an enlightening history of this subject and a semi-autobiographical journey of his role and the roles of the other leading participants in this exciting and ground breaking journey in science and technology.

The book is written from a primarily academic perspective and much of the content reflects the research carried out since the 1970’s up until the present in leading neuroscience and artificial intelligence research centres in the United States. ‘The Deep Learning Revolution’ comprises three sections, each dealing with the key events and dialogues that have taken place up until the present day. A useful summary timeline is included at the start of each of the three sections.

Section one summarises recent developments in artificial intelligence, including driverless cars, language translation and algorithmic stock trading to name a few. This is followed by the bulk of this section discussing how artificial intelligence developed by moving away from a functional, rules and logic-based approach to the development of a deep learning model, through the use of neural networks supported by big data.

Section two expands on the further development of neural networks during the 1980’s and 1990’s and draws parallels with the functioning of neurons and their synaptic connections within the human brain. This section goes on to develop the concept of machine learning and, in particular, the ability of neural networks to not just carry out tasks but to learn and acquire knowledge as they do (deep learning), becoming both more efficient and increasingly more accurate in what they are asked to do. The question is also posed – how do we humans verify the information the neural network is presenting us with, if it is teaching itself?

The third and final section of the book looks at developments in machine learning since the turn of the century and what the impact of these may be in the future. The role of big data is highlighted; focusing on the ability of artificial intelligence to consume this and learn from it in a way that the human brain is unable to. The increasing interest and levels of investment by the big technology companies – including Apple, Amazon, Google and Facebook is discussed, with the associated implications this has for the choices offered to their customers by algorithmically generated content, services and product suggestions.

In summary, this book is good value for money and provides a well-informed history of deep learning and how got to where we are. It poses some thought-provoking points as to what happens next, the challenge for us will be working out where we fit in.

Review by Andrew Lamyman MBCS, IT Consultant

[Daniele Kretli · Rating 5 out of 5]

O livro explora os paralelos entre a inteligência artificial e o funcionamento do cérebro humano.

O autor aponta que o estudo do cérebro humano pode fornecer novas ideias e insights para a melhoria dos algoritmos de inteligência artificial, como a criação de redes neurais que se adaptam e aprendem continuamente, assim como os neurônios do cérebro humano se comunicam e aprendem.

Ao fazer esses paralelos, percebemos a importância da interdisciplinaridade na pesquisa de inteligência artificial, envolvendo tanto cientistas da computação quanto neurocientistas e biólogos, para que seja possível desenvolver sistemas de inteligência artificial cada vez mais avançados e sofisticados.

5 insights que me chamaram a atenção:

- A comercialização da tecnologia desenvolvida pela pesquisa científica básica costuma levar cerca de 50 anos. O que estamos vivenciando hoje, é resultado de pesquisas iniciadas nos anos 1950-1970.

- Ainda é um desafio entender como o cérebro recupera rapidamente informações armazenadas em partes amplamente distribuídas do córtex para resolver problemas complexos.

- A medida que aprendemos mais com a construção de robôs, fica evidente que o corpo é uma parte da mente.

- A importância de integrar diferentes níveis de investigação do cérebro para obter uma compreensão mais completa do funcionamento do cérebro.

- Com todos os dados gerados diariamente, as redes sociais poderiam criar um teoria de nossas mentes e usá-la para prever nossas preferências e inclinações políticas, e podem um dia nos conhecer melhor do que nós mesmos.

[Dan · Rating 3 out of 5]

I wanted to continue reading about deep learning research since much of our society uses this technology today. We use it now in our smart phones, retinal scans, self driving cars, face scanning at airports, etc. and its use will only increase. This technology will expand into the financial, medical and judicial areas in the next 10-20 years. How much they expand will be determined by how free people want to be with their own personal information. However, by combining deep learning in medicine, finance and law, I believe large improvements in the quality of each will be a plus for mankind. It also has the opportunity of making our society safer using facial and object recognition at airports, streets, etc. Even as an engineer this is a book that is a little over my head but kept my interest for the most part. The last 35 pages though are a real bore. The author is definitely on the cutting edge of deep learning research and doesn't mind letting everyone know how smart those who are in the van of this type of research are and that he was there with them, or knows them or worked with them or supervised them. Still a helpful and interesting book.

[Hasta Fu · Rating 4 out of 5]

The AI will not make people jobless, only the hardworking manual job will be replaced by another kind of new job, which is training and repairing the AI. This will bring an increase to productivity for the whole. As bad as discovery of atom bomb which can wipe out an entire city, and DNA modification which can make a lot of badass criminal. As you may think, mankind had not yet get anywhere closer to extinction.

Deep Learning has existed since 20th century, but it only gains popularity in the 21st when computing power has gone up million of times compared to past times. We still have a long way to go. AI is way better than it was, because of the way it learn. And what made human different than other creatures is the way human learn.

The scientists held a lot of conference here and there, but strangely some of them died in a young age, involving some accident. Do they work in a very stressful environment? What is Turing machine? Boltzamann machine?

This book has sparked more questions in my mind rather than answering the questions. Should we tweak our brain or just live as is? Maybe one day we will know.

[Ozan · Rating 4 out of 5]

This was an interesting read about the history of deep learning, from one of the significant influencers of the field. If you know nothing about deep learning and looking for an introductory book, this is not a good one to start with.

Even though it doesn't contain much technical detail, it has good references to many foundational works in neural networks/deep learning. Sometimes it feels like the narrative is too focused on mimicking human brain, whereas I believe the modern interpretation of deep learning should be more along the line of Yann Lecun's view ("differentiable programming") even though the field was strongly inspired by neural networks in the past.

One thing I always look for in any kind of deep learning resource is whether they include anything at all about what deep learning cannot solve. This book has a page or two about the lack of explainability and fairness, however in reality that's far from being complete or comprehensive, as there are quite a few AI problems that deep learning is completely incapable of solving.

[Jessada Karnjana · Rating 5 out of 5]

อ่านแล้วเห็นความสัมพันธ์ระหว่าง NN, ML, DL กับ neuroscience, cognitive science, complexity science, biology, psychology, linguistics, math และ physics ผ่านบริบทพัฒนาการของ DL ตั้งแต่จุดกำเนิด AI (ก่อนกำเนิด ML) ในปี 1956 (Darthmouth AI summer research project) แบบกระจ่างชัดมากขึ้นมาก ๆ Sejnowski เล่าความเป็นมาและพูดถึงซูเปอร์สตาร์ในวงการหลายคนได้สนุก มีชีวิตชีวา … Terry Sejnowski เป็นที่รู้จักกันดี (และทุกครั้งที่มีโอกาสสอนปัญหา cocktail party หรือ blind source separation หรือ linear mixture กับน้อง ๆ นักเรียน ก็จะพูดถึงผลงานหนึ่งของเขา) ในฐานะผู้พัฒนาอัลกอริทึม ICA ที่หา independent signals โดย maximizing ปริมาณ entropy (หรือ infomax-based ICA) และถ้าพูดถึง Boltzmann machine ก็ต้องมีชื่อของเขา ... เราอาจสรุปภาพรวมเชิงประวัติศาสตร์ของพัฒนาการ AI และการปฏิวัติของ DL ตั้งแต่จุดเริ่มต้น ด้วยประโยคเดียว Evolution is cleverer than you are. (Orgel's Second Rule)

[Brad Garton · Rating 5 out of 5]

I really enjoyed this book! It's an overview of the history of Deep Learning and how it relates to cognitive science, and AI from one of the early practitioners. It covers progress in Neural Network based Deep learning and future research areas. It does a lot of name dropping, of people who were significant in past research and important papers written in the field that have had significant impact on current and past research. This is not a HOW TO book for deep learning, which I gather from some of the comments, what many were looking for. It is instead a survey of the field. Particularly important for people who are getting an introduction after working in other technical areas. It is fairly deep for an introduction, but the material I think demands it.

[Kaustubh Sule · Rating 5 out of 5]

This is book is for those who have started exploring applications of ML and AI in their own field and want a Top down perspective of what exactly is deep learning and how does it help in real world applications.

No Mathematics involved although a broad understanding of probability is required in some sections . This book explains origins of deep learning from author’s perspective and how it is fundamentally different from logical programming from which computer science evolved in 1950 to 2000.

This book clarified a lot of my questions regarding differences in statistics, ML and deep learning. All in all a must read book from a pioneer in field of deep learning explaining the broad nature of the field for a layman in the field.

I have started reading it again😀

[Ali Khaledi]

It is a very nice book, a mix of Deep learning and personal history. Terry the author is among the most well-known neuroscientists as well as pioneers of Deep Learning. If you like to know a little bit about Deep learning and how it is about to make it to different areas, this book might be a nice read.

Toward the end of the book, the author take on a serious and combative tone and start attacking people who did not believe in Deep learning. At some point, he writes "History is filled with demagogues, who are eventually abandoned when the poverty of their imaginations is exposed."
This is harsh because until just a few years ago, deep learning was not mainstream science.

Either way, the book is worth reading and it is a good one.