ChatGPT and the Future of AI: The Deep Language Revolution

by Terrence J. Sejnowski

An insightful exploration of Chat GPT and other advanced AI systems—how we got here, where we’re headed, and what it all means for how we interact with the world.

In ChatGPT and the Future of AI, the sequel to The Deep Learning Revolution, Terrence Sejnowski offers a nuanced exploration of large language models (LLMs) like ChatGPT and what their future holds. How should we go about understanding LLMs? Do these language models truly understand what they are saying? Or is it possible that what appears to be intelligence in LLMs may be a mirror that merely reflects the intelligence of the interviewer? In this book, Sejnowski, a pioneer in computational approaches to understanding brain function, answers all our urgent questions about this astonishing new technology.

Sejnowski begins by describing the debates surrounding LLMs’ comprehension of language and exploring the notions of “thinking” and “intelligence.” He then takes a deep dive into the historical evolution of language models, focusing on the role of transformers, the correlation between computing power and model size, and the intricate mathematics shaping LLMs. Sejnowski also provides insight into the historical roots of LLMs and discusses the potential future of AI, focusing on next-generation LLMs inspired by nature and the importance of developing energy-efficient technologies.

Grounded in Sejnowski’s dual expertise in AI and neuroscience, ChatGPT and the Future of AI is the definitive guide to understanding the intersection of AI and human intelligence.

Genres: Artificial Intelligence, Nonfiction, Technology

271 pages, Kindle Edition

Published October 29, 2024

About the author

Terrence Joseph Sejnowski is an Investigator at the Howard Hughes Medical Institute and is the Francis Crick Professor at The Salk Institute for Biological Studies where he directs the Computational Neurobiology Laboratory. His research in neural networks and computational neuroscience has been pioneering.
Sejnowski is also Professor of Biological Sciences and Adjunct Professor in the Departments of Neurosciences, Psychology, Cognitive Science, and Computer Science and Engineering at the University of California, San Diego, where he is Director of the Institute for Neural Computation. In 2004 he was named the Francis Crick Professor and the Director of the Crick-Jacobs Center for Theoretical and Computational Biology at the Salk Institute.

Reviews

[Richard · Rating 2 out of 5]

The preface says it all: "With the help of LLMs, this book took about half the time" to write my previous (highly-acclaimed) book. Another way to say that is: to turn this into a good book would have taken double the time.

A collection of the author's thoughts about LLMs and how they work, padded with unedited ChatGPT transcripts that look like filler, this book should have been a blog post.

If you know absolutely nothing, or you're from a future civilization that needs a quick easy-to-read summary of the early 2024 state of Deep Learning AI, go ahead and skim this. For everyone else, skip.

[Carlo Martinello · Rating 4 out of 5]

the first part of the book is amusing, then it becomes more complex since it introduces many concepts without really explaining them, with the result that the "big themes" are lost.
However I strongly suggest the book, as it is a serious intro to the AI subject, given the real and serious background of the author in neural networks and computational neurosciences.

[Adam · Rating 3 out of 5]

Very basic but should be a nice intro for people who are new to the topic. The parts that are generated by chatgpt are pretty lame and needless padding. MIT press definitely lowered their quality bar on this book.

[Daria Magdalena]

Informational.

[Robert · Rating 2 out of 5]

Did not finish because it was so poorly written. The ChatGPT written summaries at the end of each short chapter were unnecessary and downright irritating.

[Sascha Griffiths · Rating 4 out of 5]

Terrence Sejnowski has written a really intriguing book which I drew a lot from. He is good at selling LLMs as mathematically interesting objects and as actually a route to understanding the human mind. He doesn't delve very deep into this but it's been enough for me to look into the actual maths that makes these models work. The book also dispells a few bogus ideas around prompting. Sejnowski was around during the second wave of neural network research booms. That was the time when backpropagation was introduced and the multilayer perceptron became the main model. It's super interesting to read what a man who unlike most has seen the rise of neural networks thinks of the technology that is now becoming ubiquitous.

[Peter Aronson · Rating 3 out of 5]

Three-and-a-half stars. Fairly good, mostly non-technical coverage of the subject, and as recent and up to date as the publishing industry can deliver. It suffers perhaps a bit from somewhat uncritical enthusiasm ("Skepticism is the chastity of the intellect, and it is shameful to surrender it too soon or to the first comer." - George Santayana, or "Doubt is the chastity of the mind." - Roger Zelazny). It also suffers from using terms without defining them (although it notes that some of them, like "intelligence" lack a clear definition). But the book constantly talks about Large Language Models making people "smarter" without defining what is meant by that. (Which is probably not the same thing in all cases.)

One gimmick of this book is it contains lots of section and chapter summaries written by ChatGPT. This is kind of clever as they act both as summaries and as examples. Unfortunately, they come across as bland and poorly organized compared to Sejnowski's writing, which is actually rather sparkling.

The book is divided into three sections.

Living with Large Language Models, which is about how LLMs are used and what people think about them. This gets a bit defensive at times.

Transformers, which describes how LLMs work in a non-technical manner. It's helpful, but I don't think it is entirely successful.

Back to the Future, about where LLMs are going and how they and related technologies are being used in the sciences and where they might be used in the future.

The book is a pretty easy read, and is broken up into nice little business book-sized chunks. The author is distinguished neuroscientist, which allows him to quietly slip in a lot of details about actual neurons and how they build into networks in the brain, which is nice. There are a lot of good references in the end notes.

[Laura · Rating 1 out of 5]

Per the author's admission, this book was partially written by ChatGPT--and it shows. I don't know who the intended target audience is, but in my opinion this is a book for people who know close to nothing about Large Language Models. The writing is bland and stripped of any insight. There is almost no critical thinking, only an extreme enthusiasm over how good LLMs are. There is a lot of shrugging about what intelligence and consciousness even are (there are plenty of books written by specialists that are very good at investigating these questions from complex angles, this book is not one of them). The author offers way too many examples of ChatGPT prompts and responses, and they read like this book: bland, general, non-inspiring. Even some of the examples offered which are meant to illustrate how good the model is are disappointing. There's an example of an article summary. The author called it really good; if this were an summary from a real student I (a PhD in biology) would call it reason to FAIL THE STUDENT because they clearly don't understand basic concepts. So this is the current state of language models? I think us writers are safe for now. I made it to half of the book before I called it quits.

[William · Rating 4 out of 5]

This is a great overview of the broader context that led to the recent advances in AI. Terrence does a great job of drawing you beyond the headlines and exposing you to some of the complexity and reality behind the advances that are making headlines. Striking a generally optimistic tone that doesn’t dwell deeply on the critiques of AI, his writing isn’t ignorant of them either. His optimism is grounded in the incredible advances that have already been made using some of the underlying technology before it burst into popular notice.

Overall I found it a good overview. The AI generated text was somewhat helpful but bordered on gimmicky. I was hoping to go a bit deeper in the science section in the middle but he stayed pretty high level which is probably appropriate for a more general audience. His future section was great precisely because he avoided hype and focused on realistic extension of work already being done. His deep knowledge in this field is clearly evident throughout the text and was the best part of the book for me.

[WiseB · Rating 4 out of 5]

As I had already been exposed and not new to generative AI, I just surfed past the first part of the book which covers basics of large language models (LLMs). The second part is primarily on transformers, which I like because the author took different perspectives and insights about transformers ... from mathematics, infrastructure, intelligence to regulation. The third part is what I enjoyed most in the reading ... especially the section "Learning from Nature", with comparison of the evolution in human's brain and language to architectures of LLMs and how they are being trained.

[Xiaoning · Rating 3 out of 5]

A non-technical overview of AI, mostly about LLM. Some concepts are not very clear due to a lack of more precise mathematical descriptions. Adding a chapter on machine learning would serve better as a more complete introduction to the subject

[Maria]

From Friday Learning emails, Barbara Oakley