What do you think?
Artificial General Intelligence
How to make AI capable of general intelligence, and what such technology would mean for society.
Artificial intelligence surrounds us. More and more of the systems and services you interact with every day are based on AI technology. Although some very recent AI systems are generalists to a degree, most AI is narrowly specific; that is, it can only do a single thing, in a single context. For example, your spellchecker can’t do mathematics, and the world's best chess-playing program can’t play Tetris. Human intelligence is different. We can solve a variety of tasks, including those we have not seen before. In Artificial General Intelligence, Julian Togelius explores technical approaches to developing more general artificial intelligence and asks what general AI would mean for human civilization.
Togelius starts by giving examples of narrow AI that have superhuman performance in some way. Interestingly, there have been AI systems that are superhuman in some sense for more than half a century. He then discusses what it would mean to have general intelligence, by looking at definitions from psychology, ethology, and computer science. Next, he explores the two main families of technical approaches to developing more general artificial foundation models through self-supervised learning, and open-ended learning in virtual environments. The final chapters of the book investigate potential artificial general intelligence beyond the strictly technical aspects. The questions discussed here investigate whether such general AI would be conscious, whether it would pose a risk to humanity, and how it might alter society.
Artificial intelligence surrounds us. More and more of the systems and services you interact with every day are based on AI technology. Although some very recent AI systems are generalists to a degree, most AI is narrowly specific; that is, it can only do a single thing, in a single context. For example, your spellchecker can’t do mathematics, and the world's best chess-playing program can’t play Tetris. Human intelligence is different. We can solve a variety of tasks, including those we have not seen before. In Artificial General Intelligence, Julian Togelius explores technical approaches to developing more general artificial intelligence and asks what general AI would mean for human civilization.
Togelius starts by giving examples of narrow AI that have superhuman performance in some way. Interestingly, there have been AI systems that are superhuman in some sense for more than half a century. He then discusses what it would mean to have general intelligence, by looking at definitions from psychology, ethology, and computer science. Next, he explores the two main families of technical approaches to developing more general artificial foundation models through self-supervised learning, and open-ended learning in virtual environments. The final chapters of the book investigate potential artificial general intelligence beyond the strictly technical aspects. The questions discussed here investigate whether such general AI would be conscious, whether it would pose a risk to humanity, and how it might alter society.
236 pages, Kindle Edition
Published September 24, 2024
About the author
Julian Togelius
11 books23 followersJulian Togelius is a Professor in the Department of Computer Science and Engineering at New York University, the director of the NYU Game Innovation Lab, and co-founder of the game AI startup modl.ai.
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 20 of 20 reviews
February 17, 2025
Probably this is the best primer out there right now on AGI.
March 15, 2025
It surely seemed to me to be a very solid primer on the subject, of which I was in a dire need, so that I feel that now I have at least a modicum of basis to think about this, and some foundation to read further. Some of the essay was pretty illuminating, but on the essential, I'm still mystified, probably even more so. Questions of intelligence... But then, “Trying to find the “core” or “essence” of intelligence is probably a fool’s errand, much like finding the essence of funkiness, beauty, or the 1980s.” Ok...
« I would like to reflect on what it means that the terms intelligence, artificial intelligence, and artificial general intelligence are so stubbornly hard to define. Does this mean that we just haven’t understood what intelligence or artificial intelligence is yet? But that assumes we are studying a natural kind when we study intelligence, that is, a grouping that reflects something real in nature. I don’t think that is the case. I think intelligence is a word we made up to represent a somewhat arbitrary set of capabilities that humans tend to possess. Trying to find the “core” or “essence” of intelligence is probably a fool’s errand, much like finding the essence of funkiness, beauty, or the 1980s. There is no secret sauce, fundamental principle, or “one simple trick” to intelligence. Things don’t get any more definite if we place the word artificial before intelligence. Artificial intelligence was just the name of a seminar in 1956 that somehow also became the name of a sprawling research field and the various technologies that emerged from it. That the various technologies discussed in this book are all referred to as AI is mostly a historical accident and/or marketing. We could have used a different term or several different terms. In fact, some of the early neural network research was done under the moniker “cybernetics.” »
It's complicated.
The book is extraordinarily relevant and current, even if written more than one year ago, in a field that's exploding with revolutionary advances every day. Let me prove this with random instances from this week.
Yesterday, on X, Andy Boreham expounded on having asked Grok and DeepSeek who he is, and DeepSeek hallucinated (“I have to admit, I really am NOT an expert on these AI models and how they work. Because of that, I have no idea how DeepSeek got it so wrong. I wouldn't be surprised if the AI said "I don't know who that is" or something, but it literally made things up. Any ideas? (...) In case you don't know about my past, DeepSeek didn't just get my marital status / private life completely wrong, it chose Stuff, a real New Zealand media outlet, and said I work there. I have NEVER worked for Stuff.”). This book explains perfectly the technical fundamentals of this occurrences in LLM AI systems (and what these are; if you are now at a loss, read it).
This past week, the guy from OpenAI made a plea for some sort of sanctions on DeepSeek, a technology that's much more open than the evermore closed approach to AI development that “OpenAI” is pursuing. Considerations from the book:
“Our computer systems are relatively safe today because so many software developers and system administrators take security so seriously and share their knowledge freely and openly. Cybersecurity is studied in both academia and industry, and papers and source code are shared openly. You might think that this would give an advantage to attackers, but the opposite is actually true. The best way to test your defense strategy is to let other hackers or researchers try to attack it and share what they learned. And the best way to rapidly enhance security methods is to let hackers and researchers freely build on each other’s solutions. Besides, trying to stop bad actors from sharing attack strategies with each other would clearly be futile, so it makes sense for good actors to employ the same strategy.
“I think this is the future we want for AI methods and models as well. As much of AI research and development as possible should be conducted openly and transparently. This means not only that the model parameters and the code used for training should be open-source but also that researchers and developers openly publish their methods and findings. This approach will allow new models and methods to be tested by anyone in the world with the technical capacity, leading not only to more innovation but also to more safety. A society where as many people as possible have access to, understand, and can contribute to modern and can contribute to modern AI will be safer from whatever risks AI systems might bring.”
One other aspect of the essay that I did love was the author's consideration for the relevance of science-fiction in the development of the thought and even lines of progress of this technology.
«
All these theories about intelligence and artificial intelligence can feel quite abstract. They don’t necessarily help us imagine what AGI would be like. In chapter 5, we explore some visions of what AGI could be like, with ample reference to science fiction. Science fiction stories have inspired generations of AI researchers and can help us not only think about but also differentiate between potential AI futures.
To understand the many different things people mean when they talk about AGI, let us try to draw out the ways in which different concepts of AGI differ. These can be seen as dimensions along which concepts of AGI can vary. We could use these dimensions to organize and compare different visions of AGI. Because no actual AGI exists, we cannot use examples from the real world to illustrate these ideas, so we will use the next best thing: examples from science fiction.
Disembodied mind: Iain M. Banks’s Culture novels are set in a utopian civilization where humanlike beings coexist with Minds, enormously intelligent machines. Minds don’t have bodies of their own but are largely responsible for keeping things running in society and thus control a large variety of mobile robots. Banks envisions Minds as having many humanlike traits, including empathy and a sense of humor; they have a great deal of intentionality. Basically, they are much like humans, only with a thousand or a million times greater memory, attention span, precision, and processing speed. A Mind has an enormous knowledge bank but must still acquire knowledge as we do, through communication and observation. A Mind is not omniscient. The novels in the Culture universe feature many examples of Minds not knowing what to do or how to do it, because they don’t have the requisite knowledge.
The closest we can get is probably the many first-contact stories written by various science fiction authors. As mentioned earlier, the experience of first contact with a truly alien intelligence is a central theme of Stanisław Lem’s work. It is often not clear in Lem’s stories whether we experience a biological or machine-based intelligence, but the being’s thinking is in some sense orthogonal to ours. China Miéville’s stories also feature examples of very alien and fundamentally unfathomable intelligences, for example, in Perdido Street Station or Embassytown. One might argue that to the extent an AI system is built by humans, it will not be truly alien, but as we will see in chapter 8, an open-ended learning system might learn from a self-created world that is substantially different from the one we inhabit and therefore learn skills that are very different from ours.
»
Fantastic.
Last thoughts.
“I have discussed AI as a series of technical inventions motivated by being able to solve problems that require intelligence (...) The alternative perspective is that the history of AI is a long deconstruction of the concept of intelligence. This proceeds by someone confidently exclaiming that something—say, planning, image creation, or translation—is a hallmark of real intelligence. The research community then finds a way to perform this feat using some new or old AI method. We then look at the original task and say that it didn’t really require intelligence after all, because it can be done with mere computation. Therefore we need to find another feat that really requires intelligence. And then we do this again and again, chipping away at the concept of intelligence. The job will be done when we can no longer find anything that we can claim requires intelligence and that we cannot get a computer to do as well as we do it. The concept of intelligence will then become pointless, except in the colloquial use of the term. At that point, we may or may not want to say that we have achieved AGI.”
And: “there is not really anything magical about AI. It’s just a set of useful technologies, some of which might change the world—in the way that clocks, stirrups, steam engines, and telephones all once did. This realization is a little painful for those of us who chose to become AI researchers because of that magic, those of us who wanted—and in some sense still want—to understand the mind by creating minds with computers. There are indeed plenty of interesting technologies to develop and phenomena to understand. There’s just not any great mystery to solve.” N. B.: In this I do not believe.
Anyway, great little book.
« I would like to reflect on what it means that the terms intelligence, artificial intelligence, and artificial general intelligence are so stubbornly hard to define. Does this mean that we just haven’t understood what intelligence or artificial intelligence is yet? But that assumes we are studying a natural kind when we study intelligence, that is, a grouping that reflects something real in nature. I don’t think that is the case. I think intelligence is a word we made up to represent a somewhat arbitrary set of capabilities that humans tend to possess. Trying to find the “core” or “essence” of intelligence is probably a fool’s errand, much like finding the essence of funkiness, beauty, or the 1980s. There is no secret sauce, fundamental principle, or “one simple trick” to intelligence. Things don’t get any more definite if we place the word artificial before intelligence. Artificial intelligence was just the name of a seminar in 1956 that somehow also became the name of a sprawling research field and the various technologies that emerged from it. That the various technologies discussed in this book are all referred to as AI is mostly a historical accident and/or marketing. We could have used a different term or several different terms. In fact, some of the early neural network research was done under the moniker “cybernetics.” »
It's complicated.
The book is extraordinarily relevant and current, even if written more than one year ago, in a field that's exploding with revolutionary advances every day. Let me prove this with random instances from this week.
Yesterday, on X, Andy Boreham expounded on having asked Grok and DeepSeek who he is, and DeepSeek hallucinated (“I have to admit, I really am NOT an expert on these AI models and how they work. Because of that, I have no idea how DeepSeek got it so wrong. I wouldn't be surprised if the AI said "I don't know who that is" or something, but it literally made things up. Any ideas? (...) In case you don't know about my past, DeepSeek didn't just get my marital status / private life completely wrong, it chose Stuff, a real New Zealand media outlet, and said I work there. I have NEVER worked for Stuff.”). This book explains perfectly the technical fundamentals of this occurrences in LLM AI systems (and what these are; if you are now at a loss, read it).
This past week, the guy from OpenAI made a plea for some sort of sanctions on DeepSeek, a technology that's much more open than the evermore closed approach to AI development that “OpenAI” is pursuing. Considerations from the book:
“Our computer systems are relatively safe today because so many software developers and system administrators take security so seriously and share their knowledge freely and openly. Cybersecurity is studied in both academia and industry, and papers and source code are shared openly. You might think that this would give an advantage to attackers, but the opposite is actually true. The best way to test your defense strategy is to let other hackers or researchers try to attack it and share what they learned. And the best way to rapidly enhance security methods is to let hackers and researchers freely build on each other’s solutions. Besides, trying to stop bad actors from sharing attack strategies with each other would clearly be futile, so it makes sense for good actors to employ the same strategy.
“I think this is the future we want for AI methods and models as well. As much of AI research and development as possible should be conducted openly and transparently. This means not only that the model parameters and the code used for training should be open-source but also that researchers and developers openly publish their methods and findings. This approach will allow new models and methods to be tested by anyone in the world with the technical capacity, leading not only to more innovation but also to more safety. A society where as many people as possible have access to, understand, and can contribute to modern and can contribute to modern AI will be safer from whatever risks AI systems might bring.”
One other aspect of the essay that I did love was the author's consideration for the relevance of science-fiction in the development of the thought and even lines of progress of this technology.
«
All these theories about intelligence and artificial intelligence can feel quite abstract. They don’t necessarily help us imagine what AGI would be like. In chapter 5, we explore some visions of what AGI could be like, with ample reference to science fiction. Science fiction stories have inspired generations of AI researchers and can help us not only think about but also differentiate between potential AI futures.
To understand the many different things people mean when they talk about AGI, let us try to draw out the ways in which different concepts of AGI differ. These can be seen as dimensions along which concepts of AGI can vary. We could use these dimensions to organize and compare different visions of AGI. Because no actual AGI exists, we cannot use examples from the real world to illustrate these ideas, so we will use the next best thing: examples from science fiction.
Disembodied mind: Iain M. Banks’s Culture novels are set in a utopian civilization where humanlike beings coexist with Minds, enormously intelligent machines. Minds don’t have bodies of their own but are largely responsible for keeping things running in society and thus control a large variety of mobile robots. Banks envisions Minds as having many humanlike traits, including empathy and a sense of humor; they have a great deal of intentionality. Basically, they are much like humans, only with a thousand or a million times greater memory, attention span, precision, and processing speed. A Mind has an enormous knowledge bank but must still acquire knowledge as we do, through communication and observation. A Mind is not omniscient. The novels in the Culture universe feature many examples of Minds not knowing what to do or how to do it, because they don’t have the requisite knowledge.
The closest we can get is probably the many first-contact stories written by various science fiction authors. As mentioned earlier, the experience of first contact with a truly alien intelligence is a central theme of Stanisław Lem’s work. It is often not clear in Lem’s stories whether we experience a biological or machine-based intelligence, but the being’s thinking is in some sense orthogonal to ours. China Miéville’s stories also feature examples of very alien and fundamentally unfathomable intelligences, for example, in Perdido Street Station or Embassytown. One might argue that to the extent an AI system is built by humans, it will not be truly alien, but as we will see in chapter 8, an open-ended learning system might learn from a self-created world that is substantially different from the one we inhabit and therefore learn skills that are very different from ours.
»
Fantastic.
Last thoughts.
“I have discussed AI as a series of technical inventions motivated by being able to solve problems that require intelligence (...) The alternative perspective is that the history of AI is a long deconstruction of the concept of intelligence. This proceeds by someone confidently exclaiming that something—say, planning, image creation, or translation—is a hallmark of real intelligence. The research community then finds a way to perform this feat using some new or old AI method. We then look at the original task and say that it didn’t really require intelligence after all, because it can be done with mere computation. Therefore we need to find another feat that really requires intelligence. And then we do this again and again, chipping away at the concept of intelligence. The job will be done when we can no longer find anything that we can claim requires intelligence and that we cannot get a computer to do as well as we do it. The concept of intelligence will then become pointless, except in the colloquial use of the term. At that point, we may or may not want to say that we have achieved AGI.”
And: “there is not really anything magical about AI. It’s just a set of useful technologies, some of which might change the world—in the way that clocks, stirrups, steam engines, and telephones all once did. This realization is a little painful for those of us who chose to become AI researchers because of that magic, those of us who wanted—and in some sense still want—to understand the mind by creating minds with computers. There are indeed plenty of interesting technologies to develop and phenomena to understand. There’s just not any great mystery to solve.” N. B.: In this I do not believe.
Anyway, great little book.
May 16, 2025
This is the second entry I have read in this MIT series of short books on current “essential” tech issues. This book is about “artificial general intelligence”. The book begins by going through a deconstruction of the basic terms, in particular “intelligence” and “artificial intelligence” and “general” versus more specific intelligences. The result? AGI is a neat, popular, and even “trendy” topic area that is largely aspirational and there is little related to AGI that is real or substantial. The author moves to a history of AI efforts and even discusses what AGI might be. Given its lack of reality, Professor Togelius deftly moves through the potential of AGI as shown in movies. He provides some interesting chapters on the mechanics of AI models and concludes with a tour of potential directions.
This is a well intended and well written introduction that is thorough and honest, subject to the limits of a short monograph.
I think I will read more of these books.
This is a well intended and well written introduction that is thorough and honest, subject to the limits of a short monograph.
I think I will read more of these books.
June 6, 2025
Short book mostly on philosophical topics. I'd say essays collection.
Summary: both AI and AGI are hard.
Summary: both AI and AGI are hard.
July 27, 2026
I rate this book 5/5 stars; I found it very thorough, more-so than some other books about artificial intelligence (AI). I would recommend this book to anyone interested in a book with unbiased views on artificial intelligence and the idea of artificial general intelligence. It's extremely thorough.
--Notable Quotes--
- "Chapter 8 describes another, complementary approach to AGI, namely, open-ended learning in virtual environments. In this approach, AI systems are passive agents that take actions in simulated worlds instead of passively learning from human-made data. The idea here is to let AI emerge or evolve, inspired by how intelligence evolved on Earth." (p.8) One issue here is that God is an active God in today's world - not a passive one. We were designed to have a relationship with Him through Jesus Christ. Genesis 1:3 states, "Then God said, 'Let there be light'; and there was light." God spoke, and it was so. Genesis 1:26-27 states, "Then God said, 'Let Us make mankind in Our image, according to Our likeness; and let them rule over the fish of the sea and over the birds of the sky and over the livestock and over all the earth, and over every crawling thing that crawls on the earth.' So God created man in His own image, in the image of God He created him; male and female He created them. "Us" refers to the Trinity: God the Father, God the Holy Spirit, and God the Son, Jesus Christ.
- "Initially spurred by the various computational demands of World War II, such as firing artillery at moving airplanes and interpreting radar signals, the first programmable digital computers emerged in the 1940s and 1950s. While these machines were primitive by today's standards, they could do marvelous things. In particular, they could calculate. At the time, the term computer referred to a human (usually a woman) who would perform calculations entirely by hand or with simple manual tools only. Large projects, such as those involved in designing aircraft or artillery for the war effort, could employ hundreds of human computers. The new digital computers could perform calculations thousands of times faster. This was obviously a revolution in computation, but why is it not seen as the first superhuman AI?" (p.15-16). The development of the atomic bomb at Oak Ridge, TN is an excellent example of this. Top female mathematicians and scientists were recruited for that massive effort in WWII.
- "However, this human exceptionalism is at odds with everything we know about how evolution works. Evolution tends to be gradual and to build scrappily on what is already there. New anatomical or cognitive features are usually built on top of repurposed from other organs or mechanisms. We know that the cognitive abilities of our ancestors were developed gradually over many millions of years. Why would a completely new, 'general' intelligence suddenly evolve? It is much more likely that our intelligence is a collection of special-purpose capabilities that have evolved in response to specific needs, just as it is for all other animals we know of." (p.49) Actually, we do not know that macro evolution took place, and we can't recreate it. Tests done with flies have created flies that are blind, flies that are different colors, flies that have non-functioning wings, and flies that can't reproduce with other types of flies, but they remain flies. the Bible's creation story answers this question - humanity was created by our Heavenly Father. We did not evolve, but were created when the Lord spoke. Please consider my observations about p.15-16. It backs this.
- "But taking the word general in artificial intelligence seriously would seem to require a definition that somehow touches on all possible environments. On the other hand, one may wonder why humans, evolved to live in a particular ecological niche, should be able to perform well in all possible environments, as those are not part of that niche. From the standpoint of evolutionary biology, it is very hard to explain why we would have evolved to perform well in environments we did not evolve in." (p.63) " Genesis 1:28 answers this: "God blessed them; and God said to them, 'Be fruitful and multiply, and fill the earth, and subdue it; and rule over the fish of the sea and over the birds of the sky and over every living thing that moves on the earth.'” We (as humans) were created to fill the Earth. We were created to spread to environments we did not evolve in and had not encountered before.
- "There is also the lurking suspicion that LLMs, which are after all only trained to reproduce text that is already written, are limited in how much new knowledge they can produce. After interacting for a while with LLMs, you start to recognize the genericness of their responses." (p.126)
- "We humans are probably the most general intelligence on this planet (though of course, this depends on the exact definitions of general and intelligence)/. It stands to reason that we should be able to create other general intelligences by using the same algorithm that created us. That algorithm is almost certainly Darwinian evolution, first on a purely genetic level and later on also on a cultural or memetic level. As described earlier, evolutionary computation is an attempt to use Darwinian natural selection as a method for finding solutions or generating programs. And evolutionary algorithms do work well for many well-defined problems. For less well-specified problems, such as crating artificial general intelligence, it is difficult to come up with a good fitness function." (p.127) We were created in image of God, but we aren't God and never will be. We can try to copy Him, but we'll never come close. Job 38:4-5 (NASB) says, "Where were you when I laid the foundation of the earth?/Tell Me, if you have understanding,/Who set its measurements? Since you know./Or who stretched the measuring line over it?"
- "Researchers have also been exploring how to generate completely new games, although this is much harder than generating levels, quests, or items for existing games. A few successful board games have been AI generated, including the game Yavalath. Yavalath was not designed by the noted game designer and researcher, Cameron Browne; instead it was designed by Ludi, an AI system that Cameron designed. Yet Yavalath is for sale in board games stores and is moderately successful. (As far as I know, Cameron pockets the profits from the game; the program that designed the game gets none of the money)." (p. 138)
- "It is safe to say that none of the games my AI that my AI systems have created have become critical or commercial successes. In particular, my team and I have frequently found that our systems will generate video games that technically should be 'fun'-they are learnable and have skill differentiation-but they just do not make any sense to human players. But maybe the AI system had fun-if it could indeed experience anything, which we have no reason to believe." (p.139)
- "This discussion can appear a little dizzying or whimsical the first time you encounter it. It seems the English language is not really equipped to talk about the hard problem of consciousness. Some philosophers, such as Daniel Dennett, believe there is no hard problem and that the concept of a philosophical zombie is incoherent. The idea that there is a specific philosophical problem of when and how a physical body gives rise to phenomenal consciousness is shared by many thinkers across the ages, and not only academic philosophers. The Christian concept of the soul is arguably at least partially about phenomenal consciousness." (p.147-148)
- "Where does this leave us when it comes to deciding whether an AI system is conscious? Hard to tell. Simply asking the system will not do any good. Remember that LLMs are very good at role-playing. Depending on your prompt, you can get the very same LLM to say that it is conscious or that it is not conscious... Remember that the words produced by the LLM do not per se refer to anything, except in relation to other words. Engaging an LLM in this kind of dialogue can be very entertaining, but unfortunately it does not help us to understand whether it is, in fact, conscious. It is possible that our society will at some point decide that some AI systems, or some class of AI systems, are conscious and deserve some form of rights." (p.152). This quote, and really where we are today with AI, remind me very much of the TV show Person of Interest. https://www.imdb.com/title/tt1839578/...
- "In other words, once we build an AI system that is as clever as we are, that system will be able to improve itself (or construct better AI systems) as well as we an. And once it has improved itself, it will be better than we are at improving itself than we are, so it will get better even faster... Once this machine is literally thousands of times smarter than we are, there's no telling what it will do or how it will treat us... So the AI system might just metaphorically walk all over us." (p.157)
- "As of the time of this writing, several dozen influential AI researchers have signed a short statement saying, in its entirety, that 'mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks, such as pandemics and nuclear war.'... These systems have a low ceiling, meaning that their self-improvement only takes them so far. They are good at optimizing some of their own functioning to a degree, but cannot add completely new functionality. (Part of the reason for this is that there is no way of training a system to invent completely novel functionality; by definition, there is nothing to train it on.). What the intelligence explosion argues for is systems that can get drastically better very fast, so that we lose control-basically, systems that could, on their own, jump years ahead in AI development." (p.158-159) Please not, the systems "cannot add completely new functionality." They can only do what humans have programmed them to do.
- "As mentioned earlier in the book, advances in hardware... have been crucial to enabling modern AI. The latest advancements could not have been made with last-generation hardware, because training would have been too slow, or memory would have been insufficient. So for an AI system to improve itself beyond a small increment of its current capabilities, it would need to be able to make progress on multiple fronts. This is a lot to ask on the software side, and it would be complicated, to say the least, on the harder side. At least, it could not be done quickly... Accordingly, a rapid and sustained capability increase for an AI system on its own seems highly unlikely." (p.160)
- "The first concern that many people have when discussing the societal impacts of AGI is job displacement. In modern societies, people work to make money, which they spend on goods and services produced by others. If an AGI system could do everything that a human can, what is there for us to do? How will we make money? And how will we fill our lives with meaning, given that work is important to many of us for more than pecuniary reasons?" (p.168). This does not change Genesis 2:15. The NASB translation states, "Then the Lord God took the man and put him in the Garden of Eden to cultivate it and tend it." What each of us owns is what the Lord has given to us. In The Parable of the Talents in Matthew 14-29, each man is repaid for his management of the money - including the man who buried it in the ground and did nothing. He was thrown out. We cannot do nothing, but each of us has skills we may utilize if we are left without work. As for meaning, the Lord created us with the intention of having a personal relationship with each of us, and He will provide for us. In John 10:11, Jesus states,"'I am the good shepherd; the good shepherd lays down His life for the sheep..." Matthew 10:29 states, "Are two sparrows not sold for an assarion? And yet not one of them will fall to the ground apart from your Father." An "assarion" was a coin. The HCSB translation uses the word "penny" in place of "assarion."
- "No robot today can replace, or even assist, a plumber. The industrial robots that perform tasks in factories are highly scripted and the opposite of general intelligence. This phenomenon, where AI technology performs better at the kind of tasks that we would consider cognitive and much worse at supposedly non-intellectual manual tasks, is called Moravec's paradox and has been acknowledged for decades." (p.170)
- "When we could automate or partly automate a job, new jobs were created as new demands could be met. Historically, unemployment has temporarily shot up during times of great technological change but then receded as workers have transitioned to new roles. When you average over these... technological cycles, unemployment has been surprisingly stable for hundreds of years." (p.170-172)
- "We haven't even discussed all the new jobs that AGI would enable. There is a near-limitless humber of takes that are not done because we cannot afford to do them or because we are busy doing other things. I am a little annoyed that our apartment does not clean itself, that I still have to book flights and sync meetings myself, and that my wife and I still have to do the annoying parts of child care (making sure our toddler son doesn't do anything dangerous, such as running down the stairs) instead of just focusing on the fun parts..." (p.174)
- "In sum, history strongly suggests that we should not worry about long-term job losses due to more generally capable AI... This is consistent with history, where technology is increasingly doing the tasks we used to do, and we are managing the technology." (p.175)
- "How can we make sure models are less biased? Perhaps the most obvious route is to not train them on biased data. But this is hard, given that most types of models benefit from being trained on as many data as possible, and so many data are biased. Bias may also be encode in subtle ways, such as how often women are mentioned compared to men in certain contexts, meaning that discarding biased data might require complicated and labor-intensive analysis." (p.179-180)
- The impressive capabilities of modern foundation models to generate high-quality text, voice, and images come with the alarming possibility for generating high-quality misinformation. For example, it is easier than ever to generate photos of an existing person engaging in something that never happened, whether a crime, ... a battle, or even a party with the wrong kind of people. This is a problem because we are accustomed to believing photos... To the extent we rely on opinions of anonymous or pseudonymous people online, this poses a problem for democracy." (p.180-181). Yet, we can trust that what is stated in the Bible is 100% true. The original artifacts (such as scrolls in museums) existed well before AI and therefore are more difficult to falsify.
- "Seeing the power and versatility of LLMs, some people claim that they are the first steps toward AGI. I don't agree, and not only because we don't have a good definition of AGI. LLMs have plenty of shortcomings: they are generally bad at reasoning and planning, can't (on their own) really do math, and, perhaps most importantly, are extremely unreliable. Even the best LLMs hallucinate frequently in some situations. This makes them unsuitable on their own in many situations." (p.191)
- "A completely different approach to AGI is represented by open-ended learning. Here we are not training models on the cultural exhaust of humanity (text and images). Instead we build coupled systems of agents and problem generators, loosely inspired by how natural intelligence has developed through evolution in complex ecologies." (p.191). The jury is still out on this one; as computer engineers and programmers have not yet reached that point.
- "If we can build an AGI system, would it be conscious? Would there be something it would be like to be that system? Would it feel genuine pain and happiness, not just perform simulations for those feelings? We don't know... I majored in philosophy in undergrad largely because I wanted to understand whether machines could or would be conscious. After a few years of thinking about this, I concluded that I had no idea how I (or anyone else) could make progress on this question, so I switched to the easier problem of trying to create artificial intelligence. And that's where I still am." (p.192-193)
- "A question that is surprisingly, and annoyingly, widely debated as I'm writing this is whether AI risks leading to human extinction. The idea here is that once we create sufficiently advanced AI systems, they will be so intelligent that we cannot control them. They might proceed to improve themselves to be even further beyond our grasp, and then we might as well be to them as ants are to us... If you try to use well-defined words that refer to actual AI technologies and their capacities, the argument falls apart." (p.193). In Genesis, mankind was the apex of God's creation, and we cannot fathom His greatness. This book is a secular text, but it is claiming we can overcome our Creator and that simply isn't true.
- "Unfortunately, some people are convinced that further AI development poses serious risks and should be curtailed and controlled. Others are interested in bogging down AI research and governmental bureaucracy to preserve their own competitive position in AI development and therefore claim to worry about existential risk. It[s not pretty. As I'm writing this, proposals are being floated to require licenses for training large models. I think such regulations would be a big mistake, not only because of the chilling effects on AI research, but also because they resent a slippery slope in terms of free speech." (p.194)
- "This is not to say that there are no risks associated with the development and diffusion of more capable and general AI systems. Given the wide applicability of anything we may call AGI, it is likely that it will have major effects on society. While some worry that many, or even most, people may lose their jobs, I don't think that is likely." (p.195)
- "As the saying goes, AI is just computer science that doesn't really work yet; and when it does work, it's no longer AI." (p.200)
--Notable Quotes--
- "Chapter 8 describes another, complementary approach to AGI, namely, open-ended learning in virtual environments. In this approach, AI systems are passive agents that take actions in simulated worlds instead of passively learning from human-made data. The idea here is to let AI emerge or evolve, inspired by how intelligence evolved on Earth." (p.8) One issue here is that God is an active God in today's world - not a passive one. We were designed to have a relationship with Him through Jesus Christ. Genesis 1:3 states, "Then God said, 'Let there be light'; and there was light." God spoke, and it was so. Genesis 1:26-27 states, "Then God said, 'Let Us make mankind in Our image, according to Our likeness; and let them rule over the fish of the sea and over the birds of the sky and over the livestock and over all the earth, and over every crawling thing that crawls on the earth.' So God created man in His own image, in the image of God He created him; male and female He created them. "Us" refers to the Trinity: God the Father, God the Holy Spirit, and God the Son, Jesus Christ.
- "Initially spurred by the various computational demands of World War II, such as firing artillery at moving airplanes and interpreting radar signals, the first programmable digital computers emerged in the 1940s and 1950s. While these machines were primitive by today's standards, they could do marvelous things. In particular, they could calculate. At the time, the term computer referred to a human (usually a woman) who would perform calculations entirely by hand or with simple manual tools only. Large projects, such as those involved in designing aircraft or artillery for the war effort, could employ hundreds of human computers. The new digital computers could perform calculations thousands of times faster. This was obviously a revolution in computation, but why is it not seen as the first superhuman AI?" (p.15-16). The development of the atomic bomb at Oak Ridge, TN is an excellent example of this. Top female mathematicians and scientists were recruited for that massive effort in WWII.
- "However, this human exceptionalism is at odds with everything we know about how evolution works. Evolution tends to be gradual and to build scrappily on what is already there. New anatomical or cognitive features are usually built on top of repurposed from other organs or mechanisms. We know that the cognitive abilities of our ancestors were developed gradually over many millions of years. Why would a completely new, 'general' intelligence suddenly evolve? It is much more likely that our intelligence is a collection of special-purpose capabilities that have evolved in response to specific needs, just as it is for all other animals we know of." (p.49) Actually, we do not know that macro evolution took place, and we can't recreate it. Tests done with flies have created flies that are blind, flies that are different colors, flies that have non-functioning wings, and flies that can't reproduce with other types of flies, but they remain flies. the Bible's creation story answers this question - humanity was created by our Heavenly Father. We did not evolve, but were created when the Lord spoke. Please consider my observations about p.15-16. It backs this.
- "But taking the word general in artificial intelligence seriously would seem to require a definition that somehow touches on all possible environments. On the other hand, one may wonder why humans, evolved to live in a particular ecological niche, should be able to perform well in all possible environments, as those are not part of that niche. From the standpoint of evolutionary biology, it is very hard to explain why we would have evolved to perform well in environments we did not evolve in." (p.63) " Genesis 1:28 answers this: "God blessed them; and God said to them, 'Be fruitful and multiply, and fill the earth, and subdue it; and rule over the fish of the sea and over the birds of the sky and over every living thing that moves on the earth.'” We (as humans) were created to fill the Earth. We were created to spread to environments we did not evolve in and had not encountered before.
- "There is also the lurking suspicion that LLMs, which are after all only trained to reproduce text that is already written, are limited in how much new knowledge they can produce. After interacting for a while with LLMs, you start to recognize the genericness of their responses." (p.126)
- "We humans are probably the most general intelligence on this planet (though of course, this depends on the exact definitions of general and intelligence)/. It stands to reason that we should be able to create other general intelligences by using the same algorithm that created us. That algorithm is almost certainly Darwinian evolution, first on a purely genetic level and later on also on a cultural or memetic level. As described earlier, evolutionary computation is an attempt to use Darwinian natural selection as a method for finding solutions or generating programs. And evolutionary algorithms do work well for many well-defined problems. For less well-specified problems, such as crating artificial general intelligence, it is difficult to come up with a good fitness function." (p.127) We were created in image of God, but we aren't God and never will be. We can try to copy Him, but we'll never come close. Job 38:4-5 (NASB) says, "Where were you when I laid the foundation of the earth?/Tell Me, if you have understanding,/Who set its measurements? Since you know./Or who stretched the measuring line over it?"
- "Researchers have also been exploring how to generate completely new games, although this is much harder than generating levels, quests, or items for existing games. A few successful board games have been AI generated, including the game Yavalath. Yavalath was not designed by the noted game designer and researcher, Cameron Browne; instead it was designed by Ludi, an AI system that Cameron designed. Yet Yavalath is for sale in board games stores and is moderately successful. (As far as I know, Cameron pockets the profits from the game; the program that designed the game gets none of the money)." (p. 138)
- "It is safe to say that none of the games my AI that my AI systems have created have become critical or commercial successes. In particular, my team and I have frequently found that our systems will generate video games that technically should be 'fun'-they are learnable and have skill differentiation-but they just do not make any sense to human players. But maybe the AI system had fun-if it could indeed experience anything, which we have no reason to believe." (p.139)
- "This discussion can appear a little dizzying or whimsical the first time you encounter it. It seems the English language is not really equipped to talk about the hard problem of consciousness. Some philosophers, such as Daniel Dennett, believe there is no hard problem and that the concept of a philosophical zombie is incoherent. The idea that there is a specific philosophical problem of when and how a physical body gives rise to phenomenal consciousness is shared by many thinkers across the ages, and not only academic philosophers. The Christian concept of the soul is arguably at least partially about phenomenal consciousness." (p.147-148)
- "Where does this leave us when it comes to deciding whether an AI system is conscious? Hard to tell. Simply asking the system will not do any good. Remember that LLMs are very good at role-playing. Depending on your prompt, you can get the very same LLM to say that it is conscious or that it is not conscious... Remember that the words produced by the LLM do not per se refer to anything, except in relation to other words. Engaging an LLM in this kind of dialogue can be very entertaining, but unfortunately it does not help us to understand whether it is, in fact, conscious. It is possible that our society will at some point decide that some AI systems, or some class of AI systems, are conscious and deserve some form of rights." (p.152). This quote, and really where we are today with AI, remind me very much of the TV show Person of Interest. https://www.imdb.com/title/tt1839578/...
- "In other words, once we build an AI system that is as clever as we are, that system will be able to improve itself (or construct better AI systems) as well as we an. And once it has improved itself, it will be better than we are at improving itself than we are, so it will get better even faster... Once this machine is literally thousands of times smarter than we are, there's no telling what it will do or how it will treat us... So the AI system might just metaphorically walk all over us." (p.157)
- "As of the time of this writing, several dozen influential AI researchers have signed a short statement saying, in its entirety, that 'mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks, such as pandemics and nuclear war.'... These systems have a low ceiling, meaning that their self-improvement only takes them so far. They are good at optimizing some of their own functioning to a degree, but cannot add completely new functionality. (Part of the reason for this is that there is no way of training a system to invent completely novel functionality; by definition, there is nothing to train it on.). What the intelligence explosion argues for is systems that can get drastically better very fast, so that we lose control-basically, systems that could, on their own, jump years ahead in AI development." (p.158-159) Please not, the systems "cannot add completely new functionality." They can only do what humans have programmed them to do.
- "As mentioned earlier in the book, advances in hardware... have been crucial to enabling modern AI. The latest advancements could not have been made with last-generation hardware, because training would have been too slow, or memory would have been insufficient. So for an AI system to improve itself beyond a small increment of its current capabilities, it would need to be able to make progress on multiple fronts. This is a lot to ask on the software side, and it would be complicated, to say the least, on the harder side. At least, it could not be done quickly... Accordingly, a rapid and sustained capability increase for an AI system on its own seems highly unlikely." (p.160)
- "The first concern that many people have when discussing the societal impacts of AGI is job displacement. In modern societies, people work to make money, which they spend on goods and services produced by others. If an AGI system could do everything that a human can, what is there for us to do? How will we make money? And how will we fill our lives with meaning, given that work is important to many of us for more than pecuniary reasons?" (p.168). This does not change Genesis 2:15. The NASB translation states, "Then the Lord God took the man and put him in the Garden of Eden to cultivate it and tend it." What each of us owns is what the Lord has given to us. In The Parable of the Talents in Matthew 14-29, each man is repaid for his management of the money - including the man who buried it in the ground and did nothing. He was thrown out. We cannot do nothing, but each of us has skills we may utilize if we are left without work. As for meaning, the Lord created us with the intention of having a personal relationship with each of us, and He will provide for us. In John 10:11, Jesus states,"'I am the good shepherd; the good shepherd lays down His life for the sheep..." Matthew 10:29 states, "Are two sparrows not sold for an assarion? And yet not one of them will fall to the ground apart from your Father." An "assarion" was a coin. The HCSB translation uses the word "penny" in place of "assarion."
- "No robot today can replace, or even assist, a plumber. The industrial robots that perform tasks in factories are highly scripted and the opposite of general intelligence. This phenomenon, where AI technology performs better at the kind of tasks that we would consider cognitive and much worse at supposedly non-intellectual manual tasks, is called Moravec's paradox and has been acknowledged for decades." (p.170)
- "When we could automate or partly automate a job, new jobs were created as new demands could be met. Historically, unemployment has temporarily shot up during times of great technological change but then receded as workers have transitioned to new roles. When you average over these... technological cycles, unemployment has been surprisingly stable for hundreds of years." (p.170-172)
- "We haven't even discussed all the new jobs that AGI would enable. There is a near-limitless humber of takes that are not done because we cannot afford to do them or because we are busy doing other things. I am a little annoyed that our apartment does not clean itself, that I still have to book flights and sync meetings myself, and that my wife and I still have to do the annoying parts of child care (making sure our toddler son doesn't do anything dangerous, such as running down the stairs) instead of just focusing on the fun parts..." (p.174)
- "In sum, history strongly suggests that we should not worry about long-term job losses due to more generally capable AI... This is consistent with history, where technology is increasingly doing the tasks we used to do, and we are managing the technology." (p.175)
- "How can we make sure models are less biased? Perhaps the most obvious route is to not train them on biased data. But this is hard, given that most types of models benefit from being trained on as many data as possible, and so many data are biased. Bias may also be encode in subtle ways, such as how often women are mentioned compared to men in certain contexts, meaning that discarding biased data might require complicated and labor-intensive analysis." (p.179-180)
- The impressive capabilities of modern foundation models to generate high-quality text, voice, and images come with the alarming possibility for generating high-quality misinformation. For example, it is easier than ever to generate photos of an existing person engaging in something that never happened, whether a crime, ... a battle, or even a party with the wrong kind of people. This is a problem because we are accustomed to believing photos... To the extent we rely on opinions of anonymous or pseudonymous people online, this poses a problem for democracy." (p.180-181). Yet, we can trust that what is stated in the Bible is 100% true. The original artifacts (such as scrolls in museums) existed well before AI and therefore are more difficult to falsify.
- "Seeing the power and versatility of LLMs, some people claim that they are the first steps toward AGI. I don't agree, and not only because we don't have a good definition of AGI. LLMs have plenty of shortcomings: they are generally bad at reasoning and planning, can't (on their own) really do math, and, perhaps most importantly, are extremely unreliable. Even the best LLMs hallucinate frequently in some situations. This makes them unsuitable on their own in many situations." (p.191)
- "A completely different approach to AGI is represented by open-ended learning. Here we are not training models on the cultural exhaust of humanity (text and images). Instead we build coupled systems of agents and problem generators, loosely inspired by how natural intelligence has developed through evolution in complex ecologies." (p.191). The jury is still out on this one; as computer engineers and programmers have not yet reached that point.
- "If we can build an AGI system, would it be conscious? Would there be something it would be like to be that system? Would it feel genuine pain and happiness, not just perform simulations for those feelings? We don't know... I majored in philosophy in undergrad largely because I wanted to understand whether machines could or would be conscious. After a few years of thinking about this, I concluded that I had no idea how I (or anyone else) could make progress on this question, so I switched to the easier problem of trying to create artificial intelligence. And that's where I still am." (p.192-193)
- "A question that is surprisingly, and annoyingly, widely debated as I'm writing this is whether AI risks leading to human extinction. The idea here is that once we create sufficiently advanced AI systems, they will be so intelligent that we cannot control them. They might proceed to improve themselves to be even further beyond our grasp, and then we might as well be to them as ants are to us... If you try to use well-defined words that refer to actual AI technologies and their capacities, the argument falls apart." (p.193). In Genesis, mankind was the apex of God's creation, and we cannot fathom His greatness. This book is a secular text, but it is claiming we can overcome our Creator and that simply isn't true.
- "Unfortunately, some people are convinced that further AI development poses serious risks and should be curtailed and controlled. Others are interested in bogging down AI research and governmental bureaucracy to preserve their own competitive position in AI development and therefore claim to worry about existential risk. It[s not pretty. As I'm writing this, proposals are being floated to require licenses for training large models. I think such regulations would be a big mistake, not only because of the chilling effects on AI research, but also because they resent a slippery slope in terms of free speech." (p.194)
- "This is not to say that there are no risks associated with the development and diffusion of more capable and general AI systems. Given the wide applicability of anything we may call AGI, it is likely that it will have major effects on society. While some worry that many, or even most, people may lose their jobs, I don't think that is likely." (p.195)
- "As the saying goes, AI is just computer science that doesn't really work yet; and when it does work, it's no longer AI." (p.200)
January 12, 2025
I listened to the unabridged 4-hour audio version of this title (read by Steve Marvel, Ascent Audio, 2024).
We are surrounded by Artificial intelligence. Knowingly or unknowingly, we use AI on a daily basis. Most AI is narrowly-focused and has highly-specific functionality, such as spell-checking or GO-playing. A spell-checking app cannot do math and a GO-playing program cannot play Tetris. Human intelligence is somehow more general, because we can solve a variety of tasks, including those we have never encountered before. Developing artificial general intelligence (AGI) is the holy grail of today’s AI Research.
According to NYU’s Professor Togelius, even human intelligence isn’t really general. We have developed an ad-hoc collection of analysis and decision-making skills through evolution. Therefore, even if we replicated human intelligence in a robot, it’s unclear that everyone would agree that the robot possessed AGI. We’ve had AI systems that are superhuman in some sense for more than half a century. To take the next step toward AGI, we need to develop a clear definition of what we are trying to achieve. We should look at definitions from the perspectives of psychology, ethology, and computer science.
Theoretically, an evolutionary method may be successful in developing AGI, but it will probably take too long. There are two promising families of technical approaches to developing AGI: Foundation models through self-supervised learning and open-ended learning in virtual environments. The designation self-supervised means that learning occurs without someone first going through massive amounts of data and labeling them. As for learning in virtual environments, the transition from a simulated world to the real world, where you may not be able to try things multiple times, is non-trivial.
In Chapters 9-11, Togelius investigates the potential of artificial general intelligence beyond the strictly technical aspects. Among the questions discussed are whether such general AI would be conscious, whether it would pose a risk to humanity, and how it might alter society.
We are surrounded by Artificial intelligence. Knowingly or unknowingly, we use AI on a daily basis. Most AI is narrowly-focused and has highly-specific functionality, such as spell-checking or GO-playing. A spell-checking app cannot do math and a GO-playing program cannot play Tetris. Human intelligence is somehow more general, because we can solve a variety of tasks, including those we have never encountered before. Developing artificial general intelligence (AGI) is the holy grail of today’s AI Research.
According to NYU’s Professor Togelius, even human intelligence isn’t really general. We have developed an ad-hoc collection of analysis and decision-making skills through evolution. Therefore, even if we replicated human intelligence in a robot, it’s unclear that everyone would agree that the robot possessed AGI. We’ve had AI systems that are superhuman in some sense for more than half a century. To take the next step toward AGI, we need to develop a clear definition of what we are trying to achieve. We should look at definitions from the perspectives of psychology, ethology, and computer science.
Theoretically, an evolutionary method may be successful in developing AGI, but it will probably take too long. There are two promising families of technical approaches to developing AGI: Foundation models through self-supervised learning and open-ended learning in virtual environments. The designation self-supervised means that learning occurs without someone first going through massive amounts of data and labeling them. As for learning in virtual environments, the transition from a simulated world to the real world, where you may not be able to try things multiple times, is non-trivial.
In Chapters 9-11, Togelius investigates the potential of artificial general intelligence beyond the strictly technical aspects. Among the questions discussed are whether such general AI would be conscious, whether it would pose a risk to humanity, and how it might alter society.
November 23, 2024
It's a really good history of AI and he certainly could be correct in his opinions. However, I think this section reveals either excessive skepticism or was written before the latest models:
It seems that the talking heads in the industry (Altman, Zuckerberg, Amodei) believe that LLMs are directly leading to AGI in the near future.
Also, his reluctance to define AGI after spending so many words on it drags down entire book for me:
I totally agree with his point but he doesn't seriously explore those interesting questions (mostly vague predictions in chapter 11). A much more thought provoking approach is Dario's definition and optimistic exploration of what a powerful AI will do to our society: https://darioamodei.com/machines-of-l...
I also liked this related interview a lot: https://www.youtube.com/watch?v=E7JsL...
Seeing the power and versatility of LLMs, some people claim that they are the first steps toward AGI. I don’t agree, and not only because we don’t have a good definition of AGI. LLMs have plenty of shortcomings: they are generally bad at reasoning and planning, can’t (on their own) really do math, and, perhaps most importantly, are extremely unreliable. Even the best LLMs hallucinate frequently in some situations. This makes them unsuitable on their own in many situations. However, it is possible to build systems around LLMs that amplify their power in various ways.
It seems that the talking heads in the industry (Altman, Zuckerberg, Amodei) believe that LLMs are directly leading to AGI in the near future.
Also, his reluctance to define AGI after spending so many words on it drags down entire book for me:
In fact, I think it would be best if we all simply stopped talking about AGI. It is leading us astray from the more important questions, which tend to focus on particular applications of AI technology and their consequences for society.
I totally agree with his point but he doesn't seriously explore those interesting questions (mostly vague predictions in chapter 11). A much more thought provoking approach is Dario's definition and optimistic exploration of what a powerful AI will do to our society: https://darioamodei.com/machines-of-l...
I also liked this related interview a lot: https://www.youtube.com/watch?v=E7JsL...
November 8, 2024
3.5 Surprisingly, I didn't learn as much as I expected from this book. I am not sure if that is a reflection of my own knowledge, or a warning sign that the details were too vague and surface-level. Nonetheless, this book provided a helpful framework to understand why debates over AI systems are so contentious and why philosophers and ethicists struggle to even agree on common definitions. More details would have been nice, but in the author's defence, he is clearly writing for people like me with no coding background who might struggle to understand the actual mechanics of various systems. Overall, this was a good overview and a fun read.
May 23, 2026
I've read ten or so books in this series and found this to be useful, but not nearly as broad as I expected. AGI is a tough problem. Many people are racing to develop AGI, yet basic work by the Logical Positivists (Godel in particular) shows that AGI is not possible in the way that people envision. The vast majority of people's view of AGI comes from Star Trek and similar science fiction. Remember fiction is fiction, not fact; it can embrace facts but rises above any inherent limitations the world, science and math may impose.
A greater discussion of philosophy that underpins the limits of knowledge would have been very helpful.
A greater discussion of philosophy that underpins the limits of knowledge would have been very helpful.
May 14, 2025
Great book to learn concepts about AGI and its definitions (or rather its issues in defining it as described in the book)! It starts with a short history of the development of artificial intelligence, focusing on different methodologies, and touches on consciousness and its impact on society. It also attempted the ambiguous definitions of intelligence and how to measure it as well as what it would like for intelligence to be "general".
TLDR of the book - No. AGI is impossible (at least in this lifetime)
TLDR of the book - No. AGI is impossible (at least in this lifetime)
October 18, 2024
I liked the book. Can you actually learn something? I don't know. If you don't know the subject you will not understand much. If you do know the subject, you probably know also what the book contains. I enjoyed it because it probably confirms my beliefs (formed by the writings of Andrew Ng). Short version : we have no idea how to define AGI. Worrying about AGI is like worrying about traffic congestion on Mars.
March 18, 2026
It felt like the sparks notes version of a full college course. I loved that the author started from our definitions of “intelligence” and a detailed description of how current LLMs and open-ended learning models. The exploration of what current AI does well (prediction, pattern identification, text generation) and what it does poorly (fact checking, spontaneous creation, math) is fascinating. Loved this one - want to read more of the MIT series.
December 28, 2024
This was a good overview, and it covered quite a bit of ground, with a reasonable but not overwhelming level of detail. But it was also very cautious with any sort of claim. The author was careful not to dive too deep into philosophical questions and kept things mostly technical, which, I think is okay given the size of the publication.
January 10, 2026
Kerrankin ehdin lukea tällaisen kirjan tuoreeltaan, vain jokunen kuukausi julkaisun jälkeen. Mutta niin vaan tässäkin oli jo auttamatta vanhentuneita asioita, niin järisyttävän nopeaa on etenkin kielimallien kehitys ollut. Varmasti muidenkin osa-alueiden, mutta niistä ei itsellä ole kokemusta. Oikein hyvä sisältö, käännös ja toimitus oli kyllä harmittavan heikkoja kääntäjän näkökulmasta.
January 15, 2025
Should be interesting, and the MIT label suggests an expert in the field. But I found it unreadable because of the large number of factual errors even in the first few pages, such as dating the Principia Mathematica a century earlier than it's actual date. DNF (unsurprisingly).
November 19, 2024
Very thorough and educational.
January 1, 2025
I'm a total layman, so much of it was helpful. However I found a lot of the back half sketchy (perhaps there's an inherent sketchiness there).
February 5, 2025
This book targets a wider audience since it tries to explain AI and AGI with pop culture references and basic examples.
August 17, 2025
A balanced treatment, suitable for the general reader.
November 20, 2025
great book. I consider this as the sequel of The Technological Singularity.
Displaying 1 - 20 of 20 reviews