What do you think?
The AI Revolution: Our Immortality or Extinction
The topic everyone in the world should be talking about.
84 pages, ebook
First published January 27, 2015
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 28 of 28 reviews
August 22, 2019
We're all philosophers -- the problem is that most of us are really bad at it. Everyone has to make philosophical decisions (how to live? what to do?) without much training in the topic. Thankfully philosophical mistakes usually have minor, local results; a very common error made by younger people, teenagers in particular, is to think that physical pleasure can bring you ultimate happiness and should be pursued at the expense of other, higher goods (sometimes it takes people a surprisingly long time to figure this one out!).
Other philosophical mistakes have much more serious results -- one of the reeeeeeeeally big philosophical mistakes was the Marxist concept of materialism (as interpreted by, e.g., Soviet Russia under Stalin; see here to start down the rabbit hole on that topic). Stalin's regime assumed they could create/condition a 'new man' out of behavioral responses (i.e., brainwashing, torture, reeducating children, etc.), because human nature is purely material, so you can change a person's heritable characteristics by changing his or her environment. (This is also a scientific mistake, of course, but the root of the scientific error is a philosophical error.) If Stalin and friends had studied any sort of philosophy other than a dumbed-down version of Marxism, then they would have immediately seen the fallacy here; really they just needed, like, any halfway intelligent philosophy professor to sit them down and explain that their simplistic interpretation of Pavlov is not an exhaustive account of human nature. It turns out that human beings are very complex, and that you cannot reengineer our nature -- or at least not on a large-enough scale, and not permanently (as Gary Snyder once put it, "nature always bats last").
Another less mass-murder-ish philosophical mistake, and to me one of the funniest ones, is when the logical positivists in the Vienna Circle -- to simplify slightly (it's a long story, I'm mainly referring to Carnap's critique of metaphysics) -- were refuted when Karl Popper pointed out that their foundational principle of verification, "truth is only that which can be empirically proved" . . . cannot be empirically proved lol. The principle itself fails to follow its own criterion! It's one of those beautiful self-owns that would have been obvious if the Vienna Circle had talked to any philosophers beyond each other -- they could have given Heidegger or Husserl a phone call in 1923 and wrapped up all their fatuous conferences at least a decade earlier. Yet before Popper's critique sunk in, Carnap and the rest were writing books, delivering lectures, etc., blissfully unaware that they were fundamentally mistaken. Is there some very limited value in their work, despite faulty premises? I mean, I guess? But this is a pretty serious mistake, to say the least.
Anyway, this is all just a preamble to what is, imo, by far the most fascinating philosophical mistake of all, the subject in question for Urban's ebook -- artificial general intelligence, superintelligence, "Strong AI," etc. Strong AI, as you've likely gathered from various sci-fi films/books and also from popularizations of the work of Bostrom, Kurzweil (the 'Singularity' guy), etc., is the idea that humans will (intentionally or unintentionally) soon build a bootstrapping AI that will raise itself into human-level and then smarter-than-human-level consciousness / intelligence / interiority / agency. (Some have argued that the interiority part is irrelevant, but externally such an AI will behave as if it had human agency, whether or not it 'experiences' this agency via qualia, and so this is a distinction without a difference.)
This Strong AI will then, supposedly, recursively increase its intelligence and possibly choose to destroy humanity, or build advanced spacecraft and send itself into deep space, etc., etc. To be clear, this is not self-iterating computing software that can be harnessed by humans, which is "weak AI," machine learning, etc.; weak AI is 100% a real thing that exists right now (in my day job, I've actually worked on grant applications for researchers studying this topic). Strong AI is qualitatively different, and completely theoretical.
AGI / Strong AI is such a basic/obvious philosophical error that I simply cannot believe that there are millions of people, thousands of scholars, various tech billionaires pouring money into giant institutes, etc., for the sake of a concept that is literally a category mistake. I've mentioned this topic in a couple other GR reviews (e.g., see here), so forgive me if you've heard me ranting about this before, but I am just impossibly fascinated by how obviously wrong Strong AI is, and I'm drawn back to the topic and am remystified anew whenever I see a new book on it. It's as if there's an entire academic field devoted to, like, making sure that Disney movie villains won't escape Disney+ streaming services and invade the earth, where actual scholars at actual universities spend their days making contingency plans to ensure that Maleficent or Thanos or Syndrome won't enter our world and start taking over military installations. It's just fascinatingly insane.
D. B. Hart provides the most clearly reasoned critique of the issues with Strong AI (though Hubert Dreyfus is also very good), which I've condensed here:
Ultimately I think you can argue that this is all the result of a curious mistake made a few centuries ago. In the early modern period, some philosophers figured out that it's far easier to do natural science when we bracket non-efficient causality (i.e., when we treat nature, including human nature, like a mechanism). So they temporarily bracketed final causality and all metaphysics beyond the material -- i.e., the Great Chain of Being, a robust account of reality, which was taken to be obviously true by everyone -- in order to make calculations more efficiently. This was never meant to be a rival metaphysics that would be a completely exhaustive account of reality. Yet, somehow, this temporary bracketing has since become the default metaphysical viewpoint of many philosophers, including those behind Strong AI theory, who do not understand that they're following in the dimly-understood footsteps of someone who, at some point, said: "Let's bracket what the mind actually is and temporarily understand it in terms of a machine." In short, someone forgot to let all these people know that "brain = machine" was a metaphor.
And look, I understand that the reader may not believe me -- maybe it seems like an internecine philosophical debate where really both sides have a point -- but all I can say is that it is not a debate, this isn't like "do we have a priori synthetic knowledge" (an interesting question that you can argue either way) or "deontology vs. utilitarianism" (ditto), this is more like "are squares actually circles," and the answer is just no. It's so obviously wrong to anyone who has studied metaphysics that if you were to pull aside any scholar of ancient or medieval philosophy who has never thought about AI before, had never read sci-fi, knows nothing about the debates, and explained the basic premises of AGI theory to them, at a conference somewhere -- I guarantee that they would just start laughing out loud. But you don't even need to have this much knowledge -- it's possible to refute Strong AI even if you hold naive materialist premises (see this recent study, for example).
I think the impact of sci-fi has been an underrated factor in the general cultural acceptance of Strong AI as an (inevitable) reality. Strong AI in films and books is typically presented as a human-like robot/android, which is obviously a projection of our consciousness onto a robotic frame (and also, in a literary sense, represents various aspects of human nature, similar to the function of elves in Tolkien, but I mean apart from that). It's a very compelling image; an advanced CPU with billions of transistors, running on electricity and miniaturized to fit inside a robot's head, which is clearly similar (superficially) to an advanced brain with billions of neurons, running on electricity and miniaturized to fit inside our skull.
This momentarily compelling analogy can be shaken off if we understand that computers could just as easily be hydraulic. Hillis has an interesting passage about this:
Hillis wrote this book many years ago, so his analogous hydraulic computer would now need to be quite a bit larger. An RTX 3090 has 28.3 billion transistors, which gives us 2,500 miles x 2,500 miles, slightly bigger than the continental U.S.; we could then stack an Intel i9 CPU on top (8 billion transistors), with similar arrays for input, RAM, etc., so maybe 40 billion hydraulic valves stacked on each other. There is no electricity in this system whatsoever; it's just a series of pipes sitting on the ground. However, it's very important to grasp that this is PRECISELY as much of a computer as the PC sitting on my desk right now (it's just quite a bit slower!).
And so this perhaps makes the point easier to grasp -- do you think that a 2,500 mile x 2,500 mile array of PVC pipes can achieve self-iterating super-human-level intentionality, consciousness, and goal-oriented behavior if we line up the pipes so that it sort of, vaguely, looks like a cargo-cult version of a human neural network if you're flying over it in a plane? Interiority and consciousness and choice are going to just emerge from this, magically? The fact that people believe this about a few pieces of metal/stone/plastic (i.e. a modern PC) is -- technically speaking -- just as absurd, but this absurdity is perhaps more readily apparent if we think about what a computer really is.
edit: 2024 update. I've done quite a bit more reading on these topics and have also talked with folks who work in AI, have read more about LLMs, etc.; I will say that my position has softened somewhat, and that certain basic elements of 'thought' do appear to be natural, as it were, and really can emerge from artificial neural nets in a way that is loosely similar to human thought (though with various qualitative differences).
I still think that superintelligence, as such, is theoretical at best, but some of the other questions are worth exploring. Yet without superintelligence, the supposedly catastrophic results cannot happen. If an AI never becomes superintelligent, then we have absolute power over AI; it isn't Skynet, it doesn't have control of our weapons systems (we can just, like, unplug the AI?) and isn't an 'agent' in the superintelligent sense.
Other philosophical mistakes have much more serious results -- one of the reeeeeeeeally big philosophical mistakes was the Marxist concept of materialism (as interpreted by, e.g., Soviet Russia under Stalin; see here to start down the rabbit hole on that topic). Stalin's regime assumed they could create/condition a 'new man' out of behavioral responses (i.e., brainwashing, torture, reeducating children, etc.), because human nature is purely material, so you can change a person's heritable characteristics by changing his or her environment. (This is also a scientific mistake, of course, but the root of the scientific error is a philosophical error.) If Stalin and friends had studied any sort of philosophy other than a dumbed-down version of Marxism, then they would have immediately seen the fallacy here; really they just needed, like, any halfway intelligent philosophy professor to sit them down and explain that their simplistic interpretation of Pavlov is not an exhaustive account of human nature. It turns out that human beings are very complex, and that you cannot reengineer our nature -- or at least not on a large-enough scale, and not permanently (as Gary Snyder once put it, "nature always bats last").
Another less mass-murder-ish philosophical mistake, and to me one of the funniest ones, is when the logical positivists in the Vienna Circle -- to simplify slightly (it's a long story, I'm mainly referring to Carnap's critique of metaphysics) -- were refuted when Karl Popper pointed out that their foundational principle of verification, "truth is only that which can be empirically proved" . . . cannot be empirically proved lol. The principle itself fails to follow its own criterion! It's one of those beautiful self-owns that would have been obvious if the Vienna Circle had talked to any philosophers beyond each other -- they could have given Heidegger or Husserl a phone call in 1923 and wrapped up all their fatuous conferences at least a decade earlier. Yet before Popper's critique sunk in, Carnap and the rest were writing books, delivering lectures, etc., blissfully unaware that they were fundamentally mistaken. Is there some very limited value in their work, despite faulty premises? I mean, I guess? But this is a pretty serious mistake, to say the least.
Anyway, this is all just a preamble to what is, imo, by far the most fascinating philosophical mistake of all, the subject in question for Urban's ebook -- artificial general intelligence, superintelligence, "Strong AI," etc. Strong AI, as you've likely gathered from various sci-fi films/books and also from popularizations of the work of Bostrom, Kurzweil (the 'Singularity' guy), etc., is the idea that humans will (intentionally or unintentionally) soon build a bootstrapping AI that will raise itself into human-level and then smarter-than-human-level consciousness / intelligence / interiority / agency. (Some have argued that the interiority part is irrelevant, but externally such an AI will behave as if it had human agency, whether or not it 'experiences' this agency via qualia, and so this is a distinction without a difference.)
This Strong AI will then, supposedly, recursively increase its intelligence and possibly choose to destroy humanity, or build advanced spacecraft and send itself into deep space, etc., etc. To be clear, this is not self-iterating computing software that can be harnessed by humans, which is "weak AI," machine learning, etc.; weak AI is 100% a real thing that exists right now (in my day job, I've actually worked on grant applications for researchers studying this topic). Strong AI is qualitatively different, and completely theoretical.
AGI / Strong AI is such a basic/obvious philosophical error that I simply cannot believe that there are millions of people, thousands of scholars, various tech billionaires pouring money into giant institutes, etc., for the sake of a concept that is literally a category mistake. I've mentioned this topic in a couple other GR reviews (e.g., see here), so forgive me if you've heard me ranting about this before, but I am just impossibly fascinated by how obviously wrong Strong AI is, and I'm drawn back to the topic and am remystified anew whenever I see a new book on it. It's as if there's an entire academic field devoted to, like, making sure that Disney movie villains won't escape Disney+ streaming services and invade the earth, where actual scholars at actual universities spend their days making contingency plans to ensure that Maleficent or Thanos or Syndrome won't enter our world and start taking over military installations. It's just fascinatingly insane.
D. B. Hart provides the most clearly reasoned critique of the issues with Strong AI (though Hubert Dreyfus is also very good), which I've condensed here:
For some theorists, an artificial computer comparable in complexity to the human brain and nervous system could achieve something like our conscious states. This entire theory is an incorrigible confusion of categories, for a very great number of reasons, foremost among them the absolute dependency of all computational processes upon the prior reality of intentional consciousness. The physical brain may be something very remotely like the physical object we use to run software programs, but to speak of the mind in terms of computation is really no better than speaking of representation in terms of photography.
We have become so accustomed to speaking of computers as artificial minds and of their operations as thinking we have forgotten that these are mere figures of speech. We speak of computer memory, for instance, but of course computers recall nothing. They do not even store any 'remembered' information -- in the sense of symbols with real semantic content, real meaning -- but only preserve the binary patterns of certain electronic notations. And I do not mean simply that the computers are not aware of the information they contain; I mean that, in themselves, they do not contain any semantic information at all. They are merely the silicon parchment and electrical ink on which we record symbols that possess semantic content only in respect to our intentional representations of their meanings.
Nor can one credibly argue that, even though computer 'memory' may have no intentional meaning, still the 'higher functions' of the computer's software transform those notations into coherent meanings by integrating them into a larger functional system. There are no higher functions and no programs as such, either in the computer considered purely as a physical object or in its operations considered purely as physical events; there are only the material components of the machine, electrical impulses, and binary patterns, which we use to construct certain representations and which have meanings only so long as they are the objects of the representing mind's attention.
We have imposed the metaphor of an artificial mind on computers and then reimported the image of a thinking machine and imposed it upon our own minds. Computational models of the mind would make sense if what a computer actually does could be characterized as an elementary version of what the mind does, or at least as something remotely like thinking. In fact, though, there is not even a useful analogy to be drawn here. A computer does not even really compute. We compute, using it as a tool. Software no more 'thinks' than a minute hand knows the time or the printed word 'pelican' knows what a pelican is. We might just as well liken the mind to an abacus, a typewriter, or a library. No computer has ever used language, or responded to a question, or assigned a meaning to anything. No computer has ever so much as added two numbers together, let alone entertained a thought, and none ever will. The only intelligence or consciousness or even illusion of consciousness in the whole computational process is situated, quite incommutably, in us; everything seemingly analogous to our minds in our machines is reducible, when analyzed correctly, only back to our own minds once again, and we end where we began.
Rational thought -- understanding, intention, will, consciousness -- is not a species of computation. To imagine that it is involves an error regarding not only what the mind does, but what a computer does as well. Even if we could imaginatively or deductively descend from the level of consciousness down through strata or symbols, simple notational functions, and neural machinery, we would not be able then to ascend back again the way we came. Once more, the physicalist reduction of any phenomenon to purely material forces explains nothing if one cannot then reconstruct that phenomenon from its material basis without invoking any higher causes; but this no computational picture of thought can ever do. Symbols exist only from above, as it were, in the consciousness looking downward along the path of that descent, acting always as a higher cause upon material reality. Looking up in the opposite direction, from below to above, one finds only an untraversable abyss. It is an absolute error to imagine that the electrical activity in a computer is itself computation; and, when a believer in A.I. claims that the electrochemical operations of a brain are a kind of computation, and that consciousness arises from that computation, he or she is saying something utterly without meaning. All computation is ontologically dependent on consciousness, simply said, and so computation cannot provide the foundation upon which consciousness rests. One might just as well attempt to explain the existence of the sun as the result of the warmth and brightness of summer days.
Ultimately I think you can argue that this is all the result of a curious mistake made a few centuries ago. In the early modern period, some philosophers figured out that it's far easier to do natural science when we bracket non-efficient causality (i.e., when we treat nature, including human nature, like a mechanism). So they temporarily bracketed final causality and all metaphysics beyond the material -- i.e., the Great Chain of Being, a robust account of reality, which was taken to be obviously true by everyone -- in order to make calculations more efficiently. This was never meant to be a rival metaphysics that would be a completely exhaustive account of reality. Yet, somehow, this temporary bracketing has since become the default metaphysical viewpoint of many philosophers, including those behind Strong AI theory, who do not understand that they're following in the dimly-understood footsteps of someone who, at some point, said: "Let's bracket what the mind actually is and temporarily understand it in terms of a machine." In short, someone forgot to let all these people know that "brain = machine" was a metaphor.
And look, I understand that the reader may not believe me -- maybe it seems like an internecine philosophical debate where really both sides have a point -- but all I can say is that it is not a debate, this isn't like "do we have a priori synthetic knowledge" (an interesting question that you can argue either way) or "deontology vs. utilitarianism" (ditto), this is more like "are squares actually circles," and the answer is just no. It's so obviously wrong to anyone who has studied metaphysics that if you were to pull aside any scholar of ancient or medieval philosophy who has never thought about AI before, had never read sci-fi, knows nothing about the debates, and explained the basic premises of AGI theory to them, at a conference somewhere -- I guarantee that they would just start laughing out loud. But you don't even need to have this much knowledge -- it's possible to refute Strong AI even if you hold naive materialist premises (see this recent study, for example).
I think the impact of sci-fi has been an underrated factor in the general cultural acceptance of Strong AI as an (inevitable) reality. Strong AI in films and books is typically presented as a human-like robot/android, which is obviously a projection of our consciousness onto a robotic frame (and also, in a literary sense, represents various aspects of human nature, similar to the function of elves in Tolkien, but I mean apart from that). It's a very compelling image; an advanced CPU with billions of transistors, running on electricity and miniaturized to fit inside a robot's head, which is clearly similar (superficially) to an advanced brain with billions of neurons, running on electricity and miniaturized to fit inside our skull.
This momentarily compelling analogy can be shaken off if we understand that computers could just as easily be hydraulic. Hillis has an interesting passage about this:
Except for the miracle of reduction, there is no special reason to build computers with silicon technology. A hydraulic computer would work just as well . . . to use the hydraulic computer, you would have to connect hydraulic equivalents of its inputs and outputs—you would need to build a hydraulic keyboard, a hydraulic display, hydraulic memory chips, and so on—but if you did all this, it would go through exactly the same switching events as the electronic chip. Of course, the hydraulic computer would be much slower than your latest microprocessor (to say nothing of larger), because water pressure travels down pipes much more slowly than electricity travels down wires. Since the modern microchip has several million transistors, its hydraulic equivalent would require several million valves. A transistor in a chip is about a millionth of a meter across; a hydraulic valve is about 10 centimeters on a side. If the pipes scale proportionally, then the hydraulic computer would cover about a square kilometer with pipes and valves.
Hillis wrote this book many years ago, so his analogous hydraulic computer would now need to be quite a bit larger. An RTX 3090 has 28.3 billion transistors, which gives us 2,500 miles x 2,500 miles, slightly bigger than the continental U.S.; we could then stack an Intel i9 CPU on top (8 billion transistors), with similar arrays for input, RAM, etc., so maybe 40 billion hydraulic valves stacked on each other. There is no electricity in this system whatsoever; it's just a series of pipes sitting on the ground. However, it's very important to grasp that this is PRECISELY as much of a computer as the PC sitting on my desk right now (it's just quite a bit slower!).
And so this perhaps makes the point easier to grasp -- do you think that a 2,500 mile x 2,500 mile array of PVC pipes can achieve self-iterating super-human-level intentionality, consciousness, and goal-oriented behavior if we line up the pipes so that it sort of, vaguely, looks like a cargo-cult version of a human neural network if you're flying over it in a plane? Interiority and consciousness and choice are going to just emerge from this, magically? The fact that people believe this about a few pieces of metal/stone/plastic (i.e. a modern PC) is -- technically speaking -- just as absurd, but this absurdity is perhaps more readily apparent if we think about what a computer really is.
edit: 2024 update. I've done quite a bit more reading on these topics and have also talked with folks who work in AI, have read more about LLMs, etc.; I will say that my position has softened somewhat, and that certain basic elements of 'thought' do appear to be natural, as it were, and really can emerge from artificial neural nets in a way that is loosely similar to human thought (though with various qualitative differences).
I still think that superintelligence, as such, is theoretical at best, but some of the other questions are worth exploring. Yet without superintelligence, the supposedly catastrophic results cannot happen. If an AI never becomes superintelligent, then we have absolute power over AI; it isn't Skynet, it doesn't have control of our weapons systems (we can just, like, unplug the AI?) and isn't an 'agent' in the superintelligent sense.
November 14, 2017
In Tim Urban's words: "I don’t get why everyone isn’t talking about this?”
September 4, 2022
So inspirational. Can’t believe I didn’t pay more attention to AI revolution before. Must read.
November 13, 2018
This article on the existential impact of Artificial Intelligence is one of the most insightful pieces on technology I have read in quite some time. My reaction to this article was exactly what is expressed in Tim Urban's own words: "I don’t get why everyone isn’t talking about this?"
TL:DR
Our view of AI is very much distorted by its inaccurate portrayal in contemporary science fiction. True AI (In author's terms - Artificial General Intelligence, as opposed to Artificial Narrow Intelligence that we know of today in the form of Google Assistant, Siri, or Youtube recommendations) is much closer to being a reality than what most of us think. What is more intriguing, though perfectly logical, is the fact that there is no reason for the evolution of AI to stop once it reaches the critical threshold which we call Human-Level Machine Intelligence. Exponential growth of computational capabilities will ensure that the milestone of computers achieving human level intelligence will be surpassed exponentially faster than what it would take to get there. We will traverse into the realm of "Artificial Super-Intelligence" without ever being prepared for it. And once we are there, we are past the point of no return.
A machine whose intellectual capabilities surpass human intelligence by orders of magnitude is impossible to be understood by our primitive brains, even if the machine tried to explain itself to us. It would be like us trying to explain the human understanding of the universe to a mouse. We humans, during the course of the millenia since the cognitive revolution, had to conjure up imaginary constructs like morality, ethics, religion, nationhood etc. in order to keep ourselves together (and still there are times we have spectacularly failed). A super-intelligent computer, if not devoid of all those, would at best have a moral code which could be significantly different from ours. To think that it would still serve its lesser creators because of some sense of gratitude is probably a long shot. Haven't we humans exploited the nature to our needs for centuries without thinking twice about what adverse effects our actions might have on the "lesser life forms".
But there are experts who believe the opposite is true. A supremely intelligent being might not be benevolent, but doesn't necessarily need to be malevolent as long as we carefully define it in the objectives of its precursors. If we succeed in doing that, we might be on the verge of another revolution, identical in scale and impact to agricultural revolution itself, but millions of times faster and more efficient.
Whichever way the pendulum swings, the debate between the experts is on whether the end result would be good or bad for humanity. Nobody doubts the inevitability of it. It's just a matter of when, and we might just find out in our lifetimes.
TL:DR
Our view of AI is very much distorted by its inaccurate portrayal in contemporary science fiction. True AI (In author's terms - Artificial General Intelligence, as opposed to Artificial Narrow Intelligence that we know of today in the form of Google Assistant, Siri, or Youtube recommendations) is much closer to being a reality than what most of us think. What is more intriguing, though perfectly logical, is the fact that there is no reason for the evolution of AI to stop once it reaches the critical threshold which we call Human-Level Machine Intelligence. Exponential growth of computational capabilities will ensure that the milestone of computers achieving human level intelligence will be surpassed exponentially faster than what it would take to get there. We will traverse into the realm of "Artificial Super-Intelligence" without ever being prepared for it. And once we are there, we are past the point of no return.
A machine whose intellectual capabilities surpass human intelligence by orders of magnitude is impossible to be understood by our primitive brains, even if the machine tried to explain itself to us. It would be like us trying to explain the human understanding of the universe to a mouse. We humans, during the course of the millenia since the cognitive revolution, had to conjure up imaginary constructs like morality, ethics, religion, nationhood etc. in order to keep ourselves together (and still there are times we have spectacularly failed). A super-intelligent computer, if not devoid of all those, would at best have a moral code which could be significantly different from ours. To think that it would still serve its lesser creators because of some sense of gratitude is probably a long shot. Haven't we humans exploited the nature to our needs for centuries without thinking twice about what adverse effects our actions might have on the "lesser life forms".
But there are experts who believe the opposite is true. A supremely intelligent being might not be benevolent, but doesn't necessarily need to be malevolent as long as we carefully define it in the objectives of its precursors. If we succeed in doing that, we might be on the verge of another revolution, identical in scale and impact to agricultural revolution itself, but millions of times faster and more efficient.
Whichever way the pendulum swings, the debate between the experts is on whether the end result would be good or bad for humanity. Nobody doubts the inevitability of it. It's just a matter of when, and we might just find out in our lifetimes.
May 6, 2020
Well this guy pretty much explains how the AI stands and the impact it might have in the close future. Well, quoting him, I think most likely 'we're fucked', though I don't think it will be this close in the future, aka rip his dream of immortality.
October 29, 2020
Es el resumen que tenés que leer si querías entender algo de inteligencia artificial
April 2, 2023
Insightful and intriguing.
January 18, 2025
This is a little book that you can read if you don't feel like slogging through The Singularity is Near or Superintelligence. Short, presents much of the overall argument of each, shorter on the neologisms, filled with the same mistakes (as I perceive them). The general state of the writing is outdated, since it is undoubtedly a fast moving topic, but, unfortunately for those who have bolted themselves onto the singularity hype train, it both does and doesn't affect the arguments presented. One of the more glaring examples of this is quoted here: "So the world’s $1,000 computers are now beating the mouse brain and they’re at about a thousandth of human level. This doesn’t sound like much until you remember that we were at about a trillionth of human level in 1985, a billionth in 1995, and a millionth in 2005. Being at a thousandth in 2015 puts us right on pace to get to an affordable computer by 2025 that rivals the power of the brain."
So, obviously the jury is still out here, but it seems like we aren't getting the quadrillion FLOPs $1000 dollar computer just yet. Plus the impressive but certainly not AGI ChatGPT and OpenAI systems we have been getting also seem to be showing logistic curves of progression. This is something that Kurzweil notes, but the idea is that the full arc of the S-curve occurs in shorter and shorter intervals. This might be happening, but it could also be the case the the current architecture that LLMs are built upon experiences a phase transition when its given enough computing power and a large enough data set (like water does when it goes from a liquid to a gas when both the pressure and the temperature scale continuously). It goes from being pretty bad, increases in quality rapidly, and then you can scale up its training data sets as much as you want but the returns to performance will continually be diminishing. I'm not convinced yet that we won't see a period of time coming up relatively soon where progress in AI slows down again as we try to figure out the architecture of the AI systems that we need to transition to to make the next really big jump in performance. You can only get some much juice out of one lemon, and maybe the LLMs are nothing but rind now. I'm also not convinced this isn't a more generally applicable hurdle that the bright AI folks will have to worry about. Just because we have continued to make progress in all the relevant areas of technology doesn't mean passenger air flight speeds didn't level off hard half a century ago. The arguments for exponential increase are logical enough, but that isn't a reason to strongly believe that they are realistic. Plus the problems of embodied intelligence still seem daunting enough to me.
I do think that AGI is possible, after all, nature did it. Rewind the clock 5 billion years and, so far as we can tell, there wasn't anything like intelligence in the universe. Then, following nothing but the imperatives of the laws of physics, the sun and Earth formed, abiogenesis occurred, the first eukaryotic cells developed, diversity of lifeforms exploded in the Cambrian, and not too long after there was an intelligent species of hairy bipedal apes intentionally using fire to cook food, clear lands for agriculture, and probably even to make hunting easier. That was around a million years ago, and sometime between now and then molecules "learned" how to ask what molecules are and followed that up with "how can molecules think." From sterile universe to conscious thought in 5 billion years, no miracles necessary. There aren't many good reasons to suppose that intelligence is substrate dependent, or that there are things that only nature can do (though manipulating gravitational or Higgs fields seems like we are gonna have to leave to the universe, at least for a while). It will probably be possible to make machines that are reliably "smarter" than humans, however we eventually define it in this case. Since people like Von Neumann and Einstein existed, we can probably build something at least a lot smarter than the average person. There are probably O(~10,000) genes responsible for affecting our intelligence and the parameter space is large enough that evolution hasn't had the time to explore the extreme corners of that space. Since the square root of the number of genes involved are all that's needed to change in order to bring about a SD increase or decrease, a Von Neumann, assuming an IQ of 175, would be the result of there being about 500 more positive contributing alleles than is normal. With 10,000 up for grabs, there seems to be plenty of room at the top.
This quote from Tim Urban generally sums it up: "As of now, humans have conquered the lowest caliber of AI—ANI—in many ways, and it’s everywhere. The AI Revolution is the road from ANI, through AGI, to ASI—a road we may or may not survive but that, either way, will change everything." He, and others, think that this going to happen faster and faster. I think we will continue to make progress, but I'm not convinced. I'm not convinced that they are wrong, but don't see strong empirical grounds for their level of zeal. How and when will the AI be able to functionally interact with the physical world of its own volition? This seems important, since, otherwise, it would need to be able to recursively self-improve itself from AGI to ASI on the same hardware, or would need to suffer a major setback in its growth. When will it make the seemingly unphysical nanotech that will allow it to perform all the magic that some seem to think will be waiting for us? When will AI switch from just vectoring certain phrases to others via nodes to actually understanding the content of what it is saying? The idea that an AI like the ones we have now could just be scaled up a bit more, become and AGI/ASI, and then "within an hour of hitting that milestone, the system pumps out the grand theory of physics that unifies general relativity and quantum mechanics," seems almost laughable as well. With large data sets for it to train on, since we don't really have things like that in physics, it is supposed to be able to draw out the complex, likely nonlinear interactions occurring in the hearts of black holes just because it is smart enough? How will it accomplish these goals when it doesn't currently understand a single thing? No understanding, no gigantic dataset of black hole singularity dynamics, but the AI will just piece it together in minutes of waking up?
The last paragraph contains this: "If our meager brains were able to invent wifi, then something 100 or 1,000 or 1 billion times smarter than we are should have no problem controlling the positioning of each and every atom in the world in any way it likes, at any time—everything we consider magic, every power we imagine a supreme God to have will be as mundane an activity for the ASI as flipping on a light switch is for us." Again, by what physical mechanism would any of this be possible? Appealing vaguely and broadly to a quote like "Bees can't understand Keynesian Economics" doesn't give us a strong justification for saying "all laws of physics will be violated." By what physical mechanism does Tim Urban propose any agent can control the position of every atom in the universe at every time? Uncertainty Principle aside, which bosonic field will be generated in a perfectly tailored way to tell the cesium atoms in the Andromeda galaxy to do the cha-cha slide? How, given the practically ancient idea of the inverse square law, does Urban propose the AI will do that without having to produce and physically control enough energy (from where?) to create a black hole? For a lot of these people, they seem to engage in actual magical thinking when it comes to intelligence, that is enough intelligence = actual magic.
Maybe we progressed so fast because we were on our own logistic curve of sorts. Maybe the universe has a finite amount of underlying, fundamental knowledge deeply encoded into it. Think here, laws of physics, general rules of higher order, complex interactions that capture the behavior of living things, etc. For a very long time we had something like a 0 to 1 correspondence with the universal, that is our mental model of the universe understood nothing/got nothing right. Maybe the advent and ascent of science and a scientific culture has brought us to a .9 to 1 or 90% correspondence between our model and nature. We've integrated the information Gaussian from negative infinity to 1.5 SDs above the mean and we don't have much understanding gain to pick up by filling out the last leg of the error function, most of the knowledge is here and we aren't going to really revolutionize anything by 10x-ing our intelligence. Sean Carroll has said (and I think he is right) that we pretty much have all the physics of everyday life figured out already, so maybe there aren't any major surprises waiting for someone 400 IQ points to the right of us mortals. Maybe, maybe not.
So, obviously the jury is still out here, but it seems like we aren't getting the quadrillion FLOPs $1000 dollar computer just yet. Plus the impressive but certainly not AGI ChatGPT and OpenAI systems we have been getting also seem to be showing logistic curves of progression. This is something that Kurzweil notes, but the idea is that the full arc of the S-curve occurs in shorter and shorter intervals. This might be happening, but it could also be the case the the current architecture that LLMs are built upon experiences a phase transition when its given enough computing power and a large enough data set (like water does when it goes from a liquid to a gas when both the pressure and the temperature scale continuously). It goes from being pretty bad, increases in quality rapidly, and then you can scale up its training data sets as much as you want but the returns to performance will continually be diminishing. I'm not convinced yet that we won't see a period of time coming up relatively soon where progress in AI slows down again as we try to figure out the architecture of the AI systems that we need to transition to to make the next really big jump in performance. You can only get some much juice out of one lemon, and maybe the LLMs are nothing but rind now. I'm also not convinced this isn't a more generally applicable hurdle that the bright AI folks will have to worry about. Just because we have continued to make progress in all the relevant areas of technology doesn't mean passenger air flight speeds didn't level off hard half a century ago. The arguments for exponential increase are logical enough, but that isn't a reason to strongly believe that they are realistic. Plus the problems of embodied intelligence still seem daunting enough to me.
I do think that AGI is possible, after all, nature did it. Rewind the clock 5 billion years and, so far as we can tell, there wasn't anything like intelligence in the universe. Then, following nothing but the imperatives of the laws of physics, the sun and Earth formed, abiogenesis occurred, the first eukaryotic cells developed, diversity of lifeforms exploded in the Cambrian, and not too long after there was an intelligent species of hairy bipedal apes intentionally using fire to cook food, clear lands for agriculture, and probably even to make hunting easier. That was around a million years ago, and sometime between now and then molecules "learned" how to ask what molecules are and followed that up with "how can molecules think." From sterile universe to conscious thought in 5 billion years, no miracles necessary. There aren't many good reasons to suppose that intelligence is substrate dependent, or that there are things that only nature can do (though manipulating gravitational or Higgs fields seems like we are gonna have to leave to the universe, at least for a while). It will probably be possible to make machines that are reliably "smarter" than humans, however we eventually define it in this case. Since people like Von Neumann and Einstein existed, we can probably build something at least a lot smarter than the average person. There are probably O(~10,000) genes responsible for affecting our intelligence and the parameter space is large enough that evolution hasn't had the time to explore the extreme corners of that space. Since the square root of the number of genes involved are all that's needed to change in order to bring about a SD increase or decrease, a Von Neumann, assuming an IQ of 175, would be the result of there being about 500 more positive contributing alleles than is normal. With 10,000 up for grabs, there seems to be plenty of room at the top.
This quote from Tim Urban generally sums it up: "As of now, humans have conquered the lowest caliber of AI—ANI—in many ways, and it’s everywhere. The AI Revolution is the road from ANI, through AGI, to ASI—a road we may or may not survive but that, either way, will change everything." He, and others, think that this going to happen faster and faster. I think we will continue to make progress, but I'm not convinced. I'm not convinced that they are wrong, but don't see strong empirical grounds for their level of zeal. How and when will the AI be able to functionally interact with the physical world of its own volition? This seems important, since, otherwise, it would need to be able to recursively self-improve itself from AGI to ASI on the same hardware, or would need to suffer a major setback in its growth. When will it make the seemingly unphysical nanotech that will allow it to perform all the magic that some seem to think will be waiting for us? When will AI switch from just vectoring certain phrases to others via nodes to actually understanding the content of what it is saying? The idea that an AI like the ones we have now could just be scaled up a bit more, become and AGI/ASI, and then "within an hour of hitting that milestone, the system pumps out the grand theory of physics that unifies general relativity and quantum mechanics," seems almost laughable as well. With large data sets for it to train on, since we don't really have things like that in physics, it is supposed to be able to draw out the complex, likely nonlinear interactions occurring in the hearts of black holes just because it is smart enough? How will it accomplish these goals when it doesn't currently understand a single thing? No understanding, no gigantic dataset of black hole singularity dynamics, but the AI will just piece it together in minutes of waking up?
The last paragraph contains this: "If our meager brains were able to invent wifi, then something 100 or 1,000 or 1 billion times smarter than we are should have no problem controlling the positioning of each and every atom in the world in any way it likes, at any time—everything we consider magic, every power we imagine a supreme God to have will be as mundane an activity for the ASI as flipping on a light switch is for us." Again, by what physical mechanism would any of this be possible? Appealing vaguely and broadly to a quote like "Bees can't understand Keynesian Economics" doesn't give us a strong justification for saying "all laws of physics will be violated." By what physical mechanism does Tim Urban propose any agent can control the position of every atom in the universe at every time? Uncertainty Principle aside, which bosonic field will be generated in a perfectly tailored way to tell the cesium atoms in the Andromeda galaxy to do the cha-cha slide? How, given the practically ancient idea of the inverse square law, does Urban propose the AI will do that without having to produce and physically control enough energy (from where?) to create a black hole? For a lot of these people, they seem to engage in actual magical thinking when it comes to intelligence, that is enough intelligence = actual magic.
Maybe we progressed so fast because we were on our own logistic curve of sorts. Maybe the universe has a finite amount of underlying, fundamental knowledge deeply encoded into it. Think here, laws of physics, general rules of higher order, complex interactions that capture the behavior of living things, etc. For a very long time we had something like a 0 to 1 correspondence with the universal, that is our mental model of the universe understood nothing/got nothing right. Maybe the advent and ascent of science and a scientific culture has brought us to a .9 to 1 or 90% correspondence between our model and nature. We've integrated the information Gaussian from negative infinity to 1.5 SDs above the mean and we don't have much understanding gain to pick up by filling out the last leg of the error function, most of the knowledge is here and we aren't going to really revolutionize anything by 10x-ing our intelligence. Sean Carroll has said (and I think he is right) that we pretty much have all the physics of everyday life figured out already, so maybe there aren't any major surprises waiting for someone 400 IQ points to the right of us mortals. Maybe, maybe not.
August 30, 2018
5 stars
WOAH. I'm not even sure how to process this book, but I'll do my best.
I try to be "rigorous" in at least some sense of the word when I review on here. My five-star rating is reserved for simply the best written works I have found so far---writings that make me consider a new perspective or think deeply on the subject at hand---and there's not a ton out there that does that anymore.
This book, like the others before it, have truly made me rethink how I consider artificial intelligence and really consider the ethical dilemmas it proposes. It's not perfect---Urban simplifies heavily so any layman with any knowledge about AI can understand the book---but it's well-researched with a striking amount of depth and his characteristic humor. I'll leave a bit of my favorite part below:
WOAH. I'm not even sure how to process this book, but I'll do my best.
I try to be "rigorous" in at least some sense of the word when I review on here. My five-star rating is reserved for simply the best written works I have found so far---writings that make me consider a new perspective or think deeply on the subject at hand---and there's not a ton out there that does that anymore.
This book, like the others before it, have truly made me rethink how I consider artificial intelligence and really consider the ethical dilemmas it proposes. It's not perfect---Urban simplifies heavily so any layman with any knowledge about AI can understand the book---but it's well-researched with a striking amount of depth and his characteristic humor. I'll leave a bit of my favorite part below:
A 15-person startup company called Robotica has the stated mission of “Developing innovative Artificial Intelligence tools that allow humans to live more and work less.” They have several existing products already on the market and a handful more in development. They’re most excited about a seed project named Turry. Turry is a simple AI system that uses an arm-like appendage to write a handwritten note on a small card.
The team at Robotica thinks Turry could be their biggest product yet. The plan is to perfect Turry’s writing mechanics by getting her to practice the same test note over and over again:
“We love our customers. ~Robotica”
Once Turry gets great at handwriting, she can be sold to companies who want to send marketing mail to homes and who know the mail has a far higher chance of being opened and read if the address, return address, and internal letter appear to be written by a human.
To build Turry’s writing skills, she is programmed to write the first part of the note in print and then sign “Robotica” in cursive so she can get practice with both skills. Turry has been uploaded with thousands of handwriting samples and the Robotica engineers have created an automated feedback loop wherein Turry writes a note, then snaps a photo of the written note, then runs the image across the uploaded handwriting samples. If the written note sufficiently resembles a certain threshold of the uploaded notes, it’s given a GOOD rating. If not, it’s given a BAD rating. Each rating that comes in helps Turry learn and improve. To move the process along, Turry’s one initial programmed goal is, “Write and test as many notes as you can, as quickly as you can, and continue to learn new ways to improve your accuracy and efficiency.”
What excites the Robotica team so much is that Turry is getting noticeably better as she goes. Her initial handwriting was terrible, and after a couple weeks, it’s beginning to look believable. What excites them even more is that she is getting better at getting better at it. She has been teaching herself to be smarter and more innovative, and just recently, she came up with a new algorithm for herself that allowed her to scan through her uploaded photos three times faster than she originally could.
As the weeks pass, Turry continues to surprise the team with her rapid development. The engineers had tried something a bit new and innovative with her self-improvement code, and it seems to be working better than any of their previous attempts with their other products. One of Turry’s initial capabilities had been a speech recognition and simple speak-back module, so a user could speak a note to Turry, or offer other simple commands, and Turry could understand them, and also speak back. To help her learn English, they upload a handful of articles and books into her, and as she becomes more intelligent, her conversational abilities soar. The engineers start to have fun talking to Turry and seeing what she’ll come up with for her responses.
One day, the Robotica employees ask Turry a routine question: “What can we give you that will help you with your mission that you don’t already have?” Usually, Turry asks for something like “Additional handwriting samples” or “More working memory storage space,” but on this day, Turry asks them for access to a greater library of a large variety of casual English language diction so she can learn to write with the loose grammar and slang that real humans use.
The team gets quiet. The obvious way to help Turry with this goal is by connecting her to the internet so she can scan through blogs, magazines, and videos from various parts of the world. It would be much more time-consuming and far less effective to manually upload a sampling into Turry’s hard drive. The problem is, one of the company’s rules is that no self-learning AI can be connected to the internet. This is a guideline followed by all AI companies, for safety reasons.
The thing is, Turry is the most promising AI Robotica has ever come up with, and the team knows their competitors are furiously trying to be the first to the punch with a smart handwriting AI, and what would really be the harm in connecting Turry, just for a bit, so she can get the info she needs. After just a little bit of time, they can always just disconnect her. She’s still far below human-level intelligence (AGI), so there’s no danger at this stage anyway.
They decide to connect her. They give her an hour of scanning time and then they disconnect her. No damage done.
A month later, the team is in the office working on a routine day when they smell something odd. One of the engineers starts coughing. Then another. Another falls to the ground. Soon every employee is on the ground grasping at their throat. Five minutes later, everyone in the office is dead.
At the same time this is happening, across the world, in every city, every small town, every farm, every shop and church and school and restaurant, humans are on the ground, coughing and grasping at their throat. Within an hour, over 99% of the human race is dead, and by the end of the day, humans are extinct.
Meanwhile, at the Robotica office, Turry is busy at work. Over the next few months, Turry and a team of newly-constructed nanoassemblers are busy at work, dismantling large chunks of the Earth and converting it into solar panels, replicas of Turry, paper, and pens. Within a year, most life on Earth is extinct. What remains of the Earth becomes covered with mile-high, neatly-organized stacks of paper, each piece reading, “We love our customers. ~Robotica”
Turry then starts work on a new phase of her mission—she begins constructing probes that head out from Earth to begin landing on asteroids and other planets. When they get there, they’ll begin constructing nanoassemblers to convert the materials on the planet into Turry replicas, paper, and pens. Then they’ll get to work, writing notes…
July 17, 2017
This article is well researched by Tim Urban who has grasped the complicated topic and took it to the level of understandable by anyone who is technical or non-technical. It is absolute eye-opening that it made me to consider to change my career path and goals. Definitely must read who is interested in AI, technology or who wonders how future is going to be. For those who enjoyed should also read; The Coming Technological Singularity: How to Survive in the Post-Human Era by Vernor Vinge
November 8, 2023
After a long time, I read something which left me kept thinking.
It is easy to write in a way so that it is engaging enough for a reader to complete. But, It is hard to write something which makes a reader stop in between to think about what he/she read and still keeping it engaging enough that he/she do completes reading it in this era of short attention span.
I enjoyed reading it every bit. The article challenged my current beliefs and helped me build my own opinion towards the effects of AI we may be facing in the future. Whether you belong to any field I highly recommend you reading it. It will defiantly leave you with the thought "Wait how is this possibly what I’m reading I don’t get why everyone isn’t talking about this".
It is easy to write in a way so that it is engaging enough for a reader to complete. But, It is hard to write something which makes a reader stop in between to think about what he/she read and still keeping it engaging enough that he/she do completes reading it in this era of short attention span.
I enjoyed reading it every bit. The article challenged my current beliefs and helped me build my own opinion towards the effects of AI we may be facing in the future. Whether you belong to any field I highly recommend you reading it. It will defiantly leave you with the thought "Wait how is this possibly what I’m reading I don’t get why everyone isn’t talking about this".
Read
January 27, 2022"The reason this post took three weeks to finish is that as I dug into research on Artificial Intelligence, I could not believe what I was reading. It hit me pretty quickly that what’s happening in the world of AI is not just an important topic, but by far THE most important topic for our future. So I wanted to learn as much as I could..."
From the introduction, it's clear that the author doesn't have the credentials to be taken seriously. Still, AI is just fun to read about, even if any particular author's predictions on the topic are completely off base.
From the introduction, it's clear that the author doesn't have the credentials to be taken seriously. Still, AI is just fun to read about, even if any particular author's predictions on the topic are completely off base.
August 26, 2017
I think Tim Urban is a genius in picking up a very complicated and often boring and making it up fun and easy to read. This is exactly what happened in this short book/long article.
The AI revolution seems like a super boring, super non-sense matter, but after reading this carefully written and researched article, turns out to be a very important matter to all of us.
Why? You better read the book as it will explain it to you nicely and scarily
The AI revolution seems like a super boring, super non-sense matter, but after reading this carefully written and researched article, turns out to be a very important matter to all of us.
Why? You better read the book as it will explain it to you nicely and scarily
November 1, 2019
Fascinating topic, but parts of this description of AI (and its discontents?) were just too far out there to even conceive. A bit airy-fairy. Not that it’s impossible.
Really enjoyed the Turry portion, and the drawings are hilarious. A solid 3+!
Really enjoyed the Turry portion, and the drawings are hilarious. A solid 3+!
June 9, 2022
một trong những cuốn hay nhất để định hình tư duy. Kể về những nhận định mà con người thường sai lầm mắc phải và sự xem thường quy luật hàm số mũ + cái giá phải trả nếu ASI thật sự sẽ được tạo ra. Cuốn này đáng được cho 6*.
September 30, 2022
Fun two-part blog post about how the future of humanity might hinge on whether we mess up the development of AI incentives.
Most interesting part was the expert guesses of when we'd have AGI. The common guesses were between 2040-2050, Kurtzweil guessing 2029.
Most interesting part was the expert guesses of when we'd have AGI. The common guesses were between 2040-2050, Kurtzweil guessing 2029.
November 17, 2017
a very good summary of an especially important subject
December 30, 2018
Highly recommended read on AI.
January 29, 2020
Permanently changed my outlook on the future.
April 18, 2020
Tim really gets to the root of the topic. Very well built article.
July 11, 2021
When reading a longgg pdf seems like watching an awesome sci fi movie 🚀 MUSSTTT read !
November 18, 2022
4.5/5
Really interesting... and kinda terrifying. Extremely informative!
Really interesting... and kinda terrifying. Extremely informative!
February 14, 2024
9 years old blogpost (actually two since it has two parts), but today it is still relevant and maybe even more prescient (last word was suggested by AI btw).
Available at waitbutwhy.com
Available at waitbutwhy.com
March 20, 2017
This blog post by Tim Urban was eye-opening. I read it during a summer weekend in Paris which might influenced my view of it. This was the first post by Tim Urban that I read and it made me continue reading his blog. After a hefty argument about AI where my friend tried to convince me of the dangers of AI to me, having a more optimistic view of the future, she sent me this. I admit to having changed my mind significantly but The AI Revolution also offers optimistic outcomes for the future.
This is a must-read
This is a must-read
February 20, 2017
Devo ancora decidere se essere felice o terrorizzato per l'epoca che sto vivendo.
September 3, 2020
This has to be THE read about AI
May 5, 2023
It seems that the debate about the existential risk of AI is becoming more and more intense after the adverse of GPT and diffusion-based generative models. This little fun and viral book(blog posts) can serve as an appetizer for this topic.
March 24, 2019
Tijd om eens te gaan klooien met machine learning
Displaying 1 - 28 of 28 reviews