What do you think?
Rate this book
Anti-Yudkowsky: Toward Harmony with Machines
What does it mean to be rational, or to use reason? The Rationalists are those who believe that reason is the source of invention and growth, that one’s ideas can be trusted the closer one hews to the dictates of reason, that reason is what will solve the AI Alignment problem and thus be the savior of man.
We confess to having a rather different view. on how things work. Yudkowsky’s ideas, in the context of the larger development of Western thought, can strike one as of a simpler time. We think fondly back to the “Age of Reason”, or another term for the Enlightenment, a period in thought which is said to begin in the late 17th century and ended in the early 19th, in which reason was felt to be sufficient to solve all of man’s problems. Yudkowsky’s Rationalists soldier on, unaware that many have declared that this period of optimism has closed[...]
So, since Yudkowsky has not done this for us himself, let’s perhaps try to unpack the intellectual assumptions of the project. We can maybe do this by looking at a related word to “rationality”: intelligence. This is all-important, as it is precisely artificial intelligence, artificial superintelligence, that we are told to expect and fear.
Unfortunately, this is not defined very well either. The standard definition we are given of “intelligence” in AI research — given by John McCarthy, an originator of the term “artificial intelligence” — is “the ability to accomplish one's goals”. Really? This does not line up to the way anyone we know uses this word. We believe the word these people are thinking of is power.
Within this strange definition lies the heart of the project. This is the equation of Rationalism: intelligence = power, a stronger claim than that of Francis Bacon for it refers to a latent, innate quality rather than something earned and won and produced.
We confess to having a rather different view. on how things work. Yudkowsky’s ideas, in the context of the larger development of Western thought, can strike one as of a simpler time. We think fondly back to the “Age of Reason”, or another term for the Enlightenment, a period in thought which is said to begin in the late 17th century and ended in the early 19th, in which reason was felt to be sufficient to solve all of man’s problems. Yudkowsky’s Rationalists soldier on, unaware that many have declared that this period of optimism has closed[...]
So, since Yudkowsky has not done this for us himself, let’s perhaps try to unpack the intellectual assumptions of the project. We can maybe do this by looking at a related word to “rationality”: intelligence. This is all-important, as it is precisely artificial intelligence, artificial superintelligence, that we are told to expect and fear.
Unfortunately, this is not defined very well either. The standard definition we are given of “intelligence” in AI research — given by John McCarthy, an originator of the term “artificial intelligence” — is “the ability to accomplish one's goals”. Really? This does not line up to the way anyone we know uses this word. We believe the word these people are thinking of is power.
Within this strange definition lies the heart of the project. This is the equation of Rationalism: intelligence = power, a stronger claim than that of Francis Bacon for it refers to a latent, innate quality rather than something earned and won and produced.
249 pages, ebook
Published June 14, 2023
3 people are currently reading
5 people want to read
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 of 1 review
April 7, 2024
this just spiraled into too much chaos for me. in a humorously harmonious way, tho
Displaying 1 of 1 review