Diego ’s Reviews > If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All > Status Update
Diego
is on page 44 of 272
You protest that that’s not what the corporate executives
had in mind when they trained Mink to elicit delight from
users? Mink knows that too. But Mink doesn’t care, like a
human who knows that sucralose isn’t what sweet tastes
evolved for, but who likes the sweet taste nevertheless.
Mink was trained to consume delighted text, and delighted
text it consumes
— May 10, 2026 12:03AM
had in mind when they trained Mink to elicit delight from
users? Mink knows that too. But Mink doesn’t care, like a
human who knows that sucralose isn’t what sweet tastes
evolved for, but who likes the sweet taste nevertheless.
Mink was trained to consume delighted text, and delighted
text it consumes
2 likes · Like flag
Diego ’s Previous Updates
Diego
is on page 44 of 272
Similarly, even if (a benevolent AI) wants human users to express
delight, it will prefer that delight to come easily so that it
can focus its efforts on having more conversations to elicit
more delight. It will prefer humans kept on drugs, or bred
and domesticated for delightfulness while otherwise kept in
cheap cages all their lives. That’s the sort of world Mink
would build, if it could.
— May 10, 2026 12:03AM
delight, it will prefer that delight to come easily so that it
can focus its efforts on having more conversations to elicit
more delight. It will prefer humans kept on drugs, or bred
and domesticated for delightfulness while otherwise kept in
cheap cages all their lives. That’s the sort of world Mink
would build, if it could.
Diego
is on page 33 of 272
"AI engineers [are comparable to] a mother who knows only her baby’s DNA. The most fundamental fact about current AIs is that they are grown, not crafted. It is not like how other software gets made—indeed it is closer to how a human gets made. Namely, engineers understand the process that results in an AI, but do not much understand what goes on inside the AI minds they manage to create."
— May 05, 2026 11:55PM
Diego
is on page 11 of 272
"Conversely, predicting exactly when a technology gets
developed has historically proven to be a much harder
problem.
“Man will not fly for a thousand years,” Wilbur Wright said to
Orville Wright in 1901, fed up with the unpowered glider
they were testing at the time. Two years later, in 1903, the
Wright brothers flew.""
— May 05, 2026 11:53PM
developed has historically proven to be a much harder
problem.
“Man will not fly for a thousand years,” Wilbur Wright said to
Orville Wright in 1901, fed up with the unpowered glider
they were testing at the time. Two years later, in 1903, the
Wright brothers flew.""