What do you think?
Rate this book


336 pages, ebook
First published September 30, 2021
persuade humanity, rapidly and at scale, to become loving and wise enough that the AIs learn benevolence from usthen I do not think that is a plan. It is a hope.
therefore, the best we can hope for is RLHF and similar techniques inside the big AI companies.That feels like too sharp a collapse. The realistic middle ground is not “everyone becomes loving parents” versus “a small professional priesthood does RLHF.” There are several layers between those extremes.
We cannot make humanity loving in a few years. But we may still be able to make parts of the AI-development ecosystem more responsible, and those parts may matter disproportionately. The aim should not be universal moral awakening, but the creation of strong institutions, norms, incentives, and examples that make responsible AI development more likely than irresponsible AI development.This is still a tall order. But it is not pure utopianism. And it also avoids over-trusting the big AI companies. They have a crucial role, but they are structurally conflicted: they are trying to build safe systems while also competing for market share, investment, talent, strategic advantage, and prestige. Delegating “parenthood” to them alone is not enough.