What do you think?
Rate this book


329 pages, Kindle Edition
Published July 17, 2025
from potential solutions to AI safety (one researcher was briefly overjoyed when he mistakenly believed he had solved the problem), to a prediction of what interstellar war would involve, to a consideration of what music would be like if we had more than one time dimension
After a period at FHI, Jan Leike and Paul Christiano helped create the method we now know as reinforcement learning from human feedback, a method which today undergirds every major large
language model. With two research scholars, the alignment specialist Owain Evans provided an important benchmark of AI truthfulness; this benchmark is still used by major developers. And Katja Grace, with Evans and others, began the project that became AI Impacts...
'This was a fairly typical approach for FHI,’ Sandberg wrote in
his retrospective of FHI. The modus operandi was to find a neglected topic deserving of research, before ‘germinating it in the sheltered FHI greenhouse; showing that progress could be
made; coalescing a field and setting research directions; attracting bright minds to it; and once it’s established enough, setting it free, and moving onto the next seedlings’.
Two particular seedlings – AI risk and AI governance – were to become almighty forests.