A GLITCHING AI. A MYSTERIOUS NAME. A DISCOVERY TOO STRANGE TO IGNORE.
When mathematician Matthew Watkins signed on to test an early version of ChatGPT, he expected dry technical work. Instead, he uncovered a series of "rare tokens" that made the system go haywire. Some produced gibberish. Others turned hostile. One trigger word - petertodd - sent the system into what looked like full-blown paranoia. Was it a simple bug? A backdoor planted by hackers? Or a glimpse into how AI really thinks?
Watkins's search for answers pulls him deep into the secretive world of AI safety research - where billionaires play god, rogue chatbots make fortunes and enemies, and the line between science and belief blurs. What starts as a programming mystery becomes something much both a window into the eerie inner worlds of LLMs, and a cryptic clue to what might be history's greatest heist.
The system has since been patched. The window is closed. But for a brief moment, we saw inside - and what looked back at us was maybe more human than we'd like to admit.
Matthew Watkins completed a PhD in mathematics in 1994, but has always been more interested in trying to understand what mathematics "is" and "where it comes from" (as well as trying to explain it to his non-mathematical friends) than pursuing a conventional research or teaching career.
The second half of the 1990s were spent living as a nomadic musician (he plays the saz, a seven-stringed Turkish instrument), contemplating the underlying nature of reality while wandering the British Isles, busking, picking fruit, planting trees, visiting megalithic sites, etc. The music continues.
In 1999 he had a little maths and physics reference book published (also illustrated by Matt Tweed) as part of the popular Wooden Books series. This has since been licensed by Walker & Co., NYC and last time he checked, it had been translated into at least half a dozen languages.
Since 2000, he's been an Honorary Fellow in Exeter University's mathematics department. During this time, as well as having done a bit of teaching work, he initiated and has been since been curating the online Number Theory and Physics Archive and the related (but more popularly accessible) site Inexplicable Secrets of Creation, a project which naturally led to the idea of this series of books.
In 2004 he, together with playwright and mythogeographer Phil Smith, received the first arts commission from the British Association for the Advancement of Science in its 173-year history. This took place in Exeter and was a suprisingly well-received combination of a local history walk, a mathematics lecture and a piece of experimental street theatre, based on the fascinating life and work of local 19th century mathematical visionary William Clifford.
More recently, he's been showing up at various festivals (The Secret Garden Party, Big Green Gathering, etc.) as his alter ego, the (even more) eccentric mathematician Professor Raphael Appleblossom, doing "freestyle walkabout mathematics performance lecturing", honing his ability to communicate challenging mathematical ideas to highly non-mathematical (yet curious) members of the public.
In March 2010, he participated in an experimental collaborative venture with sculptor Conrad Shawcross at Urbanomic Studio, a centre of interdisciplinary arts/science/philosophy research in Falmouth, UK.
I didn't expect it to draw me in as strongly as it did. A charmingly personal and engaging exploration of the intelligence of large language models and the culture of the AI research community. Solidly built on a mathematical foundation while being easily accessible. I'm starting to understand how human knowledge can be captured by a multi-layer network of sufficient dimensionality. Matthew has written a wonderful book.
Cannot recommend this book enough, I've devoured it in 4 days. I've laughed out loud, I've shouted wtf out loud numerous times. I feel like I've been given such a deep and well guided tour of the under belly of this hard to grasp ai moment - with dry wit, heart felt humanity and care. Well done Matt Watkins, this is a blinder
A great step by step review of both how LLM ai is built and the attempts to keep it aligned! Who could have guessed all the intrigued and behind the scenes action!