Literary Theory for Robots: How Computers Learned to Write

Norton Shorts

by Dennis Yi Tenen

Chatbots are sure to have a significant impact on the way we read, write, and think. For better or worse, they are being used to find information, influence public opinion, diagnose illness, and shape political discussion online. How did we get to this point, and what can we do to prepare?



Literary Theory for Robots reveals the hidden history of modern machine intelligence, taking readers on a spellbinding journey from medieval Arabic philosophy to visions of a universal language, past Hollywood fiction factories and missile defense systems trained on Russian folktales. In this provocative reflection on the shared pasts of literature and computer science, former Microsoft engineer and professor of comparative literature Dennis Yi Tenen provides crucial context for recent developments in AI, which holds important lessons for the future of human living with smart technology.

Genres: Nonfiction, Science, Writing, History, Technology, Artificial Intelligence, Computer Science

163 pages, Kindle Edition

Published February 6, 2024

About the author

Dennis Yi Tenen is an associate professor of English and comparative literature at Columbia University. Originally a software engineer at Microsoft, Tenen is now an affiliate at Columbia’s Data Science Institute. He lives in New York City.

Reviews

[Steve Donoghue]

Former Microsoft wonk Dennis Yi Tenen here whips up an entry for the "Norton Shorts" series about all the things artificial intelligence likes to do in its spare time, the things it dreams about, the kinds of stuff it likes to read - all the kinds of exaggerations, wild overstatements, and low-rent science fiction that responsible tech-specialists spend lots of time correcting or tamping down. It certainly makes for entertaining reading. My review is here:
https://openlettersreview.com/posts/l...

[Gerhard · Rating 5 out of 5]

'Thus, while some scholars receive funding to train bots based on “all published scientific literature,” others, located at the epistemic periphery, volunteer to scan documents by hand, creating one of the world’s largest public libraries online. There, “universal” intelligence—some of the things ever written—finally joins the list of global energy resources, to be fought over, colonized, extracted, sold, bought, pirated, liberated, and exhausted.'

Review to follow.

[Mohan Vemulapalli · Rating 5 out of 5]

It is a truism, to say that machine intelligence is anything but. Yet, In Literary Theory for Robots" Dennis Yi Tenen" confronts a deeper, and perhaps less known truth about AI. That is, in essence, that AI is based on human efforts and experience. As such, history matters and understanding AI often comes down to understanding the solutions that were crafted for past problems. The resulting book is a deeply insightful exploration of how and why AI came to be that reaches far back into recorded history. This book will not reveal the ghost in the machine but it will show the traces of past human endeavors that reside at the core of any AI solution. Readers with a broad range of technical backgrounds who want to understand the roots of the current generative text revolution and other AI developments should find this book appealing.

Thanks to NetGalley and the publisher, W. W. Norton & Company, for providing me with an eARC in exchange for my honest review.

[Eric · Rating 3 out of 5]

Really interesting and insightful historical context around various forms of text generation leading up to the development of LLMs, but the writing style is unnecessarily complex and an absolute chore to wade through. Legitimately would have been 5 stars if not for the fact that I ended up having to keep going back to reread whole pages (and sometimes even whole chapters) to make sure I understood what the author was getting at.

[Branwen Sedai *of the Brown Ajah* · Rating 4 out of 5]

There was a lot of this I didn't quite understand, but it was still very interesting!

[nina · Rating 2 out of 5]

the chronology behind the development of artificial intelligence was told in a novel, somewhat interesting way, and i appreciated the highlighting of forgotten, under-appreciated, and diverse perspectives.

some of tenen's conclusions i really liked, as i do think it is important to recognize the human in AI and how it can only exist in the ways it is created to. he was definitely losing me though with some of the later points and was ignoring arguments against expanded AI use maybe a bit too much for my taste. also, the humor in this book was really just not for me.

definitely not one of my favorite books that i have read for school.

[Ryan Berger · Rating 1 out of 5]

No author has ever lost me faster than Tenen does with the opening line to this book: "Robots love to read."

[skull-splittingly loud INCORRECT BUZZER noise]

Robots not only do not "like" to read, but they cannot be said to "read" at all no matter how far even the most dishonest huckster for AI in art is willing to stretch his definitions. Initially, I thought that this might be a setup by the author to later dispel the notion or add nuance to the attention-grabbing opener, but it comes in clumsy trickles if it comes at all and Tenen never really walks this factually, spiritually incorrect statement back.

My dislike for AI in art spaces is immense and I don't need much to be provoked. That said, my curiosity and desire to know my enemy is strong enough to want to know how these technologies and systems work instead of going based on the smell test. Any worthwhile artist can tell you why AI art is bad if it could be said to be art at all (which it broadly isn't). As one of the most vocal critics and skeptics of AI art, I felt it important to come armed with knowledge.

This book will not get you much closer to understanding these things. In fact, I think it might put you further away from your target goal.

The second thing that jumps out after the blatant falsehood off the top is that Tenen is not an especially effective or interesting writer. Even when I committed to staying locked in, he doesn't develop ideas well or satisfyingly and a lot of the tongue-in-cheek tone just comes off as annoying.

It seemed like Tenen might shine some insight into some key components in how artificial writing "happens" and I was intrigued by the historical analogs with ancient word-wheels as well as how Chomsky's famous linguistic/grammatical paradox fits into how an algorithm achieves coherent output. But the ideas tend to go nowhere and if there is any insight, it only gets halfway to explaining a slight of hand trick rather than any mention of what might make something art, let alone GOOD art.

Lacking in any spiritual reckoning for what this technology means and how it operates while simultaneously vague in how it describes how the sausage gets made. Don't waste your time.

[Chris Liberty · Rating 4 out of 5]

Dennis Yi Tenen's Literary Theory for Robots offers a refreshing perspective on AI. Instead of just discussing technical leaps, it explores the historical and philosophical foundations of machine learning. The book highlights the connection between human learning and AI's ability to learn from vast datasets, framing AI as an extension of human knowledge-seeking. While acknowledging its transformative potential, the book emphasizes that AI's knowledge is built on human culture and creativity. Tenen argues that AI should be seen as a collaborator, not a competitor, and urges us to build AI that leverages human experience.

[Maggie]

“Intelligence demands artifice. Webster's dictionary defines intelligence as the "skilled use of reason." Artifice itself stems from the Latin ars, signifying skilled work, and facere, meaning "to make." In other words, artificial intelligence just means "reason + skill." There are no hard boundaries here-only synergy, between the human mind and its extensions.” (Page 4)

Compelling and provocative little book, pointing out that the current iteration of “artificial intelligence” is but a step on a path of human intelligence that started centuries ago: “the roots of computer science [are] inextricably entwined with literary and linguistic concern”. Spell check in your computer’s word processor is artificial intelligence. A Jacquard loom is too. And AI is, at core, driven by people - people make the choices that tell the self driving car what to do, in a “mess of complicity”. It’s collective wisdom.

[Mo Holub · Rating 4 out of 5]

Yi Tenen offers a thorough and thoughtful analysis of the near-entire history of human inclination towards automation: we have always, for various reasons, sought out ways to automate labor, and the conversation around automating intellectual labor did not start just because ChatGPT can now generate your entire English essay "from scratch."

The connections made between literary movements and the industrialization of literature--in the form of templates, universal outlines, skeleton forms, etc.--is an especially interesting point to add to the discussion of art and writing created by AI tools vs by human labor. The growth of automation into what we see as the creative and academic spheres is inevitable, and the challenge will be adapting how we measure success and learning in this new world, and how we regulate the use of these evolving tools. Yi Tenen follows with the equally interesting comparison of factory-made products to AI-generated literature, and how the movement from hand-crafted to machine-crafted also served to make products more universally accessible, if not as unique and meaningful (meaning, as discussed, is not something a computer can really grasp). Just like story templates made writing things like screenplays more formulaic and easier to teach, automation in writing like spell check and sentence completion has made a standard of writing more accessible across the board.

While I'm sure a certain amount of the machine-language-specifics still went over my head, Yi Tenen's perspective is a fresh and more positive take on the coming AI revolution, and his cast of smart-furniture and their programmers is an entertaining history lesson. The future of automation belongs to more than just the tech bros and Silicon Valley: we are all, collectively and collaboratively and universally, building the path forward.

[Christina · Rating 3 out of 5]

I find it difficult to rate this book. I found some arguments really interesting and enlightening but the author's style was very confusing amd in the end it was not clear to me what the target audience was meant to be: too complicated for the naive (incl.me), too non-mathematical for the experts. Maybe students of Dennis Yi Tenen in pursuit of good grades?

[Keeks · Rating 4 out of 5]

Really light and easy way to understanding modern qualms with AI. Understanding how these systems learned and still very much rely on humanity (for inputting information to learn and for usage) was very reassuring!

[Carlos Eduardo Madureira Trufen · Rating 5 out of 5]

I wish someone had told me how history puts things into perspective/context as this author did. Really interesting read for AI aficionados.

[Amanda · Rating 5 out of 5]

A brief, engaging overview of how AI became what it is today with a lot of food for thought

[Alva · Rating 5 out of 5]

My favorite book on large language models and avant-garde poetics, so far.

[Andrew Shaffer]

Most AI books are written by technocratic bootlickers ready to roll over for SkyNet. I picked this up because Tenen is an English professor (although he is a former software engineer, too).

The historical chapters were interesting. When it drifted into theory, it was too technical for the general reader (myself included). And when generative AI was billed as “collaborative work,” I began to roll my eyes. For-profit corporations training AI software on writers’ work without permission isn’t “collaboration” in any legal or ethical sense.

[John Michael Stroh · Rating 4 out of 5]

I definitely learned something new from this book.

[Robin Perkins · Rating 2 out of 5]

I received this book as part of a Goodreads giveaway.

I struggled through this book. I did read the whole thing but I don’t think I understood very much of it. I believe that this book is more fit for an academic audience than a general one.

[Paige · Rating 2 out of 5]

There are a number of issues with this book, and to save myself the time and annoyance of revisiting all of them, I'll just highlight a few.

-In the last chapter/conclusion, the author states that students writing their papers with AI is simply the newest iteration of students writing papers with the help of search engines.

-He also says that AI is a "tool" for writers to use, on par with Microsoft Word's spell check and thesaurus features.

-I was also expecting a book about AI to reference the incredible power consumption and resulting harm to our climate that AI causes. I know this book had a pretty narrow focus on the rather large subject of AI, but it feels a little odd to me to not mention that at all in a whole book about AI.

-Again, in the final chapter of the book, the author says, "The value of labor diminishes with group size. And so brave heroes climb the Mount Everest of intelligence, while the unnamed Sherpas carry their luggage." I thought this was going to be a segue into the documented terrible working conditions underpaid people all over the world perform on behalf of AI chatbots.

But no. This quote, and indeed the book as a whole, never acknowledges the "hidden" workforce behind today's chatbots. (if you're curious, the quote above is actually about the sheer number of researchers/engineers/etc who work on AI and are "forgotten" or diminished in the process).

I know this book was written in 2022 (the author mentions it at one point), and maybe some of this knowledge wasn't available the way it is today. But as an author working on researching AI, climate effects and hints of who's really doing the work of chatbots didn't even come across your radar?

-And finally, I'd just like to point out that the first source used in this book (check the notes section and you'll see for yourself!) is HENRY KISSINGER.

[Artur Coelho · Rating 4 out of 5]

Um livro que nos recorda a importãncia das ciências sociais para melhor se compreender a tecnologia. Mergulha a fundo na história da tecnologia e da literatura para nos levar a perceber as reais capacidades dos LLMs. E quando digo a fundo, vai a primórdios inesperados, como as regras combinatórias místicas de Llul e outros sábios medievais, os primeiros a estruturar sistemas de associação matemática de ideias para encontrar novos significados. Daqui segue para a interpretação da linguagem, e das ideias, com operações estatísticas que permitem automatizar processos de pensamento. Entra no campo dos primórdios da computação com o trabalho de Babbage e Lovelace, e faz um desvio intrigante para a padronização de ações, mostrando como a literatura floresceu com essas técnicas.

Por padronização, entenda-se a organização de métodos de trabalho literário, entre meios de cruzar temas à criação de organigramas, procedimentos que permitem acelerar e, de certa forma, automatizar a criação escrita. Métodos que os escritores comericiais conhecem, e exploram bem.

Tudo isso conflui na nossa noção contemporânea de IA Generativa baseada em texto. Treinada no corpus textual, estruturada pela vectorização estatística de palavras, capaz de gerar textos complexos e convicentes, mas também incapaz de compreender o que gera. Os LLMs não são papagaios estocásticos, isso é uma simplificação extrema, mas a forma matemática como lidam com a linguagem não chega à consciência do seu sentido, o que explica as falhas que se convencionou apelidar de alucinações.

O livro não é alarmista em relação à IA Generativa, mas foge dos deslumbres que caracterizam muito do discurso sobre este tema. Mostra-nos que é uma tecnologia com imenso potencial, se entendida como ferramenta e não fim em si. E que talvez o seu maior risco seja a sua excessiva antropomorfização, que distorce a forma como a entendemos, bem como desresponsabiliza alguns dos maus usos.

[Thomas · Rating 4 out of 5]

Dennis Tenen is a former Microsoft engineer who now works in Columbia's Department of English and Comparative Literature. So when he tells us in Literary Theory for Robots, “Computers love to read,” it is not so much a lie, as one reviewer complained, as a metaphor. That is the kind of thing that happens to your writing style when you hang out with literature profs.
Tenen provides a brief history of the automation of textual analysis and generation. On the one hand, there is a romantic quest for a universal language machine, and on the other, we have the history of the automation of work from the Jacquard loom to the self-driving car. The interplay between these two historical trends helps explain how a “mere tool” can “move into the subject position of the sentence—where it detects, devises, and masters—gaining a sense of agency and interiority in the process.”
Tenen sees the chatbot as part of a historical process like the one that began when mass literacy moved reading and writing “from the upper to the lower bounds of intellectual work.” The chatbot's “skill” has more to do with teamwork and collective labor than the kind of interiority that breaks through the proverbial Chinese Wall.
Here is a question for curriculum designers. Do we need to teach first-year students how to write paragraphs with topic sentences when a program on their laptop can do it with such ease? Is English composition now, like Latin in the nineteenth century, more a hobby than a necessity for an educated person?

[krispy · Rating 3 out of 5]

oops totally forgot to rate and review when i actually finished listening to this book. as per usual not a totally honest rating because i am not great at paying attention to audiobooks and this one is particularly esoteric. the three stars really just mean i am pretty ambivalent. this felt like a more condensed, more philosophical, and thus more difficult to understand version of another audiobook i recently listened to, the information: a history, a theory, a flood, with the added benefit of a bit more hindsight. i think it’s probably time for me to stop listening to audiobooks on these topics lol because it is all starting to blend together.

i don’t know why reading the title and a blurb of this book in a bookstore made me excitedly think this would be about the actual literary quality of the writing produced by large language models, but i feel like this is more about the process than the results. anyway no one steal my idea but i do think it would be fun to explore the actual writing habits and style that these models amalgamate and output and what that says about the data that they have been provided and how it might be used. and i guess what the ramifications of that are on the profession of writing (whether journalism, non-fiction, copy, literary, comedy, academic, translation, etc.) and on creativity in general.

[Jen Clancey · Rating 5 out of 5]

Yi Tenen establishes early on that the product of human output—a piece of writing, a garment, really anything created by humans— is the result of a human’s interaction with tool. This book filled me with awe. He redefined what intelligence could mean beyond a person’s ability to merely do smart things. He redefines too, what a tool can be. It is a book, a search engine, a laptop, a pen. And tools are the result of eons of human intelligence compounded. Beyond this, we must consider that we live lives collaborating with other minds shaped and guided by their own interactions with tools and their own systems for doing so. It was also interesting to learn the history of artificial intelligence and to see his above ideas illustrated through those stories from divination circles to coding of written prose (Vladimir Propp). This will shape how I approach my own judgement of my work, my interaction with community, and the boundaries around it. Incredible.

I appreciated deeply his notes about culpability in artificial intelligence. There are teams behind studies and of course applications of AI that assign the technology—a tool dreamy of and built by human thought since antiquity— undue responsibility for harm.

[Steven · Rating 3 out of 5]

The description made it sound interesting, but somehow the book itself failed to impress me. Perhaps the murkiness of its overall purpose or message—that AI is not mysterious or magical or sentient, and that it’s a culmination of a long history of helpful language tools—doesn’t really stir the imagination.

I listened to the audiobook version while driving, which might have lessened my appreciation. Perhaps reading it in printed form would have allowed me to digest its meaning better. The text skipped around a lot, from idea to idea, making it difficult to follow.

The second half of the book made more sense than the first. The author’s explanation of how AI communicates to us in words takes some of the mystery out of the process. Based on computations of Markov Chains, AIs don’t seem to “think” in the same ways humans do, if you can call their process “thinking” at all. What they do seems more like advanced word association games. If true, it would seem that fears of AI taking over the world and dominating humans seem, at the least, premature.

Therein might lie the main value of the book. It should lessen your fears of AI.

[Rob Brock · Rating 4 out of 5]

This was a fun little book about what it means when we say that AI is generating or creating something new. The pulls examples from history about how writers have pursued algorithmic shortcuts for centuries, and suggests that what is happening with both the excitement and fear we feel in response to OpenAI's ChatGPT 4 is in line with this history. He also demonstrates that what is actually being generated is not anything uniquely new, in the way we think about human creativity. Instead, what is generated in response to our prompts is a statistical estimation of what we want to see based on the body of prior human knowledge available on the internet. I found this book, written from the perspective of an English professor rather than a pure computer scientist, to be a helpful framework for understanding the ways that we can work with AI technologies while holding on to our most human creativity.

[Maria · Rating 3 out of 5]

I would say this is a broad introduction to literary theory for robots. The cover artwork is misleading because I was expecting a book geared towards children, but it's much more advanced. There are many fascinating topics presented here, but it's a little difficult to follow most of the time. I listened to the audiobook which was narrated well, and I used the hardcopy to study the diagrams and various tables. The section that was easiest to digest was about writing in the 19th century and the creation of a demand-driven supply of pulp fiction. Everything else was interesting but a little jumbled with topics weaving into each other then popping out to return later. I'd have to reread the book several times carefully, make notations and go off down each rabbit hole to get a clearer understanding of this expansive study.

[John · Rating 3 out of 5]

Tenen’s book turns over some good questions. He shows how easy it is to start talking about AI like it is dreaming or hallucinating, when really it is just running the long relay of choices people made. The metaphors make it feel alive. They also make it harder to point a finger at who built what and why.

Still, I kept waiting for the book to do what the title said. I thought I would get the story of how computers learned to write. Instead, the pages wandered through old ideas and mechanical dreams, sketches of what people thought machine writing might be. Some parts caught the light, but it was not the story I came looking for.

Even so, a few of the questions he leaves behind keep working after you close the book. I would like to see him take another stab specifically a book about writing and creativity in AI.

[Molly Willis · Rating 3 out of 5]

I listened to this as an audiobook on Spotify and I really wish I'd bought the physical copy instead.

A lot of this book is one long history lesson on computing, with some lesser-known tidbits about how it crosses over with literature. I think I glossed over the huge volume of information being relayed by listening to it rather than reading it.

There were a few chapters where I felt the point could have been made more saliently or succinctly. The meat of each chapter is in the last few pages, everything else is sometimes pre-amble that doesn't feel necessary to make the resulting point.

It's at its most interesting around Chapters 4-5, in my opinion.

[Mattschratz · Rating 4 out of 5]

The historic parts of this functioned really well, and gave a good account of, historically, what a unit of language is. I don't think I am as sanguine as the author about what this history amounts to. He thinks that language will be resilient because it's not shaped like the shallow vectors that MML uses to make text. But....the other possibility is that people will start just talking and writing like that--that they could skate along on a web of words that kind of go together and not think more rigorously about them. That seems like a live one to me!

[Tara · Rating 2 out of 5]

I had an extremely tricky time getting through this book… despite a try excitement when I picked it up at our local bookstore.

It jumped around too much to make sense to my mind and way of mental processing. I kept waiting for a thread to tie together a variety of random threads.

I did appreciate the invitation to consider the historic journey towards automation and how this relates to both language and culture. The conclusion which covers 9 summarizing principles was the most clear chapter for my brain to interpret.