Auditing AI

by The Marquand House Collective

How tech companies, journalists, and policymakers can prevent AI decision-making from going wrong.

Our lives are increasingly governed by automated systems influencing everything from medical care to policing to employment opportunities, but researchers and investigative journalists have proven that AI systems regularly get things wrong.

Auditing AI is a first-of-its-kind exploration of why and how to audit artificial intelligence systems. It offers a simple roadmap for using AI audits to make product and policy changes that benefit companies and the public alike. The book aims to convince readers that AI systems should be subject to robust audits to protect all of us from the dangers of these systems. Readers will come away with an understanding of what an AI audit is, why AI audits are important, key components of an audit that follows best practices, how to interpret an audit, and the available choices to act on an audit’s results.

The book is organized around canonical from AI-powered drones mistakenly targeting civilians in conflict areas to false arrests triggered by facial recognition systems that misidentified people with dark skin tones to HR hiring software that prefers men. It explains these definitive cases of AI decision-making gone wrong and then highlights specific audits that have led to concrete changes in government policy and corporate practice.

The Marquand House Marc Aidinoff, Lena Armstrong, Esha Bhandari, Ellery Roberts Biddle, Motahhare Eslami, Karrie Karahalios, Nate Matias, Danaé Metaxa, Alondra Nelson, Christian Sandvig, and Kristen Vaccaro.

Genres: Artificial Intelligence

204 pages, Paperback

Published April 21, 2026

Reviews

[Lord Bywrong · Rating 3 out of 5]

Auditing a transformer is not auditing a credit-scoring rule, and a panel where over half the room comes from policy and journalism is going to write a book that treats AI systems primarily as objects of governance rather than as engineered artifacts with specific failure modes, which is exactly what the publisher description suggests it does. A lawyer, historian or a journalist on the team is useful for explaining audits to the public, but you would not put a humanities expert on the team designing the audit methodology for a frontier model, and the book's framing slightly elides that difference. Calling them all "experts in AI auditing" on equal footing is marketing, not an accurate description of who is qualified to do what. Overall, an okay intro, but doesn't cover the depth it should and that I expected from the forefront names that were selling it.

[Lucas · Rating 3 out of 5]

The term "AI" in this book refers to any algorithm or recommendation engine, not always modern LLMs as we know the term today. It covers technology audits that have happened and provides interesting stories that I found myself looking more into. The book is very surface level, would have liked to dive into LLM specifics more. A quick read if your interested on how technology companies have previously been held accountable for the actions of their "algorithms".