Longlisted for the 2024 Financial Times Book of the Year. How life and the economy became a black box—a collection of systems no one understands, producing outcomes no one likes.
Passengers get bumped from flights. Phone menus disconnect. Automated financial trades produce market collapse. Of all the challenges in modern life, some of the most vexing come from our relationships with a large system does us wrong, and there’s nothing we can do about it.
The problem, economist Dan Davies shows, is accountability sinks: systems in which decisions are delegated to a complex rule book or set of standard procedures, making it impossible to identify the source of mistakes when they happen. In our increasingly unhuman world—lives dominated by algorithms, artificial intelligence, and large organizations—these accountability sinks produce more than just aggravation. They make life and economy unknowable—a black box for no reason.
In The Unaccountability Machine, Davies lays bare how markets, institutions, and even governments systematically generate outcomes that no one—not even those involved in making them—seems to want. Since the earliest days of the computer age, theorists have foreseen the dangers of complex systems without personal accountability. In response, British business scholar Stafford Beer developed an accountability-first approach to management called “cybernetics,” which might have taken off had his biggest client (the Chilean government) not fallen to a bloody coup in 1973.
With his signature blend of economic and journalistic rigor, Davies examines what’s gone wrong since Beer, including what might have been had the world embraced cybernetics when it had the chance. The Unaccountability Machine is a revelatory and resonant account of how modern life became predisposed to dysfunction.
Stafford Beer is a figure who evokes a certain slightly mystifying reverence from a very specific kind of technically inclined, leftish sort of thinker. A management consultant and writer cashing in on the big overhyped tech panacea of his own day ("cybernetics"), he was lent an aura of tragic heroism due to his role setting up for the Allende government in Chile the abortive Project Cybersyn, which, depending on who you ask, was either the first real attempt at putting into practice a distributed computational socialism before it was crushed by the forces of neoliberalism or a network of fax machines connected to a center office with some projectors and some very swanky chairs. This story provides a dramatic moment of betrayal that serves, in this historical narrative, as a symbolic starting point for the global rise of neoliberalism.
This book takes that framing as a jumping off point for espousing several theories of features of modern business and government that Davies describes in terms borrowed from Beer. The main point, about "Accountability," per the title, is that rules-based systems have the dual purpose of ensuring reliable system performance and also of shielding from accountability of the participants in those systems, who can deflect blame to the rules, possibly passing it along in a chain to the point that nobody at all can be made accountable to deal with the consequences of those rules. Examples are given from the everyday, to the humorous, to the global and political. This idea is used to make sense of certain institutions whose main point is to deflect blame, including management consulting and for-profit academic publishing. Modern AI systems are mentioned, but mainly as just a continuation of the trend. Befitting of a book about setting up systems in which no individual can ever be blamed for the outcome, it doesn't dwell heavily on who is responsible for this trend, though it highlights Milton Friedman's writings on the (absence of) social responsibility of business as exemplifying it, and really excoriates the Journal of Applied Corporate Finance, a funny bit of intra-business school sniping from a book about management science. In the broadest reach, the idea is used to explain the rise of populism as the remaining inchoate reaction available to a citizenry who have been denied, through increasing delegation of life-affecting decisions to the politically unaccountable and decentralized "market," any ability to seek direct redress for their grievances. The book goes into more detail about why such structures, contra the usual pro-market argument, can destroy rather than propagate information, bringing in cybernetic diagrams and control-theoretic jargon that mostly amounts to "you should have as many points of control as there are free variables to control," and concludes with the semi-quixotic call for a world with more middle managers, who can facilitate this multiway communication.
I'll be honest here and say that while many of the examples rang true (and will likely do so for anyone who has ever been shuffled between customer service reps, none of whom is allowed by policy to help you or is even capable of doing so), I'm not entirely sure how well it hangs together as a grand theory of 21st century dysfunction. But Davies is a very fun writer and it's a quick and thought provoking read, so I'd recommend it anyway.
Probably the best book I have read this year and probably the only one that was better than the hype around it (in the community of London-based policy professionals).
Dan Davies applies the theories of management cybernetics to the current polycrisis and develops the theory of what he calls ‘Accountability Sinks’ – the tendency of large complex systems to develop mechanisms to diminish any sense of personal responsibility on the side of individual managers/operators. Most of the theory comes from the five systems within the Viable Systems Model (VSM), developed by the British management consultant (from times when that meant something very different) Stafford Beer.
Stafford Beer is most famous for – well, in leftist circles at least – his work on the Cybersyn project in Allende’s Chile. That story is briefly mentioned but does not constitute too much of the narrative. Rather, Davies builds on Beer’s second edition of Brain of the Firm from 1981, which is informed by Beer’s experiences in Chile.
The book begins with more anecdotal descriptions of systems where a lack of accountability or rigid structures led to horrible or absurd outcomes – like when KLM shredded 440 squirrels at the Schiphol airport due to a lack of paperwork on the side of the transport company. No single individual made a mistake and the company followed the government regulation, which made sense in itself – there was just no sense of management of exceptions to the broad, general rules.
But later, the book dives into the broader analyses of the limits of the current approach to governance of large complex systems, using the theories of Stafford Beer and management cybernetics. There are a few chapters that are quite theoretical that analyse the five systems within complex systems (1-operations, 2-resource allocation and coordination, 3-tactics, 4-strategy within the external environment and 5-decision-making and identity within the system).
Davies frequently mentions that these parts are dense and hard to follow, but I did not find them to be such – rather, he uses useful anecdotes on the nature of complexity, and when he finally gets to the use of the concept of variety (the number of states the system can be in) by Ross Ashby, it makes a great deal of sense. All that is used to deepen the understanding of how complex systems work, punctuated by frequent use of the phrase ‘The purpose of a system is what it does’ or POSIWID. While this is a fairly common phrase in system thinking and theory (I have even included it in papers I have written), its application in this analysis is very useful and brings this into the mainstream understanding of inefficiencies in bureaucracies.
Cybernetics is a science of complex feedback mechanisms. When a system consists of two parts, the feedback between these two agents provides complete and usable information about the system. However, as the number of agents increases, the number of circuits between nodes multiplies, making it impossible to understand the combinatorial outcomes. In such cases, knowing the properties of individual connections no longer offers a true understanding of the system as a whole. Complex systems, therefore, must be comprehended in their entirety, or they cannot be understood at all. In fact, having deep knowledge of only part of a complex system can be more dangerous than complete ignorance. This is seen for instance in failures in military intelligence, which often stem from over-focusing on specific parts of a complex situation – think The Best and the Brightest scenarios.
At the corporate or policy level, the inability to manage complexity often leads to the rise of these "Accountability Sinks." This is where decision-making becomes reduced to rigid rules—rules that are publicly stated, seemingly defensible, and follow a transparent process. However, such rules disrupt feedback loops within systems, create issues with no clear owners, and fail to address edge cases effectively.
Every "rule" represents an implicit model of the world, but it is a model that must inherently omit significant amounts of information about the real world. It carries no intentionality; instead, it functions as a network of cause and effect—a system that renders certain outcomes inevitable.
Later parts of the book analyse the growth of economics as a ‘system five’ function within modern societal and economic structures. This is now a fairly common critique, but what is quite novel is Davies’ thorough critique of business studies as a discipline and business schools as institutions, and especially his ridiculing demolition of the Journal of Applied Corporate Finance, which he calls ‘one of the most explicitly and thoroughly ideological publications [he] has ever read – including those printed by the Soviet of Chinese Communist Party’. Especially Davies’ critique of the modern approach to accounting costs across time and business units so that it fulfils the objectives of the management is quite neat and insightful.
A large part of Davies’ analysis is the fall of middle management and its replacement with outsourced brainpower in the management consulting industry. This is implied to be driven by a desire to cut overhead costs (which are difficult to allocate), with the goal of the pursuit of maximising the shareholder value (which used to be something to be satisfied, not maximised). This cut out large parts of the variety of engineering functions from large organisations, enabling the rapid spread of the ‘Accountability Sinks’ in the past decades.
The final chapter and prescriptions of the book are probably its poorest part – basically, Davies calls for the end of leverage buy-outs (which saddle the bought-out companies with debt from the purchase) and restraining of the predatory practices of the private equity industry. This seems wholly inadequate to what he was describing throughout the book and is a bit of a disappointment, which nevertheless does not diminish the quality of the analysis from the book as a whole.
I do not think the book provides ultimate answers for the current state of the world, but it definitely asks some important and right questions – about the identity-correcting systems within our global system, about the nature of business management and its relationship with academic economics or about the nature of modern corporations and large bureaucracies. As such, it thus needs to be read by both people from the policy world (where I imagine it will have the greatest impact) and also by people in private corporations, which could benefit from more systems thinking-management cybernetics and less rigid economics.
The book has a great premise, starts well but peters out into superficial platitudes.
The author identifies a growing trend in large organisations to replace personal accountability with processes, meaning that no one is accountable for mistakes: the examples are vivid and blackly amusing (especially squirrel shredding at Schiphol Airport). This is accompanied by an introduction to cybernetics, which was new to this reader, and the writings of Stafford Beer. So far so good.
Unfortunately, the rest of the book is about the shortcomings of accounting and economics, and the baleful effects of shareholder value, management consultants and private equity. This is well-trodden ground and doesn’t help us reach a satisfying conclusion. Instead, the author points at the things that have gone wrong and eventually admits that cybernetics doesn’t shed much light on possible solutions.
An amazing and intellectually VERY invigorating book! Davies analyzes the roots of the 2008/9 financial crisis in a fairly original way—through cybernetics and information theory. Essentially, he argues that that liberal democratic governments + large corporations have essentially structured themselves so that (a) they are not capable of processing information about risks and negative outcomes; and (b) there is no one you can hold accountable when that lack of information leads to economic and ethical crises—the system is designed such that horrible outcomes occur, regardless of the people involved.
(a) has occurred, Davies argues, because the rise of economics as THE discipline for modeling and understanding the world tends to flatten a fairly complex/information-rich world into a simpler one that fits with economic modeling approaches. That, combined with neoliberalism (Davies makes the very intriguing and unexpected argument that neoliberalism, with its exclusive focus on profits and its assumption that the market contains all the intelligence you need to interpret the world, essentially acts as an information-reducing filter) means that large corporations, central banks, and governments have all become increasingly incapable of handling the complexity of contemporary life and political/economic systems.
There’s a quite interesting information-theoretic-ish argument at the end as to why so many democracies have seen the rise of demagogue ethnonationalist politicians (like Trump in the US, Modi in India)—people are unhappy with the status quo politically, but the channel they’re given to communicate this information to politicians is basically just elections, which can’t carry that many bits of information (especially when you think about the US’s two-party electoral system using first-past-the-post). So aggrieved voters may end up voting for a candidate like Trump to express overall discontent with the status quo and a desire to remake it.
Very interesting book with lots of unexpected and original arguments (I liked how Davies characterized the function of management consulting especially) and really good clear explanations of complex things (cybernetics history, the history of private equity leveraged buyouts and how it’s contributed to economic instability).
I thought the title and cover was publisher’s poorman solution to market another popular science book but the inside didn’t disappoint. Beginning sections read like a tribute to Stafford Beer. However Davies does a good job explaining Beer’s management cybernetics against the Friedman Doctrine of free competition through maximizing shareholder values. Critics to Friedman is not new, especially on lack of social responsibility or lack of accountability causing harm to workers or lower income class as seen in many financial crises. Cybernetics doesn’t seem to offer any grand theory either; maybe because I don’t understand it well yet. It is a great systems thinking method so I will go back to original sources with Beer’s Brain of the Firm or Designing Freedom. I might as well reread Friedman’s Free to Choose and Capitalism and Freedom. I thought I got these sorted out when I first read some of these whey back when. I guess it’s time to refresh some mental models. For now noting POSIWID as main takeaway/ reminder: the purpose of a system is a what it does.
“Consider the example of a young person going down a YouTube rabbit hole. There’s a human intelligence taking in the content. There’s also an algorithmic system at work, showing them one video after another. You wouldn’t necessarily call the algorithm an ‘intelligence’, although it does seem to do things with a purpose. But then there’s a third entity in the background – the company that owns YouTube, and the structure of cause and effect that brought the other two things together. There’s no one person, or group of people, making the decision about what videos our hypothetical young person is being shown. In fact, the executives in charge of the parent company are sometimes horrified and distraught at the decisions made – in 2017, there was a scandal at YouTube when it was discovered that people were producing parody cartoons featuring beloved characters burning down houses or undergoing painful dentistry, and that these were being shown more often to innocent children than to the ironic adult consumers that were their intended target market. But somewhere, at some point in time, it’s been decided that ‘engagement’ is the purpose of the system – what it does – and somewhere else, a set of decisions have been made about what methods are going to be used to achieve that purpose.”
“The post-war mathematical optimisers proved that it was possible to show that an abstract representation of a market equilibrium, for instance, had some properties which made it difficult to improve upon. And from an entirely different direction, the concept of ‘market efficiency’ in financial economics predicted theoretically, and then established empirically, that it was very difficult to beat the stock market and so in some sense, the market was more ‘intelligent’ even than investment bankers. Together, these things contributed to a sense that the market economy was a kind of computing pond; it organically grew optimal solutions to problems and, like a true artificial intelligence, you got out more than you put in. It’s somewhat ironic that economics, the discipline that’s meant to tell you that ‘there’s no such thing as a free lunch’, ended up believing so strongly in an informational version of the same thing.”
“If the economy is an information-processing system, does that mean that every corporation is an artificial intelligence? If people are worried about out-of-control AI taking over the world and destroying everything, shouldn’t we have been trying to do something about them at least seventy years ago – and probably more like two hundred?”
“In the future, ‘I blame the system’ is something we will have to get used to saying, and meaning it literally.”
This starts out with an important idea--The Purpose Of The System Is What It Does. Unfortunately, it then veers off on assorted abstract tangents like squirrel thermostats, and never gets back to meaningful evidence of how this cybernetics stuff actually helps.
The first story in the book is about the hero trying to help patients when he didn't know the cause of their illnesses. The approach taken here is to just consider the problem/patient a black box, not worry about what's inside the black box, and then try stuff to see what makes the person better. Yes. Even if the doctor has some fancy tests to order and fancy treatments to prescribe that operate on tiny little parts inside the body, what matters is still whether or not the person gets well.
Similarly, the proof of cybernetics cannot be found in pretty equations or sophisticated diagrams but in the ways it has worked to make systems function better.
Я думаю это было бы гораздо лучше в форме эссе (я посмотрел и кажется нет, книга не построена на его основе).
Автор определяет концепцию accountability sink — возникающих внутри организаций систем правил или протоколов которым (не всегда предумышленно) делегируется принятие настоящих решений. И я думаю что это интересная и наверное даже в определённой мере полезная метафора для рассмотрения функционирования сложных систем, в том числе в профессиональных контекстах. Если книга закончилась бы где-то на пятидесятой странице, то всё было бы великолепно.
Но дальше автор признается что обожает Талеба и, как бы сказать, местами это определенно угадывается (если вы понимаете то понимаете).
Ещё чуть потом книга вообще уходит в рассказы о управлении экономикой по Стаффорду Биру, что может быть и интересно, но имеет уже достаточно опосредованное отношение к тезису с которого всё начиналось.
If you're already familiar with cybernetic theories, then there's nothing new for you here. I always wanted to learn more about it but it was never a priority for me, so this book gave me a nice entry point.
There are fascinating observations the author made that are very relevant to how we might think about the challenges in large orgs and institutions (more often than not, it's not a bunch of bad people with evil intentions end up doing bad things; instead it's a bunch of good people with good intentions end up doing bad things). It's also not hard to make connections between that notion and what Lawrence Lessig calls "institutional corruption".
Key concepts like "accountability sink", "Richardian vice" and cybernetics' viable system model are very useful thinking tools for us to think about how large orgs work, and more specifically, how people in orgs approach emergent things like design, innovation, transformation, etc. (yup, that's my context of relevance.)
The comparison between complex org and AI is both profoundly insightful and thought-provoking.
If you're only interested in borrowing good thinking tools to think about your own topical interest in the relevant context (e.g. accountability, oversight, organizational model), then Part 1-3 would suffice.
Taken as a broader rumination on the society and its challenges, this book is a nice addition to your list of thought-provoking books, alongside other gems like The Management Myth or any David Graeber book.
Feel like this could have been written with much greater clarity - didn't convince me that much of the particular usefulness of cybernetics as a discipline and many of its more salient points were either obvious or a bit tenuous
Honestly, 2024 was the year that I had sort of given up on reading pop sci books, and this redeemed the genre, so bear this in mind for my star rating. This book is not without its flaws, but let me start with the good.
First, without even trying, it explained governance to a general audience better than most books on governance. Second, it explained management cybernetics in a clear, concise way, without all the jargon nor all of the dramatic story lines that Davies could have easily lent on to move the story along. Third, it did a very good job situating economics and accounting with respect to cybernetics, and again doing so in a way that didn't buy into the narrative that economics is a proper science. Davies is very candid about the flaws of both cybernetics and economics, and that's refreshing from an academic. His critique of management studies is spot on, and explains that weird feeling I had when I started delving into the organisational studies literature during my PhD. I did so to help me understand organisational norms, culture, and decision making processes, but found even my university library held a huge collection of nonsense books. There was a whole section I learned to avoid, and it's of the kind he discusses in this book.
As with many books that attempt to weave together academics theory and popular events, there were some issues. The one that bothered me the most is perhaps the most common in pop sci books: it sort of falls apart in the middle in terms of keeping up the narrative of reviving the lost lessons of cybernetics. Davies gets too lost in the weeds explaining the basics of economics and accounting, and for much of the rest of the book (until the final few chapters) the link back to Management cybernetics is pretty weak and at many points non existent, which is disappointing since this is the stated 'thread' that is meant to run through the book. And the book also is light on solutions to the very real issues it outlines, though the analysis is sensible.
Ultimately I am left with the sense that it shocks me that the governance of social-ecological systems literature and this literature don't seem to talk to each other, but also a sense that I want to dive deeper. I love digging up old ideas from the past and thinking about how they might help us understand how to improve environmental governance, and this gave me a lot of ideas for where there are missing connections and useful concepts that could help in this regard.
Ultimately, this is one of my favourite books of the year, in part because it actually made me think, and it's been a very long time since a pop sci book had done that. I don't think Davies overstates his conclusions or proposes he has some grand unified theory of anything here, so I am also not taking it as a book that has changed my life. But it will send me down a rabbit hole.
Interesting. It gives more sources to continue a more thorough investigation. The foundation stems from Operations Research, which given my background intrigues me. I have found that I have started to think about organizations as information flows in my day to day life and that has changed some of my perspective.
This book seems to be very popular at the moment, at least in some tech circles. It was very interesting and I learned a lot about finance driven capitalism. Academics discuss capitalism a lot, but I guess the mechanics and ideological changes since Marx remain often unexplored, unmentioned or just assumed to be known. At least, *I* was unfamiliar with how leveraged buyouts work and what motivates them ideologically. I would have liked to know more and thus a minor criticism of the book is that it did spend few time with such concepts (maybe the assumed audience is more familiar with leveraged buyouts and how bookkeeping works?); another flaw (at least aesthetically) is that the book covers many concepts that then do not wrap up in a satisfying way. It is a bit as if a movie would plants several arcs that then gets dissolved quickly. The author seems to be aware of it and writes something along the lines of wanting to write a thriller about the clash of cybernetic planning with neoliberal ideology but learning along the way that it needed far too much build-up to work well as such. Nevertheless, the book is enjoyable, I learned a lot of and it cites many interesting resources to learn more about what might have been covered too briefly.
I read the first chapter or so and skimmed parts of the rest. Interesting, but I found it packed with anecdotes and examples when I would rather it just cut to the chase. In other words, more conversational and therefore much longer than I would have liked.
That being said, I know my preferences for this kind of educational/informative nonfiction writing are in the minority, and I can already think of at least one person for whom this will be a great Christmas gift.
I’m already sure this will be my favorite book of the year. Many great quips and while it could have been better structured, the ideas are so powerful it’s worth a read. Please also read and let’s discuss!
The Unaccountability Machine attempts a grand project of diagnosing and curing the malaise of the 21st century: namely, everything is getting worse and it seems like nobody can do nothing about it. Davies' theoretical approach is to explain and use Stafford Beer's cybernetic management theories against Milton Friedman's neoliberal doctrine that the primary duty of the firm is to maximize shareholder value. I'm not sure it quite works, but it's a brave attempt. The initiating idea is one of accountability sinks. Think about getting bumped from a flight due to a system outage and complaining to the desk agent. It's not like they can do anything to help. They're just a face there to insulate the airline from customers who can no longer make their connections. Information should be flowing, but it instead it's stopped.
The first thing is explaining Stafford Beer and cybernetic management. Beer was one of those mid-century British prodigies, going from philosophy to the Army in WW2 to psychology to operations research to management consultant to public intellectual. His enduring contributions were two books, Brain Of The Firm and The Heart of the Enterprise, the acronym POSIWID (the purpose of a system is what it does), and a consulting hand in Chile's Project Cybersyn, a centrally planned economic operations center that was overtaken by Pinochet's coup.
Project Cybersyn
The key parts of management cybernetics, according to Davies, is that any system managing another system must have sufficient variety to handle the kinds of information coming from the managed system. Secondly, according to Beer's Viable System Model all systems have five interacting components: operations, coordination, regulation, intelligence, and identity. I'll confess that even as someone with a sympathy towards cybernetics, I'm not full convinced. The phrase "sufficient variety" conceals the complexities of what organizations are paying attention to, and applying the viable system model is to a real enterprise is far from easy. With cybernetics, I'm reminded of Douglas Adams' observation about astrology or tarot, in that it doesn't really matter what the rules are (even pure nonsense can work), as long as there are sufficiently many of them in enough fineness to reveal the shape of whatever lies beneath.
But taking cybernetic seriously, the problem is one of economic management, namely how to match inputs and outputs across the billions of human preferences that make up the economy. Davies is an economist, but a critical one, who regards the '-ist' as exemplifying an ideology rather than academic rigor. Economic modeling is powerful, but makes dangerous simplifying assumptions. To whit, individual preferences are atomic, and the effects of complexity in systems can be ignored. While economics is extremely powerful in a policy role, it is also shockingly bad at dealing with things like real accounting practices as done by businesses, time, and debt.
Economic as a discipline fostered a series of moves which represented the triumph of the capitalist class (those who own things) over the managerial class (those who make decisions), with the working class already placed on the ash-heap of history. These moves are most strongly identified with Milton Friendman, and the doctrine that a company's primary and indeed sole duty is to maximize shareholder returns. All sources of information get crushed down into a single number, the stock price, which floats on the whims of speculators. Further, the practice of the leveraged buyout means that companies are forced to ignore everything that isn't generating cashflow to service increasingly high debt obligations. And the neoliberal wisdom of the market means that there's no one to appeal to: this is just how the world works now.
I'm not sure that Davies' advice to adjust the information and incentives is sufficient. Power speaks, and it's no accident that capitalism won and forced the working classes to absorb all shocks to the system. But he offers an insightful and novel prescription, like some new drug extracted from a rare and ancient flower deep in the rainforest.
I read this book because Rory Sutherland gave it a ringing endorsement in his Masterclass talk. Every book he’s recommended so far has been worthwhile. This one, unfortunately, was not. It’s an interesting thesis: It’s about the industrialization of decision-making. It’s also got a lot of systems thinking throughout, such as this:
"The decline in individual accountability for unpopular decisions is not – or not only – a form of moral decline on the part of our rulers. …Nearly all the commands and constraints which afflict the modern individual, the decisions which used to be made by identifiable rulers and bosses, are now the result of systems and processes. …every year, more of the decisions that affect our lives are made not by people but by systems."
This lack of accountability the author calls “accountability sinks.” The book is an attempt to explain—and revive?––cybernetics, which is “the study of decision-making systems. Key figures in the early days were Norbert Wiener, John von Neumann, and Claude Shannon, who are far more famous for computing (and information theory) than cybernetics. He explores how mainstream economics replaced the management theory of cybernetics, which I suppose would have produced better outcomes. I’m not persuaded.
I agree with the author that how economists have ignored information theory is striking. However, there is one exception, and the author never mentions him: George Gilder, especially his book, Knowledge and Power, Life After Google, and Life After Capitalism. The omission is glaring, because Gilder has laid out a new information theory of the economy.
I don’t agree with his attacks on “shareholder capitalism.” Seems to me that is an incredibly accountable system: you are accountable to shareholders, not the vague concept of “stakeholders.”
He introduces Stafford Beer, a management consultant and cybernetician. His uncomfortable axiom: "The purpose of a system is what it does (POSIWID).” Ok, but systems thinking already knows this. Cybernetics was a big deal in the old Soviet Union’s economic departments. Didn’t seem to work too well there. Was that simply because back then they didn’t have access to the computing power of today? I doubt it. He does cite Hayek, and the Austrian school on the information problem. He’s right, economics ignored information theory, which he says is close to the cybernetic concept. But Gilder didn’t ignore it, and used it to create a new theory of economics.
All in all, this book was a bit of slog. There are some very interesting points, but the glaring gap of not acknowledging Gilder’s work lessens its impact. I’d love the author to explore Gilder’s work, and add a chapter or two to a future edition of this book.
Unfortunately I can't say other than that this book was a thorough disappointment for me. I became aware of it when it was referenced in an article in The Economist and sought it out, expecting to receive a thoughtful analysis of how organizations of all stripes avoid accountability and systems slide towards directionless. What I got was a lengthy, frankly a little excruciating and unneeded explanation of cybernetics, such as it is, followed by a rehash of some common complaints about economics and the consulting industry. The book is almost all theory - and questionable theory at the - with the few real world examples feeling forced. Ultimately, I didn't feel that the book did what it says on the tin and what was inside wasn't the least bit attractive to me. I also found the author's tone to sometimes be a little overfamiliar, so there's that as well.
I really liked this book, although it's a little scattered. I appreciated the description of "management cybernetics", and his account of how Milton Friedman's principles of profit maximization for companies, and debt as a tool to create that focus, were both elegantly simple and absolutely destructive of the ability for leaders to choose more humane and less myopic possibilities. I think the cybernetics were actually a bit of a distraction from the latter story, as it's interesting history but I never really understood his application of "variety engineering" and other cybernetics concepts. But I appreciated the call to create companies that no longer maximize profit, and instead choose other more human means and ends.
The book aims to provide a grand theory of why so many of the large systems that make up modern life have became dysfunctional lately. To do so we re going some pretty abstracts concepts from ‘cybernetics’ a rather obscure discipline and without any convincing demonstration of how these theories have actually prove useful in the past. There is a large dose of rather shallow critique of economic theory, accounting and corporate management. The most insightful part is the description of how accountability sinks are created within corporations to avoid the need for discretion and mid-management decision. Overall, quite a fun book and the questions raised are super important even if the answer remains unsatisfactory.
It could have been a 2-3 star at most, but ended up being yet another book trying to make absurd connections between events (and use it to propagate a hidden agenda, I’d risk to say). Takes unaccountability of systems (which exist) to drive a rant against shareholder value, Milton Friedman, management consultants and private equity. And, the cherry on top, it claims that the systems’ unaccountability led to the rise of populist movements… If you think so, and act accordingly by avoiding the underlying reasons, then expect that very same populism to keep growing. And no, I’m not arguing in favor of that populism, if that’s not clear.
I think I would have gotten a whole lot more out of this book if I had a non-zero level of familiarity with economics. Still, it was fascinating and perturbing—an indictment of private equity and of Milton Friedman and free market capitalism as the runaway train leading to our current time. I guess it’s always seemed intuitive to me that maximizing growth forever vs. moving towards equilibrium was an insane concept but I thought maybe I was missing something given the aforementioned lack of economics training… but yeah no this is what gives us squirrels being massacred in woodchippers and crashy 737s and horrible health insurance that’s also expensive and the destruction of the American heartland and the rest of it.
Anyway this is the second time in two weeks I’ve had the lesson that the government is not a business and that we should not want it to be (also a theme on diabolical lies podcast). I do think we are at the beginning of a transition to post-capitalism. Whether that looks like socialism or like our annihilation though is tbd.
Jennifer Pahlka, author of Recoding America, recommended this book in her Substack, Eating Policy ("In business, culture eats strategy. In government, culture eats policy").
For more than a decade, I've been trying to describe something I keep noticing about the organizations and bureaucracies I work in and with... I gestured at these concepts with phrases like, "a conspiracy without conspirators" and "technologies of abdication" (I'm sure I've used these phrases in book reviews here and there). Occasionally I'd ask people better read than I am if they'd ever encountered the idea I was trying to convey and, if so, whether there were words I could search to find books on the matter. I mostly got blank stares.
And then, Jennifer Pahlka recommends a book in her newsletter, I look it up, et voila: the very ideas I've been trying to find names for this whole time. Turns out what I've been looking for is Cybernetics.
Reading Davies is like reading Lewis Hyde... by the end of the book, you have acquired a new lens through which to look at the world. I'm not sure I can do anything much with this new perspective, but it sure is sparkly.
By the end of the book, Davies' recommendation, "Businesses ought to be like artists, not paperclip maximisers," seems both brilliant and practical.
I love a book that makes me evaluate my worldview, and while I don't agree with everything Davies had to say, that clearly wasn't the point. This read more like a fun dinner party conversation than someone trying to make me believe anything specific.
study of complex systems, economics and global governance, feedback systems, etc...
**TLDR**
Systems evolve as a natural forces, their intentions are only to be derived from the empirical fact of what they end up doing.
Individuals try to leverage systems to accomplish goals because they need to if they want to reach a certain scale, but systems grow a life of their own. They are artificial entities.
Note: this makes scaling things hard. Your system will eat you.
They evolve in ways that often trend towards the rejection of individual accountability, accountability sinks, because:
- you create "policies" that become less and less attached to individuals or motives with time, and you need these to deal with general cases without overexerting the agents of your system, but this removes individual responsibility - policies can be easier to handle than individual decision making - people don't want to be responsible, courage is scarce and systems allow you to remove blame
This book argues for a science of understanding these systems and their feedback mechanisms. It presents context on cybernetics - the study of decision making systems, from *kubernētikos*, greek for "good at steering" (origin of cyber!), interprets recent events from this perspective and describes how society creates systems that have direction and what we can make of that. To me this is very relevant to future development in AI.
It presents frameworks for understanding sources of control and motion in systems, and decomposing their control functions and sources of life.
How can we approach these kinds of complex systems and give them life?
It presents the claim that our current approaches are flailing
- it attacks econ on a few of its core methodological approaches, and claims modeling also acts as an acc sink - by allowing numbers to stand in place of lived experiences - Expands on the identity creating/replacing function of neoliberalism in economic history - And argues management has lost its role and power in modern society, in part due to this
Calls for a science of the sytems of the future. One that tries to integrate feedback systems on longer timescales, pushes away from neoliberal cultures to allow corporations to have more broadly defined purposes, and takes into account cybernetic thought.
This book is also nice because it has a lots of examples of systems fucking themselves up which I find cool. I had not realized the extent to which this system misalignment is cooking us.
I think it presents some important context and I appreciated reading it, even though sometimes it is unrigorous and tries to apply cybernetics everywhere, a bit too much imo.
## Systems approach and the system of the world - part 1
Takes seriously the idea of treating systems as their own force of nature. Do not ascribe purpose to the system, all you can do is ascribe purpose to individuals. The purpose of a system is what it does. Do not assume it will let you steer it unless you have very careful mechanisms.
Cool example: criminogenic organisations that are structured so the system drifts towards fraud without necessary individual responsibility.
Sometimes individuals create systems that take blame. For eg in capitalistic structures you can use policies and detachment between levels of hierarchy so that the people at the top avoid responsibility for issues.
- Aside: presents a story for recent populist rise: - we were rich for a while and the economy was growing, people lost interest in political governance because it was less key - hosuing crisis, other austerity -> elites can't give us a good life - people started paying attention to politics again without having been clued in, annoyed at elites, bam
Frustration and systemic issues cause a desire for blame that needs to be pushed out, even if the system has no head. Claims the blame was pushed onto the whole of society and the existing democratic complex.
Accountability sinks can be nice for certain people but they mean loss of human control with bad consequences.
light claim: extent to which you can make a decision/steer a system is extent to which you are responsible for its actions. Problem: people avoid responsibility, people lose control.
I resonate with this framing that people want their agency taken away from them and that the system is one of the great ways to do that. I think this pattern pops up a lot. The world lacks true courage.
Aside: says the university publishing situation and how expensive it is, is because it is an accountability sink for prestige systems and citation counts that allows the academic system to control hiring and organize itself by creating a layer that decreases social tension between colleagues all working and assigning credit in the same field. Is this true? It was interesting.
Interpreting systems in terms of scapegoat functions as their reason for existing is interesting. Argues the IMF is also just doing this sometimes for elites in poor countries, creating local disalienation.
This notion of system design and systems as constructed sinks for certain things really interests me.
---
Stafford Beer, one of the fathers of cybernetics. Inspired by *Cybernetics* by Norbert Wiener -> I should read this #todo#. In the time where cybernetics was starting the whole mathematical framework that could be useful was only being created.
Note: this book gave me a bit more context on the whole claimed history of the economic system of the world. Also markets are really interesting, Hayek/Shannon ideas of markets as communication sources for information.
When you see how something works and you are part of it then you are more inclined to feel responsible for it.
Smart people can join systems and then do bad things because you can get put in a structure and need to deal with its demands from the position you can join it from no matter how bad it is.
## pathologies of the system - part 2
Argues cybernetics got overruled by computing.
Control tip: the variety of possible states somewhere in a system should be in accordance with the flexibility of the control mechanism.
Each black box needs a regulatory mechanism at each level of hierarchy. Are there sources of variation in the environment not explained by the control mechanism?
Proposes a hierarchy of system functions: ( I don't get all the separations)
1. operations, at the ground 2. low level regulation 3. optimisation or integration, direct management to coordinate individual activities 4. intelligence, planning, internalizing info that might not be seen at the operations, predicting for future structural changes 5. philosophy or identity, self creation.
Difference 2 and 3 is 2 is about conflict management and 3 is about achieving a purpose.
I am not sure how useful this hierarchy is. 5 is the most interesting to me. The self creation ability and intrinsic purpose of a system is just one of its potential forces, that can allow you to reduce variety by having a source of consistency. Systems with a lot of 5 seem very effective, eg EA orgs, religions, etc...
Important aspect is how does information communicate between the levels. Here information is things that causally influence actions. Studying the communication structure and bandwitch sources is key, but I am not sure what to learn here on the abstract level.
Aside: decerebrate cats, didn't know that was possible. System that has lots its brain and paying to certain kinds of information. Eg financial system before 2008.
## Economics & systems - part 3
Economics is the study of human behavior when faced with scarce means and desired ends with different uses.
Thought: this part didn't feel super rigorous/well organized to me, and I could probably read a better critic of economic practice.
Is critical of economic practice and assumptions, in particular in cases where economic scientific results are then applied as realities in dubious cases to ill effect.
Says economics cannot do the following properly:
- handle crisises and hard to quantify goods/events, don't want to do an expectation over all possible futures - model irrational agents - Ricardian vice
Markets as computing fabric is always pretty but overused. Claims economics uses markets as an accountability sink for the difficulty of good modelling. And that then policy makers try to turn everything into market outcomes, and faith in the free market info signal. Argues we need to understand systems and work outside the cultural hegemony of economics, to touch grass.
question: how do you understand this tradeoff well?
Argues that the ricardian vice is too powerful. If you create a model, gather numbers, make a prediction, you naturally win against someone who hasn't done that because they don't have the methodology, even if they are living the thing on the ground. They don't have a system to fall back. And so optimization already becomes a decision making well, and this makes you lose touch with the actual lived state of the world.
Note: I like this kind of information misalignment. reminds me of the pchristiano stuff. Kind of arguing against forms of aggregation/fallbacks of empirical decision making in economics because inducing false degrees of belief.
The book is arguing that these kinds of patterns are responsible for a lot of problems in society, management consulting, economic modelling fallbacks, private equity and accounting mispractice -> general laws and rules of the field get applied without context and destroy small systems to which it doesn't fit. Chesterton pilled. Consulting is just another accountability sink.
But I don't really see what the counter is? Like yeah be smarter model better, I guess look deeper and try to make sense of it all but I am not sure this abstract system stuff is truly applicable. I also don't know the managerial counter?
Shits on management science, says it has no identity creating function or way to link theory and practice. Why is cybernetics better exactly?
## part 4 - Looking towards the future
Interesting context on Friedman and neoliberalism. I did not realize the extent to which this seems to have shaped society. This is something I know way too little about.
Friedman makes claims about what happens when you let systems or agents try to allocate resources or things for stuff they don't really care about. Presents a historical case that in the 60s corporations were lacking a sense of identity or purpose, and that Friedman gave them exactly what they needed. A reason to not worry about that and simply maximize profit and trust the system. To me this seems so ingrained in a lot of modern capitalistic structures that it surprises me to hear that things were not this way? To what extent is he respnsible?
Basically made corporations maximize shareholder value when it had not been that way in the past? I guess yeah corporations used to have more identity creating mechanisms -- artisanat like france and Japan.
- private equity sharks is an interesting thing I did not know much about - ++ these loan deals - argues that this is possible because of lack of system 5 identity
- book argues that private equity + milton friedman identity made it so that companies undervalued management functions for short term gain and this is leading to more and more accountability sinks and bad outcomes
- alternative model, Jerome levy: - working class is at the center - investing class has a justified existence by the service it gives to the working class in terms of goods and wages - its function is to insure the working class against risk and to provide those services - this model was cooked by neoliberalism
Claim is that the post neoliberal economic management system removed vital management functions and swamped its functions with fear of debt and short term priorities. then this removed the ability to do longer term modeling and absorb volatility so the working class was made to absorb this.
Government functions also started taking up the worst of this pattern of externalizing labor
I The unaccountability machine beskriver Dan Davies en potentiell orsak till den växande frustrationen som individen upplever i interaktionen med samtiden. Sedan sent 70-tal och den nyliberala revolutionen har modeller och optimerade processer tillåtit allt fler beslut att fattas utan beslutsfattare och således utan personer som kan hållas ytterst ansvariga. Detta gäller både i interaktionen med multinationella företag, där policyer dikterar relationen till kunder, och i politiska system med liberala förtecken, där samma mekanismer dikterar relationen till medborgarna.
Skiftet från individer (med tacit knowledge) som fattar diskretionära beslut till system som implementerar optimerade processer ger upphov till det som Davies kallar en ”accountability sink". Kafkaeska situationer där besluten som föranledde fel inte går att kreditera en individ utan är en naturlig följd av en anonym process.
Det hade inte behövt vara så här. Huvuddelen av bokens första del ägnar Davies åt att presentera ett alternativt anslag, stöpt i en epistemisk ödmjukhet med vetskap om modellers brister och inneboende alienerande antihumanism. Management cybernetik gjorde anspråk på att konkurrera med den reduktionistiska ekvilibrium-ekonomin omedelbart efter andra världskrigets slut men ett par årtionden senare hade disciplinen fallit i glömska samtidigt som pojkarna på MIT och i Chicago belönades med Nobelpris och formulerade sanningarna som kom att definiera nästkommande 50 år.
Medan cybernetik tillät för en differentiering mellan det komplexa och det komplicerade systemet ignorerades den obekväma realiteten i den dominerande ekonomiska disciplinen genom förenklade antaganden om homo economicus, ekvilibrium och spelteori. När modellerna som baseras på dessa osanningar inte korresponderar med verkligheten, är verkligheten mer sannolik att falsifieras än modellen. Det är exakt det som sker när makroekonomer och prognosmakare förklarar att ”om allting hade fortlöpt så som modellen förutsatte hade modellens prognos varit riktig.” Prognoser är svåra att göra – särskilt om framtiden.
Systemets inneboende inhumanitet är enligt Davies det som driver samtiden till galenskap. Och det är svårt att se trenden upphöra i takt med att allt fler beslut automatiseras per artificiell intelligens och nyttomaximerande algoritmer. Detta är en anledning till varför jag personligen starkt ogillar sports analytics – jag helhjärtat avskyr artiklar av Michael Mauboussin och Richard Thaler om huruvida man ska gå för en fjärde down i amerikansk fotboll eller när man ska plocka ut målvakten i ishockey för att optimera. Efter att ha läst Davies är det omöjligt att inte betrakta fotbollens VAR som en accountability sink. Det är inte längre en enskild domare eller linjeman som ska stå till svars för sitt felaktiga beslut – ansvaret har ersatts av en optimeringsprocess som ingen, bortom den rådande tidsandan, kan sägas ha infört.
En förutsättning för att situationen, så som vi finner den, ska ha kunnat uppstå är aversionen mot att bygga system på en mänsklig skala och mantrat om optimering. Mycket av kritiken som Davies torgför mot det ekonomiska nyttomaximerande etablissemanget tangerar tankar associerade med figurer som Santa Fes Brian Arthur, LSEs Ole Peters, Max Planck-institutets Gerd Gigerenzer, Oxfords Iain McGilhrist och Nassim Taleb.
Det är svårt att inte, åtminstone delvis, koppla den beskrivna utvecklingen – premieringen av optimerade processer över robusthet och antifragilitet, viljan att ackumulera/koncentrera istället för att adressera risker – till den korresponderande ränteutvecklingen under samma period i kombination med en geopolitisk peace dividend (Sovjets kollaps, Kinas inträde på världsmarknaden). Det senare var sannolikt en förutsättning för det förra. Om det senaste halva seklets ökade globalism börjar rullas tillbaka, samtidigt som det växande missnöjet mot transnationella organisationer fortsätter växa – utfallet i senaste EU-valet – och riskerna som tillåtits ackumuleras under samma period börjar brisera, bör det finnas utrymme för en alternativ ansats att få genomslag.
En ansats som premierar problemlösning på lokal nivå, där beslutsfattare 'äter sin egen matlagning' och därför är mer benägna att överskatta risken för det osannolika. En ansats som möjliggör popperiansk felkorrektion och ersätter fokus på optimering med robusthet och humanitet – präglad av en epistemisk insikt om de ekonomiska modellernas oerhörda begränsningar och otillräcklighet.