Bad Data: How Governments, Politicians and the Rest of Us Get Misled by Numbers

by Georgina Sturge

'Essential reading ... An incisive and urgently needed book' Tim Harford

'[An] entertaining introduction to the uses (and misuses) of data ... a penetrating analysis of why statistical literacy matters to our politics and our daily lives' Professor Jonathan Portes


Our politicians make vital decisions and declarations every day that rely on official data. But should all statistics be trusted?

In BAD DATA, House of Commons Library statistician Georgina Sturge draws back the curtain on how governments of the past and present have been led astray by figures littered with inconsistency, guesswork and uncertainty.

Discover how a Hungarian businessman's bright idea caused half a million people to go missing from UK migration statistics. Find out why it's possible for two politicians to disagree over whether poverty has gone up or down, using the same official numbers, and for both to be right at the same time. And hear about how policies like ID cards, super-casinos and stopping ex-convicts from reoffending failed to live up to their promise because they were based on shaky data.

With stories that range from the troubling to the empowering to the downright absurd, BAD DATA reveals secrets from the usually closed-off world of policy-making. It also suggests how - once we understand the human story behind the numbers - we can make more informed choices about who to trust, and when.

Genres: Nonfiction, Politics, Society, Business, Science, Technology, Mathematics

299 pages, Kindle Edition

First published January 1, 2022

Reviews

[Callum's Column · Rating 3 out of 5]

My sister-in-law's parents live in China. Their visa applications were rejected twice by Australian immigration to attend her wedding to my brother. It was only after an immigration lawyer was involved that their applications were granted. Initial rejections were not explained, yet Georgina Sturge's book may provide an explanation. A UK citizen with Nigerian descent had family remaining in Nigeria. Their visas were also rejected when she wanted to have her wedding. Upon review, it was determined that they were rejected because they came from a 'suspect' country. They were automatically considered 'risky' no matter the evidence they provided. Bad data, or faulty algorithms, was the problem.

Data, according to Sturge, "simply means information and, in the context of this book, mainly refers to numerical data." Data is important, but understanding its limitations—e.g., narrow sample size, poor modelling, biased selection, etc—and acting accordingly is more important. Governments and politicians are too ready to utilise bad data for ideological ends. This can have deleterious consequences. For example, Ivermectin was approved and prescribed to treat Covid-19 based on ostensibly promising clinical trial results. On closer inspection, however, many of the trials were "seriously flawed or likely fraudulent".

Sturge critiques the methodological underpinnings of various forms of data like surveys. However, she uncritically utilises survey data later in the book to undermine another disputed piece of data. This is a curious discrepancy. Moreover, I was disappointed by the rudimentary nature of the book. Sturge is a statistician for the House of Commons, and it felt like she restrained her intellectual prowess to ensure a larger audience. Despite these criticisms, I agree with Sturge's concluding remarks: "If we are going to be governed by numbers, let's not live in a data dictatorship. Let's recognise that we are the ones in control."

[Sebastian Gebski · Rating 4 out of 5]

Yet another book on using the data the wrong way. So, doesn't it make sense to bother yourself with it? Actually, I think it does. Why so?

1. it's written by a professional who deals with data on a daily basis - not only with analyzing available data but in fact also with identifying, sanitizing and validating data; hard to find someone more credible
2. the author is British, and she works for UK Govt - FINALLY, a non-Yankee book that isn't oriented around the US; seriously, that made a tremendous difference; just consider one aspect: she's not looking for the most sensational stories, but rather the most impactful, meaningful or just the best-documented ones

So should you just run straight away to buy it and read it? Hold on, it's not that simple. In the end it was an interesting lecture and good entertainment, but are there any insights that will stay with you? Not really, the book doesn't have much purpose - you want learn nothing. Well, maybe except being a bit more cautious when reading some "analytical report" in media. This lack of actionability reduced the rating by 1 star. But if I could turn back time to face a choice: read it or not, I'd still choose to do so.

[Sophia Terry · Rating 5 out of 5]

This is a fantastic overview of explaining the chaos that is the British government. Explaining the flaws and unknown in why we only know what we know. It’s incredibly helpful in showing the downsides of our current data collecting processes. This book focuses on immigration, Brexit, and Covid so if any of those topic are interesting to you give this book a read!

[Paola Goggi · Rating 4 out of 5]

I absolutely loved the content, but it doesn't flow very well. Paragraphs are not well connected to each other, she keeps going back and forth on certain topics, and the rest is just a bunch of anecdotes sticked together in no particular order. I would have happily read another 100 pages if that meant she took her time contextualising why the examples she gives are relevant to the overall point she's trying to make in the chapter. It's all extremely interesting and insightful, it definitely deserves 4 stars. I just wish the book followed a more cohesive structure

[Elizabeth · Rating 5 out of 5]

Very well written - no need for specialist knowledge - and essential reading for journalists and anyone in local or central government. As well as the general public if we are to understand/call out key decisions.

[Kelsey Breseman · Rating 4 out of 5]

Really valuable check on how data methods work/common ways to overvalue statistics. Would love to see an adaptation of this for American audiences.

[Pynk Moon · Rating 2 out of 5]

Just kind of empty to be honest. There are a lot of questions asked and a lot of problems raised, but very few solutions put forward. I would never have bought this book for myself.

[Hassan Shah · Rating 4 out of 5]

This is a good book for anyone who wants to know how statistics and data affect us daily and how they affect the decisions around us. This book is good for people in a corporate setting or working with the government. It's perfect in the age of artificial intelligence. Significant decisions are made based on statistical models behind them without people knowing how these are driven and how bad or irrelevant data can influence the decisions taken by them or by the government. Compare this to some of the Books with catchy titles, hyperfocus, or deep work by renowned writers. These are baits for readers to buy but are disappointingly repetitive, full of the obvious, and incredibly dull to read. This book is a Gem, and I must say that the writer has done a far better job than Yuval Noah Harari and Cal Newport, for example.

However, while reading the interesting facts in the book, there is a tendency to repeat the obvious from time to time from a different angle and point of view. Sometimes, I skimmed through these details, and other times, I moved on to the next chapter, which is valid for most books. Only the legendary Piece of work can be the one that One reads every page of. It also seems like the writer is a classic example of an interviewee during discovery work in management consultancy or when developing a strategy. She has an excellent grip on the problems, But at the same time, the solutions are not convincing Enough. This is probably because not many people spend time-solving problems rather than pointing them out, and solving problems has always been the chunk of the issue. Solutions are the most challenging part of a problem statement, rather than pointing them out.

However, the book is a must-read for anyone who wants to challenge their problem-solving abilities and for all strategists and leaders, especially those in corporate or institutional settings; as a reader, I must say well done Georgina.

[Emily · Rating 3 out of 5]

Another rare non-fiction read for me, and another Backstory Balham recommendation! More than just a rec in fact, my first author talk, how fun! I only got about a 1/3 through the book in time for the talk but hey ho, it was accessible for those that had or hadn't read the book yet.

At the start of the talk Tom spoke about the value of authority in the non-fiction world: the authority an author has to talk about a topic, or why we should trust what they say. You can't get anyone more informed about the political consequences of bad data than a House of Commons statistician! Cool job, I wonder how she got into it in the first place. It was definitely an interesting read, and will make me think twice about every stat I read in the news, about how it might have been measured or collected and how much it should be trusted. Having said that, just because she has the authority to point out the downfalls of the data used, doesn't mean she has a solution to propose. At times it felt a bit repetitive with all the problems being brought to light without any suggestions or fixes.

I thought it was a bit odd that the Freedom of Information act was only introduced midway through the book, when it feels like that should be a relevant part of it the whole time. Interesting comments on the way coalition governments are formed here and how much easier/faster it is for bills to be passed in the UK, compared to elsewhere where it has to go through the two houses with different parties in power.

[Andrew Bishop · Rating 3 out of 5]

Sturge offers a compelling and critical examination of how various entities can manipulate and misrepresent data to influence public opinion and policy. Sturge, a statistician with extensive experience in data analysis, provides a meticulous analysis of the misuse of statistics in the political and corporate spheres.

The book is structured around real-world examples, showcasing how data is often selectively used, misinterpreted, or outright falsified to serve specific agendas. Sturge delves into instances where governments have presented skewed data to justify policies, or where corporations have exaggerated statistics to boost their market value. Her insights are particularly relevant in today's data-driven society, where numbers are often taken at face value without critical scrutiny.

Sturge's writing is clear and accessible, making complex statistical concepts understandable for a general audience. She emphasizes the importance of data literacy and encourages readers to question the sources and methodologies behind the numbers they encounter. The book also highlights the ethical implications of data manipulation, urging professionals in the field to adhere to rigorous standards of accuracy and transparency.

Overall, "Bad Data" is an enlightening read that underscores the pervasive impact of data on our lives and the necessity of maintaining vigilance against its misuse. Sturge's expertise and engaging narrative make this book a valuable resource for anyone interested in understanding the power and pitfalls of data in the modern world.

[Qiuyu · Rating 4 out of 5]

this is such a good book that explores the issues with data (as hinted by the title), specifically in the UK context. the overarching idea is how overly confident people are when it comes to quantitative numbers and modelling, pushing for more emphasis on the qualitative analysis of the data we currently have.

this book encouraged me to see statistics beyond the numbers, and explore alternative meanings apart from those that are given at face value. eg crime rates reduce, instinctively, we would assume that the district has become safer, however, it might just be because of the police forces recording less crime rates to claim that they successfully deter crimes from occurring. many times, we take in numbers for granted too fast that we forget about the possibilities of inaccuracies which is the case all the time.

despite the seemingly pessimistic views of the book, how our current data sucks, this book encourages us to keep trying, and affirms us for our progression in collecting data. as the world continues to be digitalised and recognises the value of data assisting us in making better decisions, there is much to look forward to regarding the future of data.

[John Isles · Rating 5 out of 5]

I left the UK Government Statistical Service almost 37 years ago, and I was therefore very interested to learn from this book how much things have changed there, or perhaps I should say in what are now called the (central) Office for National Statistics or the (wider) UK Statistics Authority. There's a French expression that well describes what happened, roughly translated: "The more things change, the more they stay the same." UK government statistics are certainly better than those of many other countries, but there are still serious shortfalls in what data are collected, and problems in how they are used. At least UK road accident statistics (one area in which I worked) receive praise here. I also learned that, had I not quit my job to live in a Greek village in Cyprus and study and write about the stars, my post as a statistician in Westminster would likely have been moved to Newport in Wales - not my first choice. I enjoyed this book very much; the author has a great command of the subject.

[Tim Lucas · Rating 3 out of 5]

An ok book.

Partially it just wasn't quite what I wanted. The blurb want totally clear and I also just glossed over some of the description. It's a pretty good non-technical introduction to statistics for people interested in policy. However, I know plenty of statistics. I thought it was going to be a much more specific book on how statistics are used in the UK government. How are they requested, generated and communicated? And it wasn't really that.

The chapter headings looked like they made sense, but there wasn't that laser focus on keeping to topic within chapters. There are many pages that, if read in isolation, you wouldn't know which chapter it was from for example. So the point of each chapter ends up a bit unclear, and also the whole book smudges into one long list of examples and anecdotes.

Still an interesting read and probably really useful for the right reader.

[O* · Rating 4 out of 5]

Super interesting book, even for someone who knows a fair amount about the data used by governmental institutions. Putting on my non-British glasses it is funny to read a Briton comment on use of data in other countries writing things like 'in Germany you need to register with the council when you move in/out and this data helps their planning of carehomes, schoools etc etc'. Yes, isn't that super evident and a normal thing to do...? Not in the UK!

Some examples just make you shake your head in disbelief, such as the UK using surveys of people coming to/leaving the UK at four airports for their official statistics. Seriously!?

Highly recommended, very informative and easy to read.

[PPEsquire · Rating 5 out of 5]

Based on her real-life experience of having to provide data to MPs on basically anything they need or want, Sturge paints a mind-boggling portray of how misused and misunderstood data is. Her strength is to remain committed to data quality and use but demonstrates how data is usually used only as a weapon, a factoid to win political arguments, as opposed to an actual source of insight to build policy. The problem is not data itself, she pleads, but how badly it is used. Both hilarious and enlightening.

[Doug Newdick · Rating 3 out of 5]

A good introduction to the problem of how bad data plays out in government decision making. Those with a basic knowledge of statistics won't learn much about the kinds of mistakes that government agencies, politicians and the public make in creating, understanding or applying data to big decisions, but the detailed examples are still illuminating. A weakness of the book is that all of its examples are from the UK - fair enough I suppose given that Sturge worked for the UK government - but it does limit its applicability for readers outside the UK.

[Benjamin · Rating 4 out of 5]

Not dumbed down but too deferential. A good guide to the pitfalls of statistics as used in the public debate but to my taste should have drilled down to greater details in the examples the author gives, eapecially as they tend to be a little samey. Confirmed my suspicions about many statistics on costs of various activities to the economy, health system, etc., but still feel the author should have been bolder in her criticisms.

[Faye Hollands · Rating 4 out of 5]

This book shows the alarming prevalence of poor data used in policy decision-making. I really loved the closing remarks about how using bad data can have real-life, and often detrimental consequences for individuals who lose out.

Although based in the UK, there are so many parallels to Australian policy (i.e. robodebt) and so many things politicians could learn.

This is a must for political science students, and especially for those looking to get into policy research.

[Libby Zhang · Rating 5 out of 5]

Although all the arguments in the book are not new, the rich real-life examples related to mandated data and official statistics are eye opening, making us more alert about any numbers we heard from the government and politicians. It is a good book for general public who are not savvy with statistics but also a good read for anyone who collect and use data in daily life. It is especially good for educators who need vivid examples when teaching statistics.

[Sam · Rating 4 out of 5]

3.5 - this took me longer to finish than usual (due to life rather than the book itself) so thoughts less detailed than i would like! this was really thought-provoking, and as someone working close to govt statistics i think really useful. my main criticism is i occasionally felt like this was a series of slightly related anecdotes rather than a logical monograph, but maybe that’s just the nature of the beast!

[Paul · Rating 5 out of 5]

If you are a data nerd, or genuinely interested in data, then there is a good chance that you’ve already covered the concepts in this book.

What’s so refreshing about this book is that it’s using the UK as the thing to drive the points.

You should not be allowed to take a government job/position without first reading this book!

[Andrew · Rating 4 out of 5]

As books about statistics go, this is a great read and is full of real-world examples in Britain of how “bad data” can mislead us and cause all sorts of problems. There is a passionate plea to improve how governments collect and interpret data, with an outlook of hope for greater accountability and more success in policymaking.

[Haich · Rating 4 out of 5]

Rating: 4/5

This took me a lot longer to read than I thought it would because it was so fucking interesting and I ended up going down some rabbit holes of following up points and information.

A little too - motivational speech of an ending and pop stats language in some parts which knocked of a bit for me, but overall a very interesting and worthwhile read.

[Limme · Rating 4 out of 5]

A remarkable data science illustrations in a magnificent reflections of all types of public policies data impacts on our political , Social and national strategies planning , designing and projections.Really interesting non-fiction book I red this year.

[Jim · Rating 4 out of 5]

The author lists ways where “figures don’t lie, but liars can figure.”

It’s well written, but it’s written for a British reader. If readers keep that in mind, I think they will be informed and will enjoy this book.

[John Lord · Rating 3 out of 5]

A good read and well-written, particularly if interested in government data.
Having read a few similar books there does seem to be getting a little duplication of examples used, particularly crime data.

[Chris · Rating 5 out of 5]

Essential reading for citizens of contemporary democracy, and those interested in how policy is formed. Highlights the shaky foundations of many modern assumptions regarding society and policy. Excellent read.

[Koo Ping Shung · Rating 5 out of 5]

It's a great book for anyone who wants to be data literate including data professionals.

[Tom Ives · Rating 2 out of 5]

Nice idea, doesn't really go anywhere

[Tom · Rating 5 out of 5]

Excellent book - really enjoyed it. Eye opening about the amount of uncertainty of data, how data collection could be improved, and how some of the data used to make decisions is currently useless