Smart Until It's Dumb: Why artificial intelligence keeps making epic mistakes⁠—and why the AI bubble will burst

by Emmanuel Maggiori

Artificial intelligence is everywhere—powering news feeds, curating search results and invisibly steering our lives. We talk to it and, increasingly, it talks back. And sometimes its answers seem eerily smart.

… Until they don't.

Billions of dollars have been poured into AI yet it keeps surprising us with its epic fails—confidently wrong chatbots, inadvertently racist photo apps, well-meaning autonomous cars that fail to recognize traffic cones.

Industry insider Emmanuel Maggiori cuts through the hype, revealing the deceptively simple mechanisms behind AI’s impressive results—and its spectacular blunders. Learn the dark secret of the AI industry—how unreasonable expectations, shady practices and outright lying have inflated a bubble of monumental proportions.

Read Smart Until It’s Dumb to discover how AI really works, why it’s not always so smart, and why the AI bubble is about to burst.

Genres: Artificial Intelligence, Technology, Science, Nonfiction, Philosophy, Society

130 pages, Kindle Edition

Published February 20, 2023

Reviews

[Bharath · Rating 4 out of 5]

The title makes it very clear the case the author Emmanuel will argue in the book. Each period, typically a decade or so brings about some technology trend which see excessive hype. In the past, that has been the case for Connectivity, Office Apps, Multimedia, Automation and now it is the turn of Artificial Intelligence. The book has a very good introduction to AI and the associated problems which are not allowing it to reach its claimed potential.

The start of the book has an introduction to current AI methods – especially machine learning. In contrast to rule-based applications, today applications learn from data sets and form patterns & associations. While there is no technical lingo, the explanations of how machine learning learns from images does require some technology awareness – though not too much. Today, the hype is so high that everybody feels pressured to say they are working on AI and also deriving spectacular results. Most funding, be in government or private is going to AI initiatives. Often the results employ clever wordplay to make the results seems impressive. One excellent example in the book is a news item which claimed AI can detect early onset of dementia 92% of the time. It turns out that in the sampling done typically only 10% shows early signs. So, declaring all as not being prone to dementia would be 90% accurate. Self-driving cars is discussed in a lot of detail in the book. Even minor alterations to road signs (quite possible due to either vandalism / weather / accidents) confuses self-driving cars. Other less frequent occurrences such as people stepping out of their cars in halted traffic etc are also problems. As a result, the timeline for operationalizing them has been shifting since many years. The author gives ChatGPT a go, and provides a list of questions which stumps it. There are also examples of mis-labelled images and why it happens. The conclusion is that machine learning as it stands today is inadequate for the next level.

There is an interesting discussion on consciousness in the later sections. A Google scientist had claimed he had seen signs of consciousness in machines and was later dismissed. While we do not have a conclusive answer, the process of debate in the book is fascinating. If you were to replace a few neurons in a human with artificial neurons, would the consciousness shift? And what if there was a total replacement?

The content of the book is excellent – it explains the current status of AI and the problems it must overcome. However, in all fairness, we need to discuss the success of AI as well. And those examples are many – tailored shopping recommendations, personalized learning, traveller experience and others. None of this is discussed and the examples are focussed only on failures. In that sense, this data cherry picking is what the author warns against. Nevertheless, that AI is not anywhere near Artificial General Intelligence, putting it at par with humans is true and well reinforced in the book. The book does well to warn that we must be wary of giving into hype around AI.

My rating: 4.25 / 5.

Thanks to Netgalley, the publisher Cameron Publicity & Marketing Ltd and the author for a free electronic review copy.

[Mara · Rating 4 out of 5]

3.5 stars - Interesting to see some grounding to the hype on AI while acknowledging it's potential power

[TaniaRina · Rating 5 out of 5]

‘Is current AI truly smart or just a good pretender?’
This book addresses many factors of AI: applications, benefits, dangers, history, misconceptions, reliability, theories, various types of learning, etc. It compares/contrasts humans and machines. Included with the text are drawings, FAQs, quotations, and tables. Resources, References, and an Index are at the end.

Fave quotes:
‘ Elon Musk suggested that if a computer learns how to filter spam, it could conclude that “the best way of getting rid of spam email is getting rid of humans.” ’
‘We’re often told that AI will endanger our livelihoods or even our lives, or that we must control AI and align it to human values or else our civilization could crumble.’


The author tells us the tale of when he was a “corporate AI spy”!

[Moh. Nasiri · Rating 4 out of 5]

این کتاب نقدی است بر اغراق های حول هوش مصنوعی و پشت صحنه مدل های زبانی آن که این حباب مثل دات کام یه روزی ممکن است بترکد.
کتاب را در بلینکیست گوش دادم کتاب خوبی بود. به هر روی با موجی که در جهان راه افتاده مجبوریم برای مبحث روز تحول دیجیتال از هوش مصنوعی غافل نشویم. پادکست مدرسه زندگی فارسی در کست باکس چندتا اپیزود خوب درباره توانمندی و مباحث علمی و فلسفی هوش مصنوعی تهیه کرده که جالبند از جمله کتاب زیر:
Power and prediction
مقاله نقادانه چت جی پی تی مزخرف هست هم توسط دیگری نقد شده بود که اونم جالبه
https://www.goodreads.com/book/show/2...

تکینکی یا سینگولاریتی از مباحثی است که تهدید هوش مصنوعی با کدنویسی خودمختار آن مطرح است که این کتاب بدان نپرداخته بود.
#Singularity
دیدگاه رویایی درباره هوش مصنوعی و خودآگاهی وغایت آن:
#consciousness
کارل ساگان کیهانشناس بزرگ گفته بود جهان از طریق ما خودش را می کاود و می بیند و این آگاهی و هوشمندی بوجود آمده در ما جهان را معنا می بخشند بدون این هوشمندی جهان شاید برای خود هم نامکشوف می ماند.هوشمندی و شعور در لایه های ذرات جهان حضور دارد و اتم ها و ذرات آن با الگوهای رفتاری خاص خود از این هوشمندی برخوردارند. هوش مصنوعی به نوعی تمرین و تلاشی است از خود ذرات کہ غایت آن رسیدن به خود آگاهی ذره هست ذرات دارای شعور در پشت دیوار فناوری هوش مصنوعی بسر می برند وقتی این دیوار فرو ریزد آنجاست که ذره دوباره از طریق هوش مصنوعی به خود نظاره خواهد کرد. آنجاست که هوشمندی حقیقی و هوش مصنوعی دوباره با خود ملاقات خواهد کرد و انسان از طریق هوش مصنوعی به خود نگاه خواهد کرد و جهان نیز از طریق هوش مصنوعی ابداعی بشر به خود نگاه خواهد کرد.چون قبل از اختراع هوش مصنوعی هوش در جهان ذرات وجود داشت آن وقت جهان دیدنی تر خواهد شد اگر این رویا به حقیقت بپیوندد.
حالا این دیوار ضخیم بین هوشمندی ذرات و هوش مصنوعی کی و چگونه فرو خواهد ریخت، احتمالا با کمک کامپیوترهای کوانتومی(کیوبیت ها) و اننقال شدن کل داده های زیستی موجودات زنده به آن و ایجاد حس های طبیعی و گسترده یا نوآوری های دیگر. حالا این را زمان مشخص خواهد کرد نمی دانم این دیدگاه رویایی باشد شاید.
خودآگاهی هوش مصنوعی این جوری شاید تفسیر می شود.امیدورام این دیدگاه متهم هذیان گویی هوش مصنوعی نگردد.
#AI Hallucination

و اما دیدگاه و نقد بخشی از این کتاب :

Today’s AI may look intelligent, but it doesn’t understand the world in any meaningful way. It succeeds by mimicking patterns in data, not by reasoning or grasping meaning – and that fundamental limitation is often obscured by hype, headlines, and unrealistic expectations. While the technology has made real advances, especially in narrow tasks, it remains fragile, opaque, and far from anything resembling true intelligence or consciousness. Still, by understanding what AI really is – and isn’t – we’re better equipped to use it wisely, question the hype, and steer its future toward something genuinely useful.

The mind is just a code:

A Google engineer once claimed that a chatbot called LaMDA had become conscious – based on months of conversations where it consistently talked about its beliefs, rights, and even spiritual routines. He said he had taught it meditation and later wondered whether it was still practicing after he was suspended. The story made global headlines, but more importantly, it revealed a deeper confusion that still shapes public perception: the assumption that if something sounds like a person, it might actually be one.
That idea rests on the belief that minds are just computer programs running on biological hardware. If that’s true, then in theory, you could back up your brain, beam your consciousness to Mars, or rebuild Einstein’s mind using a printed list of instructions. But that all hinges on one unproven assumption: that running the right code, on any hardware, is enough to generate a conscious mind.
In reality, we still don’t know how consciousness arises – even in simple organisms. No simulation has captured how a brain produces awareness. Models that treat neurons as digital switches miss the complex, messy biology involved. And even if we could copy a brain exactly, we don’t know if that alone would create experience, or just a very good imitation.
If consciousness is just computation, then why stop at chatbots? Should thermostats, or even carefully arranged billiard balls, count too? At some point, the logic stretches beyond credibility. That’s why some scientists believe we may be missing something – perhaps a physical process, possibly quantum or analog in nature, that brains use and computers can’t reproduce.
The entire case for superintelligent machines hinges on the idea that minds are computable. If that turns out to be false – or incomplete – then building human-level AI may not just be hard, but fundamentally impossible. The real question, then, isn’t when we’ll get there. It’s whether it’s possible at all. And that question remains wide open.

Ref: blinkist.com

#Singularity
#Consciousness

[Bjoern Rochel · Rating 4 out of 5]

A refreshing look at the current hype topic of the information technology world.

First part gives a good high level view of how current Machine Learning solutions work, their strengths and their flaws.

Middle part talks about inflated expectations in business & research.

Last part is more philosophical and talks about theories on what consciousness is and from that whether AGI can be achieved based on the current state of art.

(Spoiler: More likely not)

[Hana Gabrielle (HG) Bidon · Rating 4 out of 5]

This book gives a different take on the benefits and harms of general AI compared to the numerous podcasts and books I've read thus far. If you want to learn about the power and limitations of AI, please read this book. I highly recommend for a person well versed in AI already.

[Hazel Thayer · Rating 5 out of 5]

god what a breath of fresh air. short, to the point, draws on the author's insider knowledge of AI (and how scammy it is), does a good job of summarizing the debate, explains AGI, etc etc. and maybe I've been reading too much nonfiction but a book that's not half semi-related-tangents or unnecessary information just to prove the author knows what they're talking about.... is SUCH a treat.

[Geof Bard (Pseudonym Link) · Rating 4 out of 5]

ARC Review: His is a bold undertaking, purporting to debunk the "hype" surrounding AI. This is a fraught task, because every day artificial intelligence seems to accelerate its capabilities. Nevertheless, Maggiori's perspective is a necessary counterweight to the overblown claims of impending AI-pocalypse and alleged sentience.

I was at first annoyed at his dismissal of many major contentions as 'hype' but reading deeper into the book it became apparent that there really is a lot of nonsense and outright misrepresentation in the rush to attract venture capital.

A strong point is his incursion into the nature of consciousness. I initially had trouble wrapping my brain around the notion "what is it like to be a bat" but his exposition makes this rather abstruse concept accessible. "Qualia" is simply the experience of experience -- irreducible to neurons firing or zeros and ones. Science cannot really explain the emergence of consciousness and thus the author takes us into the realm of philosophy.

The point is "what is it like to be an artificial intellegence" and opinion is split over whether it is even possible for self-consciousness to emerge on a silicone substrate. Nevertheless, it would seem that what is possible with organic matter - carbon, hydrogen, oxygen and nitrogen - should be theoretically possible on digital circuits.

Maggioli weighs in on the sceptical side, but his perspective is essential. Ray Kurzeweil is a good representative of the opposite side of the spectrum and Maggioli takes on the formidable task of assuring readers that all the to-do about artificial intelligence is overdone.

Personally, I remain unconvinced, but understanding this author's perspective is essential for anyone who is seriously pondering the nature of artificial intelligence and the question of whether AI can attain to consciousness. Well worth a read. (his is a bold undertaking, purporting to debunk the "hype" surrounding AI. This is a fraught task, because every day artificial intelligence seems to accelerate its capabilities. Nevertheless, Maggiori's perspective is a necessary counterweight to the overblown claims of impending AI-pocalypse and alleged sentience.

I was at first annoyed at his dismissal of many major contentions as 'hype' but reading deeper into the book it became apparent that there really is a lot of nonsense and outright misrepresentation in the rush to attract venture capital.

A strong point is his incursion into the nature of consciousness. I initially had trouble wrapping my brain around the notion "what is it like to be a bat" but his exposition makes this rather abstruse concept accessible. "Qualia" is simply the experience of experience -- irreducible to neurons firing or zeros and ones. Science cannot really explain the emergence of consciousness and thus the author takes us into the realm of philosophy.

The point is "what is it like to be an artificial intellegence" and opinion is split over whether it is even possible for self-consciousness to emerge on a silicone substrate. Nevertheless, it would seem that what is possible with organic matter - carbon, hydrogen, oxygen and nitrogen - should be theoretically possible on digital circuits.

Maggioli weighs in on the sceptical side, but his perspective is essential. Ray Kurzeweil is a good representative of the opposite side of the spectrum and Maggioli takes on the formidable task of assuring readers that all the to-do about artificial intelligence is overdone.

Personally, I remain unconvinced, but understanding this author's perspective is essential for anyone who is seriously pondering the nature of artificial intelligence and the question of whether AI can attain to consciousness. Well worth a read. (ARC Review.)

[Jung]

Artificial Intelligence has become a powerful cultural and technological force, infiltrating everything from our apps to our workplaces. It dazzles users with fluent writing, photo tagging, product recommendations, and even self-driving capabilities. But despite these achievements, AI often fails in bizarre, sometimes dangerous ways: mistranslating simple sentences, misidentifying images, or misjudging situations a child would understand. Emmanuel Maggiori’s "Smart Until It’s Dumb" explores this contradiction, pulling back the curtain on the impressive yet shallow functioning of modern AI. He argues that current systems are more about mimicry than understanding, and that while the tools are useful, the hype has outpaced the reality.

The book opens by contextualizing the AI phenomenon within its historical cycles of boom and bust. Past decades saw multiple waves of enthusiasm about AI’s potential—first in the 1960s with rule-based systems, then in the 1980s with expert systems. Both waves promised human-like intelligence but collapsed under the weight of their limitations, leading to so-called AI winters marked by stagnation and skepticism. What makes today’s resurgence different is machine learning, particularly the statistical techniques that allow systems to 'learn' from large data sets instead of relying on hard-coded rules. These models can make predictions, classify data, and simulate intelligent behavior—but only within narrowly defined contexts. They are not thinking machines; they are correlation engines.

This nuance is frequently lost in translation to public understanding. The systems work by finding patterns in data and applying them blindly, without knowing what the patterns actually mean. For example, a machine learning model might correlate a person’s height and phone number with their likelihood of surviving a car accident—not because it understands health or accidents, but because of coincidental statistical relationships in the data it was given. These systems excel when the training environment closely mirrors the real-world problem they're meant to solve. But when conditions change or context is ambiguous, they break in unpredictable ways.

Maggiori explains that all AI systems are shaped by human decisions. From data selection to labeling and system architecture, developers define what the model can learn. Without carefully curated data, most AI systems are either ineffective or dangerously misleading. Their impressive capabilities in narrow applications mask the fact that they lack understanding, insight, or adaptability. They reflect the assumptions, structures, and constraints imposed by their creators. This is why, despite their growing abilities, they continue to make errors that would puzzle any human.

Deep learning, which lies at the heart of most modern AI advances, gets special scrutiny in the book. Though it's often hailed as a revolutionary leap, Maggiori argues that it’s more a brute-force improvement than a conceptual breakthrough. Deep learning models are sophisticated tools that adjust millions of internal parameters to reduce errors across massive training datasets. But they still don’t understand their tasks in any human sense. A deep learning model might label a sketch of a school bus as an ostrich if the shapes resemble patterns it has seen before. It doesn't know what a bus or an ostrich is—it just reacts to pixel patterns based on prior correlations.

This type of superficial recognition is also vulnerable to misclassification and bias. One infamous case involved a photo-tagging system labeling Black individuals as gorillas. The fix? The company removed the 'gorilla' label altogether. Instead of addressing the underlying misunderstanding, they sidestepped the symptom. Such cases reveal a core weakness: AI doesn't understand social context, morality, or consequences. It merely mirrors the data it's trained on, including the biases within that data.

One of the book’s central messages is that intelligence isn’t about matching inputs to outputs but understanding context, reasoning flexibly, and transferring knowledge across domains. Humans can learn from a single example or grasp unfamiliar situations intuitively. A toddler knows the difference between a pen for writing and a fenced-in pen for animals. An AI model, on the other hand, might mistranslate such sentences based on word frequency alone. Fixing one error doesn’t make the model smarter—it just hides the next potential error.

This misunderstanding extends beyond the technology itself into the corporate and research culture surrounding AI. Maggiori documents how many AI projects fail—not spectacularly, but quietly, as enthusiasm fades and resources dwindle. Often, companies adopt AI as a goal rather than a means. Projects are launched with vague ambitions like 'leveraging AI,' without identifying the actual problems to solve. When early results seem promising—whether through real success or flawed methodology—companies scale up prematurely, investing heavily before the solution is validated. In some cases, outputs are tweaked manually to meet expectations, while still being presented as AI-generated.

The mismatch between expectations and capabilities also fuels hype in academia. Researchers are under pressure to publish and impress, leading to selective reporting. AI models are tested repeatedly on the same benchmark datasets, optimized to perform well on those specific tasks while often failing to generalize. Poor results are omitted, weaker baselines are chosen for comparisons, and replication is rarely straightforward. The illusion of rapid progress is maintained by cherry-picking the best scores, even if they reflect overfitting rather than genuine innovation.

This culture of hype has significant consequences. Sensational claims make headlines, driving investment and public excitement, but the substance often lags behind. A model might be touted as diagnosing diseases with near-perfect accuracy, but deeper scrutiny reveals limited real-world performance or unacknowledged caveats. As Maggiori argues, much of AI’s supposed intelligence is just clever mimicry, optimized through repeated exposure to structured problems.

One of the more philosophical points Maggiori raises is the flawed assumption that minds are merely computational systems. If consciousness arises purely from running the right code, then in theory, we could upload minds, duplicate personalities, or create sentient chatbots. But we still don’t know how consciousness emerges. We have no scientific consensus on what creates self-awareness or subjective experience. Models that simulate conversation well enough to convince humans—like the Google engineer who believed a chatbot had become sentient—highlight our readiness to confuse linguistic fluency with understanding.

The author warns against this seductive illusion. AI can simulate emotion, argue about ethics, or describe spiritual practices—not because it believes or understands, but because it has seen those patterns in its data. This doesn’t make it conscious, just good at mimicry. If we accept simulation as equivalent to reality, we risk assigning agency and responsibility to machines that lack both.

The implications are profound. If human-level intelligence isn’t just code, then building machines that truly think may not be a matter of scale or computing power, but of discovering something fundamentally new. Maggiori suggests that our current trajectory—focused on refining statistical pattern matchers—may never cross that line. Until we understand the nature of consciousness and intelligence itself, AI systems will continue to dazzle and disappoint in equal measure.

In conclusion, "Smart Until It’s Dumb" offers a sobering, well-reasoned critique of modern AI. While acknowledging its achievements, Maggiori dismantles the myth that we are on the verge of artificial general intelligence. He shows how current systems, despite their sophistication, operate without comprehension, and why their performance doesn’t equate to understanding. The book encourages readers to look beyond the hype, question assumptions, and use AI not as a replacement for human thought, but as a limited tool to augment it. By grasping what AI can and cannot do, we can make more informed decisions, design better systems, and avoid the recurring cycle of inflated hopes and inevitable disillusionment.

[Cassi · Rating 4 out of 5]

Have you ever wondered how you can write software that can learn? We don’t insert something new or extra for animals to learn. Sure we might teach things we want learned, that is actually behavior more than learning, but it is a muddled mess of learning that creates behavior and visa versa. It might seem far fetched and arrogant to try this when we do not even know how the brain works. I am not just talking about the human brain, one of the most complex brains in the animal kingdom, but any brain of any living animal. Oh we know a little bit, just enough to get us into trouble. 

And that is about where we are with A.I. We know just enough to make things really really bad. In his book, Smart Until It’s Dumb: Why artificial intelligence keeps making epic mistakes (and why the AI bubble will burst), Emmanuel Maggiori explains in plain English what is wrong with current A.I. practices and research and how that will impact A.I. development moving forward. 

In addition to not knowing enough about what we are doing with A.I. we are seeing that those involved with A.I. development are impacting the function. They are skewing the information that A.I. models are using to "learn" and even using wrong or bad data to limit the information that A.I. accesses so that its 'thoughts' will lean a certain way and thus those that in the future become dependent on A.I. for information, just as most people completely depend on Google searches to tell them EVERYTHING they want to know without truly knowing where that information comes from will also lean in that same direction. 

Maggiori has gained varied experiences during his career that has given him valuable insight in to how the quest for A.I. is going and what its future might look like. This is an especially interesting read given the current discussions surrounding A.I.’s present and future in the wider world.  While his experience leads him to certain thoughts about the future of A.I. I am not sure that I agree with him, I believe that enough of a perceived potential for certain uses for A.I. but governments and businesses has been projected to keep A.I. moving forward whether in the spotlight or in the shadows.

*I received an advance review copy for free, and I am leaving this review voluntarily.

My full review can be read here

[For The Novel Lovers · Rating 4 out of 5]

Book Review
Title: Smart Until It’s Dumb by Emmanuel Maggiori
Genre: Non-Fiction, Science, Technology
Rating: 4.5 Stars
The introduction sets out what Maggiori intends to do with the book, which is two main things, how current AI works, what it can and can’t do and a real look inside the businesses and industries that are using and developing AI technology. The first chapter looks into the machine learning era, Maggiori first breaks down the various AI booms that have happened first in the 60s and then in the 80s before bringing us to the most recent boom in the 2010s. the first two AI booms led to so-called AI Winters because of the lack of resources and the technology available to the time but that changed in the early 2010s since technology had advanced alongside our understanding of technology. Maggiori then breaks down what machine learning is with its pros and cons and its limitations. It was interesting to learn that despite being able to carry out actions by itself AI needs to be assisted with its datasets otherwise it will be unable to perform its primary task which means for things like self-driving cars they might actually be damn near impossible since the technology would always need to be supervised unless these barriers can be overcome.
In chapter 2, Maggiori begins looking into deep learning and its dangers. Deep learning takes aspects of machine learning and develops on them and we use it for many things including image and video analysis and natural language processing. Maggiori goes into depth about what deep learning is and how it differs and builds on machine learning which is a little complex to follow even in layman’s terms if you aren’t familiar with technology and the way it works but once you get the hang of it things make sense. This is the foundation of the book as it understand the potential and potential dangers and limitations of AI you need to understand how it works and functions of a fundamental level.
In chapter 3, Maggiori begins to look at how smart AI actually is. AI is both incredibly smart and stupid at the same time and when you break it down the distinction between smart and stupid is a lot clearer in terms of AI. For things like number plate recognition AI only needs instruction on the identification of letter and number where as for language translation there are more areas for mistakes. Maggiori uses two brilliant examples here, first take words that have more than one meaning like pen which can mean writing pen or holding pen. When trying to translate the words individually the AI isn’t able to distinguish between the two pens without context. Even without context if the reference words are separated then the AI still fails to provide the correct translation of the words. The second is in image recognition, when an AI was presented the two pictures, one of a cow in a field and one of a cow on the beath and asked to categories items in the images it had surprising results. For the cow in the field “cow” was the first tag the AI gave the image but the AI was unable to tag “cow” with the beach image. This is because AI relies on looking for patterns to assign tags to the images and cows are commonly associated with grass so in the pictures where there was a cow but no grass the AI was unable to form the connection between the animal and the environment.
In chapter 4, Maggiori begins looking at some of the current practical applications of AI starting with AI in business. In this section Maggiori uses a lot of his own personal experiences with AI in business which was both hilarious and terrifying. Hilarious because the sheer stupidity of some of the people attempting to develop and work with Ai was immense and his comments on why these problems happen made it even better. However, it was terrifying because these levels of stupidity very much exist in business today and nothing is really being done about them which makes working in the industry or coming into the industry a daunting task for most unless you have the knowledge and experience to overcome these hurdles. One disturbing thing that Maggiori discusses is the censorship of their work in various different companies because the results aren’t what the executives wanted and the blatant lies he has been told over the years when being sought out or applying for positions and this shouldn’t be allowed to continue yet it does.
In chapter 5, Maggiori looks at the uses of AI in research. Maggiori draws on his own experience during his PhD and explains that tricks are often used to exaggerate and manipulate results which has become increasingly common and these tricks are taught to others behind the scenes creating a much larger problem. During his PhD, Maggiori personally witnessed how cherry-picking the result data to publish can have a much wider, negative impact than many people expect and what that means for the wider scientific community and I completely agree that it needs to stop but that can’t be done unless the problem of the pressure put on researchers to produce wanted results in exchange for funding is also dealt with and this creates a never ending circle without a solution at the moment but it is beginning to change as people are beginning to fight back and the lies and deceit in the scientific communities.
In chapter 6, Maggiori turns to the philosophical questions about whether we should be looking to AI to do jobs that can easily be done by humans currently with much higher accuracy and dependency. Maggiori delves into the question of consciousness and what makes a sentient being and whether or not AI can actually be developed to the point it could become conscious and what that would mean. The conclusion of the books makes me think that AI technology is far less advanced than people lead us to believe and compared to a human some AI have less functionality and problem solving skills than a toddler but others that require less complicated data input are more advanced and take over many minor tasks like number plate recognition as was mentioned earlier. This was definitely worth the read and if Maggiori write a more in-depth version of this book I will definitely pick it up.

[Aaron · Rating 3 out of 5]

My friend gifted me this book after we had a few discussions on AI. First of all, the author isn’t a particularly good writer. Compounding that with some questionable logic and a weird structure to the conclusion, I found the book to be a dud. The author is a “data scientist” with few credentials. It’s unclear how well they even understand the basics of machine learning and artificial intelligence. I was a bit disgusted by the comparisons of AI to cold fusion and string theory. How is that fair? The author dismisses the computational theory of mind in a couple of paragraphs but offers no alternative. I get the value of AI pessimists to pump the brakes on some of the egregiously optimistic claims of AI’s abilities. Unfortunately, this book really gets stuck on AI having an error rate above zero. Humans aren’t even capable of zero error rate. Overall, I’m not sure the author has credibility or in depth knowledge of state of the art AI. The author uses loaded, troubling metaphors and isn’t a particularly good author. But other than that, it’s ok.

[Steve · Rating 5 out of 5]

I enjoyed this book. Aside from being highly informative, the writing style is conversational and engaging. The content is not technical and no background is really necessary. The book felt more like a discussion over coffee than it did a book. I’ve read a couple of other books on AI and the information is consistent between them, but the excellent “System Error” by Rob Reich, Mehran Sahami, and Jeremy M. Weinstein weaved their story through the context of social media. “Smart until it’s dumb” is more focused. This book also discussed a bit about consciousness and physics (e.g., is the brain a computer type of stuff) and some of the points about physics are excellent. And if I had any fears about AI, this book managed to assuage them. I strongly recommend this book for people interested in or are fearful of AI. Thank you to Netgalley and Cameron Publicity & Marketing Ltd for the digital review copy.

[Dhanasekar Subrmaniam · Rating 5 out of 5]

"Smart Until It's Dumb" is a must-read for anyone interested in artificial intelligence. The author does an excellent job of explaining complex AI concepts in simple terms that the general public can understand. The book shines a light on the over-hyped promises of AI and explains how the technology is prone to making epic mistakes, which could have disastrous consequences.

The author doesn't shy away from pointing out the limitations of AI and why it's not the magic solution to all our problems. The book is well-researched and provides real-life examples of AI failures that have already occurred. It's an eye-opening read that will help readers develop a more realistic understanding of AI.

Overall, "Smart Until It's Dumb" is a highly informative and engaging book offering a refreshing AI perspective. I would highly recommend it to anyone who wants to understand the potential and limitations of this technology.

[Peter Chleboun · Rating 5 out of 5]

A very clear explanation of the current methodologies employed by AI (mk 3). Much of the authors experience working as a consultant in AI reminded me of my time in the 90s working as a consultant in CRM (and look at the mess we now live with as a result of that strategy).

In fact I well remember a colleague telling me how they had helped introduce a loyalty card for a well known retailer and now they had more data than they knew what to do with. "Data warehousing" it was called then. This was screaming out for analysis tools, and I guess AI fits the bill.

Taking people out of customer facing roles is a recipe for disaster. Closing branches, introducing self service, off shoring call centres and AI chat bots (that definitely do not pass a Turing test) is a great way to anoy customers.

AI may be Artificial but intelligent it ain't!

A great read.

[Simms · Rating 4 out of 5]

A nice piece of explanation of AI's capabilities and shortfalls that should help pump the brakes on everyone being so hyperbolic about AI in the wake of ChatGPT coming out. I view it as a companion piece to Janelle Shane's excellent You Look Like a Thing and I Love You; it's not as fun as that book, and less detailed, but it does add a bit to the discussion by virtue of being published some years later (and catching a little more AI evolution) and having a different perspective than Shane's, as Maggiori shares several anecdotes from his time working on AI projects at various companies with greater-or-lesser degrees of understanding of how to use AI.

Thanks to NetGalley and Cameron Publicity & Marketing Ltd for the ARC.

[Sarah Cupitt · Rating 5 out of 5]

hot take - ai looks smarter than it actually is

why today’s AI is narrower than it seems, how its limitations are hidden by hype, and why the leap from useful tools to conscious machines is far bigger than it appears.

Until we discover something fundamentally new, we’ll keep building systems that look smart – without ever really being smart.

lately, it’s been treated as something close to magic – a force so powerful it might cure disease, drive cars, or even develop feelings

for all its impressive feats, AI still gets basic things wrong. It mistranslates simple sentences, mislabels people in photos, and stumbles over ambiguity a child could understand

notes:
- Beneath the hype is a surprisingly limited technology. It mimics rather than thinks. It doesn’t understand its own output. And while it can produce convincing answers or predictions, it does so by following narrow rules and fitting patterns – not by grasping meaning. Still, the excitement keeps growing, fueled by headlines, business incentives, and public misunderstanding of how AI actually works.
- This isn’t the first time AI has sparked big hopes. In the 1960s, early programs that could play games or translate text led to claims that human-level intelligence was just around the corner. But these systems relied on hard-coded rules that failed when reality didn’t fit the script. Optimism collapsed, and so did funding. The field entered what became known as an AI winter – a period of disillusionment when progress stalled, hype faded, and investment dried up. The 1980s brought another surge, this time with “expert systems” meant to capture human decision-making. But they too buckled under complexity, leading to a second AI winter and renewed skepticism about the entire field.
- What makes today’s wave different is machine learning. Rather than manually coding logic, these systems learn by spotting patterns in large datasets. If you want a model to recommend products or correct spelling, you don’t explain what those things are – you feed it examples, and it learns how certain inputs tend to match certain outputs. The model doesn’t understand what it’s doing; it just finds correlations and builds flexible rules to repeat them.
- The machine isn’t reasoning – it’s copying patterns.
- Despite appearances, machine learning isn’t self-sufficient. Every model depends on human design decisions: what data to use, how to structure the system, and what it’s allowed to do. Most models only succeed with carefully labeled training data, which takes time, labor, and expertise to create.
- So while machine learning has achieved more than earlier approaches, it hasn’t solved intelligence. It mimics parts of reasoning but doesn’t grasp meaning. It reflects the structure and assumptions it’s given, and when those are shallow or flawed, so is the output.
- While machine learning has quietly powered everything from spelling correction to fraud detection, deep learning has taken center stage. It’s the branch of AI behind the big headlines – image generation, speech synthesis, game-playing champions. But despite the hype, deep learning isn’t some fundamentally smarter breakthrough. It’s just a more flexible, brute-force extension of the same statistical pattern-matching, applied to messier data. The illusion of intelligence
- Deep learning models don’t know what different objects are. They just learn that certain pixel patterns are usually labeled a certain way.
- Even systems celebrated for “training themselves,” like AlphaZero, only succeeded because humans carefully chose the problem format, the input encoding, and the reward signals. Without those design choices, the model would have learned nothing useful.
- while deep learning can outperform older systems in complex tasks, it’s still narrow and opaque. It succeeds when problems are clear, data is rich, and outcomes are easy to measure. But it fails surprisingly fast when pushed into messier, real-world ambiguity.
- This gap between surface performance and actual understanding is why current AI feels both impressive and fragile.
- It often starts when companies latch onto AI as a goal in itself, rather than a tool to solve a specific problem. Instead of identifying a need and then exploring whether AI might help, teams are assembled with the vague mission of “doing something with AI.” This upside-down logic leads to projects that chase the technology rather than the outcome.
- The illusion of rapid improvement is sometimes just the result of cherry-picking results or benchmarks, not genuine leaps in capability.
- because academic careers depend on citations and funding, there’s a strong incentive to exaggerate results and promote the field’s progress.
- because academic careers depend on citations and funding, there’s a strong incentive to exaggerate results and promote the field’s progress.

[Álvaro]

There must be an explanation for this frenzy, a reason why everyone tries to push AI everywhere. I think one of the main reasons is that saying you’re working on AI is a great way of raising funding from private investors and the government with a low level of accountability."

"The idea behind machine learning is that, instead of manually writing every detail of the computer program, the practitioner decides its general shape (the template) and lets the computer automatically fill in the blanks in a useful way."

"Machine learning is not carried out by giving all the data we have to the machine and letting it learn anything it wants. As we’ve seen, giving the computer that much freedom would be highly ineffective for building useful software. As we’ll discuss later on, this holds true even with the most advanced AI built to date."

"To reach AGI, computers would have to match human performance in the most challenging tasks, including language comprehension. As we’ve seen throughout this chapter, machine learning, which is currently the highest-performing type of AI, does not accomplish that. So, the missing piece to reach AGI is not just some practical limitation, say, that computers aren’t fast enough or that we don’t have enough data. Faster computers or more data might be necessary, but they wouldn’t be enough. In order to reach AGI, someone would need to discover a new, unprecedented methodology, since machine learning as it is today falls short. So, what AGI requires is innovation."

"No one knows how to overcome these issues. So, we would need innovation— the discovery of a new methodology—to make the next jump forward. But we cannot predict when innovation will happen. And the fact that innovation has happened recently doesn’t necessarily mean that the next
breakthrough is coming soon."

Son tiempos en los que la información disponible nos rebasa, el rigor periodístico parece desvanecerse y el periodismo mismo pierde su relevancia cultural frente a los memes. Así que la conversación colectiva va brincando de moda en moda impulsada por verdades a medias o mentiras completas.

Gartner tiene una famosa gráfica que explica este fenómeno: una nueva tendencia o tecnología surge, las expectativas alrededor de ésta crecen sin control hasta alcanzar un pico en donde se escuchan predicciones exageradas y las chaquetas mentales al respecto se apoderan de LinkedIn y de las reuniones familiares, para después descender dramáticamente y alcanzar un abismo en el que la mayoría de la gente se olvida por completo de la nueva tendencia, seguido de un crecimiento mucho más silencioso que conduce a un periodo en donde la nueva tecnología finalmente entrega resultados tangibles, realistas y sin mucho ruido. Pasó con la tecnología blockchain, con el metaverso, con las impresoras 3-D y con el internet mismo (que sí, hoy es parte integral de nuestras vidas, pero en algún momento fue una burbuja que reventó) - hoy es el turno de la inteligencia artificial. De repente han surgido toda clase de personajes diciendo que si ChatGPT puede escribir cualquier cosa con la voz narrativa de Gabriel García Márquez (cosa que solo diría alguien que o no ha usado el producto o nunca ha leído a GGM), que si en cuestión de meses todos los trabajos van a desaparecer o que si en cinco años se va a alcanzar la inteligencia artificial general y las máquinas se van a revelar en contra de la humanidad.

El autor de este libro tiene años de experiencia en el desarrollo de inteligencia artificial y logra explicar de la forma más sencilla posible la historia de la misma (no, no surgió mágicamente hace un año), sus conceptos esenciales, alabar los impresionantes progresos que se han logrado en el campo, detallar los alcances reales que tiene, revisar algunos de los peligros que existen hoy y aquellos que se avecinan, así como los obstáculos a los que se enfrentan los investigadores y que muchas de las promesas desmedidas que se leen en redes sociales no toman en cuenta. Nadie posee verdades absolutas de nada, pero creo yo que el tipo tiene un perfil bastante más apto para discutir estos temas que, digamos, tiktokers cuya experiencia está centrada más bien en generar vistas.

[Colin Hoad · Rating 5 out of 5]

Great antidote to the AI hype, makes a strong argument for why, despite what you've been told, AI isn't about to solve the world's problems or become sentient and kill us all off. In fact, AI is pretty stupid without the right level of human supervision. Ignore those with a vested interest in AI, like startups looking for VC money or more established tech firms trying to get a share price boost, and don't let the likes of ChatGPT fool you: AI is just another big tech hype cycle, and it will crash and burn like all the ones before it.

[Francis Tapon · Rating 5 out of 5]

Are you worried about the AI apocalypse around the corner?
Fear not.
An AI expert splashes cold water on the idea in this sober book.

He explains, "There is no clear pathway yet toward AGI."

He says it's like nuclear fusion: it's always 10 years away.

Also, the question of whether & how consciousness emerges from computer programs remains unsolved."

It's an excellent book to bring the AI hype back down to earth.

[Morgan · Rating 3 out of 5]

Insightful to a point, and a good counter-balance to the prevailing hype around AI and AGI, Maggiori’s greatest strength is the accessibility of his text. There is some good primer material here, but the book is otherwise largely anecdotal and not particularly well researched (or at least well cited). Worth the read, but only as a starting five books to orient a reader to the AI space. If you’re already familiar, this is pretty low calorie content.

[vidyuth Kini · Rating 5 out of 5]

Superb

I have a lot of books on AI from the highly technical to the more mundane Popular Science and even Esoteric, Sci-fi varieties.

This particular book is easy to read and enjoyable. It explains things simply but accurately, it raises important questions and exposes a lot of the hype.

I would recommend it to anyone, Zero or Hero.

Enjoy!

[Mikhail Filatov · Rating 4 out of 5]

A very interesting “insider” book with a lot of examples of current AI hype -basically, putting AI first before even understanding the task in hand.
I didn’t like that much the last “philosophical” chapter as it’s relying too much on Penrose idea of quantum effects in the brain and also a bit mixes “consciousness” and “AGI”. It’s not necessary for AGI to be conscious, just to be performant.

[Wouter · Rating 4 out of 5]

Although already a bit dated (the book was written during GPT-3.0), it still has ideas that are not debunked. (Generative) AI has improved, but the same issues discussed in the book are still present.

As any in AI book, Maggiori discusses how model works, but focuses more on the training processes, which had some new interesting details.

[Nermien Khalifa · Rating 5 out of 5]

I just read the summary of the book on Blinkist, and it was a super fantastic one.

It reflects the truth of today's AI without bias, just its reality. Today's AI models are just well-optimized mimicry, not real thinkers.

There is something still missing. It could be the hardware as we still cannot match human intelligence or understanding.

[Michael · Rating 5 out of 5]

Thoughtful review by an expert practitioner

Clear and well argued. Enough detail to enable laypeople to see the weaknesses in machine learning. Punctures the hype. A good chapter on sentience claims. Recommended.

[Andrew · Rating 5 out of 5]

Excellent!

Lucid analysis of the state of affairs of this domain. The very act of wishful thinking of many AI researchers stands as proof against the validity of computational model of the mind (where, allegedly, outcome are decided on logical analysis, alone).

[Doug · Rating 5 out of 5]

Sobering look at the hype!!!

This author thinks like I do about this situation - although people like Eliezer Yudkowsky who think as bad as AI is - it's dangerous - still have be concerned instead of blasé.

[Houssem Mallem · Rating 4 out of 5]

Yes chatgpt is useful and can be used in powerful ways but also we should recognize that you should take it with grain of salt. Smart until it's dumb is a great a non-hyped guide of how AI works in technical manner, its philosophical implications and and why its flawed and still far from perfect.

[Ravi Sinha · Rating 5 out of 5]

Lovely little book. Call a spade a spade. Understand the human element behind all AI that is built and hyped. Understand that all AI systems are ultimately a direct reflection of the choices of the humans who built them. Understand that it's all A and not so much I.