Hype and Hallucination: Narayanan and Kapoor Identify the Limits of AI

Robert E. Buntrock reviews a new book on AI hype and shortcomings

Posted on June 23, 2025 in Featured Reviews

Authors:

Robert E. Buntrock
Cover image of AI Snake Oil by Narayanan and Kapoor

As a part of our efforts to support scholarly work on the ethical and creative usage of technology in libraries and higher education, Choice highlights new and upcoming titles on this urgent topic through long-form reviews. In doing so, we hope to emphasize the importance of these issues and promising resources for addressing them. The following review, written by Robert E. Buntrock, considers AI Snake Oil: What Artificial Intelligence Can Do, What It Can’t, and How to Tell the Difference (Princeton, 2024) by Arvind Narayanan and Sayash Kapoor and also appears in the September 2025 issue of Choice and online. We hope these reviews will introduce new titles to our audience for their personal or institutional collections.

By Robert E. Buntrock

Narayanan, Arvind. AI snake oil: what artificial intelligence can do, what it can’t, and how to tell the difference, by Arvind Narayanan and Sayash Kapoor. Princeton, 2024. 360p bibl index ISBN 9780691249131, $24.95; ISBN 9780691249643 ebook, contact publisher for price.

AI Snake Oil is an important and timely book by two experts in the field (both in the computer science department at Princeton Univ.). Reminiscent of Clifford Stoll’s Silicon Snake Oil: Second Thoughts on the Information Highway (1995), the book is described as “a guide to identifying AI snake oil and AI hype.” The introduction sets the stage and provides much of the essential information.

AI—what is it? Artificial intelligence, yes, but what is that? AI has been around for some time (in the early days, an old joke was that AI is more like artificial stupidity), but has publicly hit the fan with the release of ChatGPT from OpenAI in November 2023 and has spawned a tsunami of interest and ink ever since. Generative AI creates written documents based on a “library” of the written word in a style typical of human writing. Some of the first results exhibited growing pains and were often quite humorous. The program had more than a million users in the first two months after its release. OpenAI was surprised by the sudden rush in usage and at first didn’t have the necessary computing capacity to support it. Computer programmers loved it since it could write good code based on only the requirements they gave it. Microsoft licensed the technology and incorporated it into its Bing search engine as a chatbot that could answer questions based on search results. Google followed suit with its own search chatbot first named Bard, then Gemini.

Misuse became widespread. The prime weakness of chatbots is their difficulty with factual information since they’re built from statistical patterns of data from the internet, from which they write remixed text. The authors make the claim that many AI-generated books are on Amazon—often dangerous pseudo-factual information—although they don’t say how those books can be identified. These programs have costs and risks with their use, which are not always apparent. Their use in education is very controversial, presenting difficulties in identifying their use in student writing and leading to bans.

🔥 Stay up-to-date with LibTech Insights by signing up for our free newsletter. Just one weekly email with our new blog posts, top tech news stories, and other bonus content. Check out some posts from our archive:

Another species of AI is predictive AI, which has been used for some time. Unlike generative AI, it often doesn’t work at all. Application developers use this technology to predict outcomes for people, including predicting recidivism of defendants or predicting if job applicants would do well on the job. Predictions of needed stays in hospitals for Medicare patients are particularity erroneous, often with tragic results. Allstate insurance company notoriously tried to create a “sucker list” of clients who could tolerate increased insurance rates. Senior citizens were drastically overrated.

It’s difficult to determine whether or not something is AI. Three questions may help: (1) “Does the task to be performed require creative effort or training for a human to perform?” (2) Is the system behavior due to a programmer’s code, or does it come from “learning” through database analysis? (3) Are the resulting decisions autonomous and environmentally adaptable? If the answers are yes, the program may be considered AI, but other factors complicate labeling it.

The authors describe other “sins” of AI usage. Facial recognition is not infallible, despite how it’s depicted in TV crime shows, but even if it works well, it can be harmful and possibly abused by governmental agencies. Textbook errors from the use of AI are very common, as are fallacious research results, especially in biomedical studies. The authors have found it difficult to locate publishers for their studies on incorrect predictive AI. Although science claims to be self-correcting, it’s extremely difficult to publish corrections.

The 35-page introduction is a good abstract, outlining the rest of the book, and may be sufficient for many readers. The “snake oil” analogy recalls the widespread advertising of “cure-alls,” some of which were literally snake oil. The goal of the book is to determine AI snake oil, highlighting cases where, even if it works well, it can be abused, including when its intent is to replace human expertise rather than enhance it. It has a graph of AI snake oil, with hype and snake oil plotted on the vertical axis and benign and harmful plotted on the horizontal axis. Nine uses are charted and discussed further in succeeding chapters.

The body chapters begin by analyzing predictive AI. Chapter 2 discusses the rise of automated decision-making through predictive AI, focusing on its often harmful failures, and questions whether decision-making can be performed without it. Chapter 3 discusses why predicting the future is so difficult. The prime reason is the inherent difficulty of predicting human behavior. The evolution of pandemics is also discussed as well as the limits of predictive AI.

Generative AI is even more complicated than predictive AI, and both its beneficial and harmful applications are analyzed in chapter 4. False claims are often made about accurately labeling essays as AI-generated. However, the authors also discuss potential valid uses of generative AI. Chapter 5 covers the existential risk of AI use. Will the advanced growth of AI become uncontrollable? The authors think not, due to the measured evolution of AI (“ladder of generality”), but acknowledge the seriousness of the issues. Current claims of out-of-control AI are based on false premises. Social media is examined in chapter 6. The authors analyze controversial uses of content moderation, specifically the debate between free speech and the deletion of harmful content. Regulation of free speech already exists without AI, and accountability is often lacking.

The closing chapters offer broader considerations. Chapter 7 discusses the pervasive myths about AI and the ability of academia and the press to counteract misleading and fallacious claims made by the producers. Chapter 8 is titled “Where Do We Go from Here?” The three directions are (1) set ground rules for the construction and advertising of products, (2) create paths to integrate AI into society, and (3) concentrate on the demand for AI rather than the supply. How AI is used in education is especially critical.

Summing up: Recommended. General readers through faculty. Consulting the related Substack is recommended, especially for use in a course.

Subject: Science & Technology – Information & Computer Science

Robert E. Buntrock, Ph.D. in chemistry, is a former laboratory chemist with five decades of subsequent experience as an information specialist for Amoco Corp. and in private practice. He has reviewed more than 200 books for several venues, including Choice.

Subscribe to the LibTech Insights newsletter 🌟

Related Posts