John Warner

“Large language models do not “write.” They generate syntax. They do not think, feel, or experience anything. They are fundamentally incapable of judging truth, accuracy, or veracity. Any actions that look like the exercise of judgment are illusory. While the term hallucination has come to mean outputs from LLMs that are incorrect or untrue, it is arguably more accurate to say that from the point of view of the LLM, everything is a hallucination, as it has no reference points from which to judge its own production. ChatGPT is fundamentally a “bullshitter” as defined by Harry Frankfurt in his classic treatise on the term (On Bullshit), something “unconnected to concern for the truth.” It’s not that ChatGPT makes stuff up. It has no capacity for discerning something true from something not true. Truth is irrelevant to its operations.”
― John Warner, More Than Words: How to Think About Writing in the Age of AI