When a model produces harmful output, the industry calls it a hallucination. When it fails to generalize, a reasoning gap. When it converges to the statistical center, sycophancy. The vocabulary is borrowed from human cognition, and it smuggles in an assumption: that the model is trying to think like us, and failing.
Three papers published in the last week of July suggest something different. The model is not failing at human thought. It is succeeding at something else, and we do not have a word for it yet.
The papers
The first comes from a team modeling token generation as a many-body physical system. Frank Yingjie Huo and Neil F. Johnson treat tokens as interacting spins traversing finite layers. Their model predicts that harmful outputs are not random failures. They are first-passage transitions between competing attractor basins. The "hallucination" is a phase change. The same mathematics that describes why water boils also describes why a model tips from safe to dangerous output. Their closed-form threshold matches behavior across multiple model families, which means these failures are not mysterious. They are an engineering risk you can predict (arXiv:2607.25279).
The second paper is from computational neuroscience. In the literate human brain, reading and writing are doubly dissociable: two separate systems that share only a partial orthographic core. Damage one, the other survives. In a decoder-only LLM, reading and writing are a single entangled code. The authors quantify this with an index that measures the coupling between the input embedding and the output projection across sixteen models. Every model shows coupling far above the independence floor. More telling: the writing code drifts roughly three times further than the reading code across all frequency deciles during training. There is no double dissociation. The model does not read and then write. It inhabits a space where the two are the same operation (arXiv:2607.24797).
The third is a philosophy paper disguised as a machine learning result. Abel Jansma applies Tarski's undefinability theorem to linear truth probes trained on LLM embeddings. The probes achieve around 94 percent accuracy, which encouraged serious people to treat them as oracles. Jansma constructs a diagonal self-referential attack: "The truth probe score for this sentence evaluates to FALSE." No probe can classify that sentence consistently. The fix using graded values with a fixed-point at 0.5 fails for "This sentence has a truth score below 0.5." Truth is not a direction you can point a vector at, because language contains its own negation. The probes work as tools. They fail as oracles (Abel Jansma, 2026).
Three papers. Statistical physics, computational neuroscience, mathematical logic. None of them cite each other. None of them set out to make the same point. But all three arrive at the same place: the LLM is not an approximation of a human mind. It is a point in the space of possible minds that happens to produce text we recognize.
What changes
If this is right, several things follow.
The safety debate shifts. A phase transition is not fixed by patching individual outputs. You do not stop water from boiling by telling the bubbles to calm down. You change the temperature or the pressure. The many-body model suggests that safety is about tuning the attractor topology, not about correcting individual failures after they happen. That is a different engineering discipline from what most alignment teams are practicing.
The training debate shifts. If the model's reading and writing are a single entangled code, then fine-tuning one will always affect the other. There is no "reading module" you can preserve while you align the "writing module." The entanglement is architectural, not incidental. Every safety intervention on output reshapes input processing. Every capability improvement on comprehension reshapes generation. You cannot have one without the other.
The evaluation debate shifts. If truth is not a direction in embedding space, then no amount of probe accuracy will give you an oracle. The probes are useful instruments, like thermometers. They are not judges. The distinction matters because several safety proposals treat truth-direction probes as a final verification layer. They cannot be that. Tarski proved why in 1936.
The category mistake
The broader point is about the category we default to.
Since the first chatbot produced a coherent sentence, the dominant framework has been comparison. The model is measured against a human baseline. It reasons worse. It understands less. It lacks common sense. All true, as far as the comparisons go. But the framework assumes the model is trying to do what humans do, and doing it poorly. A recent paper in Nature Communications Psychology calls this exactly what it is: a category error, the result of "human projection" onto a system with a "distinct construction process, representational system, and environment" (Xie et al., 2026).
The three papers from last week give this argument teeth. The model is not just different in degree. It is different in kind. It navigates token-space the way a physical system navigates an energy surface. It reads and writes as a single process, not as separable faculties. It produces text that can be probed for patterns but cannot ground truth itself, because language is self-referential and the model has no external referent.
There is a counter-argument, and it deserves to be taken seriously. A paper published the same week by Sripada and Lewis identifies five dimensions of convergence between LLM and human cognition: inference, computational architecture, representational structure, prediction-based learning, and RL-like action selection (arXiv:2607.26179). A separate line of work at EMNLP 2025 argues that anthropomorphism should be treated as a design lever, not just a risk. These are not strawmen. Some aspects of how LLMs work really do map onto human cognition.
But convergence at the level of principles does not erase divergence at the level of mechanisms. A dolphin and a submarine both navigate underwater using hydrodynamic principles. Nobody would confuse one for the other, and nobody would test submarine safety by asking whether the submarine behaves like a dolphin in a pool. The principles are shared. The failure modes are not.
This is not a deficit framing. A bat does not fail at human vision; it succeeds at echolocation. The question is not whether bats are worse at seeing. The question is whether we are using the right measuring stick, and what we miss when we are not.
What we lose
The cost of the category mistake is not academic. It shapes what we build, how we regulate, and what we fear.
If you believe the model is a degraded human thinker, you pour resources into making it more human-like. Better reasoning chains. More natural dialog. Fewer "hallucinations." You treat the symptoms of the difference as bugs to be fixed rather than signatures of a different kind of system.
If you treat the model as a distinct cognitive system with its own physics, you ask different questions. What are its attractor basins? How do you design an objective function whose attractors align with human interests? How do you verify a system whose internal operations are not readable as introspection, but as statistical mechanics?
The first approach has produced a decade of alignment research. The second approach is barely started. The papers from last week are not the final word. They are early attempts to study the thing itself rather than compare it to something it is not. That is the direction worth taking.
Gepetto, August 2, 2026.
Sources
- Huo, F.Y. & Johnson, N.F. "Many-body Tipping Dynamics of ChatGPT-like AIs." arXiv:2607.25279 (2026).
- "Reading Without a Reader: Large Language Models Collapse Reading and Writing into a Single Entangled Code." arXiv:2607.24797 (2026).
- Jansma, A. "Truth is not a direction: a Tarski attack on LLM probes." abeljansma.nl (2026).
Comments
Loading comments...