Skip to main content

BLOG

From Token Prediction to Dao: The Epistemic Limits of Large Language Models

An examination of the fundamental cognitive limitations inherent in large language models (LLMs) whose core mechanism is next-token prediction, set against the language philosophy of Laozi's "the Dao that can be spoken is not the eternal Dao" and Wittgenstein's proposition that the limits of language are the limits of the world. This article analyzes Yann LeCun's critique of the LLM paradigm and his billion-dollar pivot to world models, traces the resonance between Daoist epistemology and modern AI theory, and demonstrates how the I Ching's hexagram-based cognition — structural mapping rather than linguistic token manipulation — offers a path beyond the inherent ceiling of language-based AI.

The operating principle of contemporary large language models can be stated with stark clarity: given a preceding sequence of tokens, predict the statistically most probable next token. GPT, Claude, Gemini — regardless of architectural refinements, the fundamental cognitive unit remains the discrete linguistic symbol. This approach has demonstrated remarkable capabilities in text generation, translation, summarization, and code synthesis, yet it simultaneously exposes a fundamental epistemological deficiency: what these models capture is the statistical distribution of language, not the structure of the reality that language references. Yann LeCun, Meta's chief AI scientist until his 2026 departure, stated the matter without equivocation: pursuing superintelligence through large language models alone is a dead end. The core problem, in LeCun's analysis, is that language itself is merely a projection of reality — a lossy compression — not reality itself. A language model that perfectly captures the statistical regularities of how humans describe falling objects still possesses no understanding of gravity. In March 2026, LeCun founded AMI Labs with over one billion dollars in seed funding, explicitly pivoting toward "world models" — systems designed to directly apprehend the causal structure of the physical world rather than merely fitting patterns in linguistic data. This pivot carries epoch-making significance in the history of artificial intelligence, because it constitutes an acknowledgment of an ancient philosophical proposition: between the symbol and its referent lies an unbridgeable gap. The map, however detailed, is not the territory. The menu, however eloquently written, is not the meal.

What is remarkable is that this epistemological insight was precisely articulated in Chinese philosophy twenty-five hundred years before LeCun's billion-dollar pivot. The opening line of the Dao De Jing — "dao ke dao, fei chang dao; ming ke ming, fei chang ming" (the Dao that can be spoken is not the eternal Dao; the name that can be named is not the eternal name) — carries philosophical depth far exceeding its common mystical interpretation. It is a rigorous proposition in the philosophy of language: any "Dao" that can be captured by linguistic tokens is necessarily not the generative, dynamic Dao that underlies the ten thousand things. What Laozi identified is precisely what LeCun has rediscovered in the language of computer science: linguistic representation is in principle incapable of exhausting the deep structure of reality. Ludwig Wittgenstein's famous proposition in the Tractatus Logico-Philosophicus — "the limits of my language mean the limits of my world" — is regarded in Western analytic philosophy as the canonical statement on the relationship between language and cognition. Yet the Daoist insight is more radical than Wittgenstein's. It does not merely acknowledge language's limits; it actively points toward a cognitive path that transcends language. Chapter 41 of the Dao De Jing declares: "the greatest sound is silence; the greatest form is formless" — suggesting that the deepest patterns reside precisely beyond the reach of linguistic symbols and sensory surfaces. This is not anti-rational mysticism. It is a serious methodological reflection on cognition itself: when your tool (language) is in principle incapable of reaching your target (Dao, the deep structure of reality), what you need is not a better version of the same tool but an entirely different cognitive framework.

The I Ching provides exactly such a framework. Unlike language models that depend on token sequences, the I Ching employs hexagrams as its fundamental cognitive unit — six-line structural symbols that map directly onto the patterns of change in reality, bypassing the mediation of natural language. When a practitioner receives a hexagram, the cognitive process is not semantic decoding (parsing the meaning of words) but structural mapping: aligning the dynamic structure of the present situation with the archetypal patterns encoded in the sixty-four hexagrams. This methodological approach resonates powerfully with LeCun's JEPA (Joint Embedding Predictive Architecture), whose core idea is to predict the structural dynamics of the world in abstract representation space rather than performing point-by-point prediction at the pixel or token level. In other words, LeCun is attempting to build machines that do what the I Ching has been doing for three thousand years: bypass surface symbols and directly apprehend the deep structure of change. The philosophical lineage is striking. Confucius, in the "Appended Texts" (Xi Ci) commentary on the I Ching, wrote: "Writing cannot fully express speech; speech cannot fully express thought" — and then immediately pointed to the hexagram images (xiang) as the means to express what language cannot. This is a twenty-five-hundred-year-old argument for representation learning beyond natural language, articulated with remarkable precision. KAMI LINE's design philosophy is rooted in this insight. We do not use AI merely as a language generator; we employ the sixty-four hexagrams as a structural world model within which AI performs its reasoning. This means KAMI LINE's output is not the statistically most probable next word but the structurally most fitting hexagram interpretation for the present situation — a return from token prediction to the cognition of Dao.

Latest articles

KAMI LINE's Vision: A Guardian Deity for Everyone

240 deities. One platform. A guardian for every kind of seeking. Here is why KAMI LINE exists and where it is going.

VR Meditation: Can Immersive Technology Deepen Spiritual Practice?

VR meditation is not just guided breathing with better visuals. Some practitioners report genuine spiritual depth. Here is what the research and the traditions say.

Virtual Temples: Can Sacred Space Exist in the Metaverse?

Temple designers, VR developers, and religious scholars are asking the same question: can a digital space hold genuine sacred quality? The answers are more nuanced than you might expect.