Position: LLMs Can't Jump
Summary
A position paper arguing that large language models have fundamental limitations, using the metaphor 'can't jump' to highlight gaps in reasoning or generalization.
View Cached Full Text
Cached at: 08/05/26, 01:53 PM
Similar Articles
A fundamental flaw leaves LLMs strikingly vulnerable to attack
Researchers present a paper at ICML arguing that a fundamental flaw in how LLMs identify instructions makes them impossible to fully secure against attacks, demonstrating successful exploits against models from OpenAI, Anthropic, Alibaba, and DeepSeek.
Language models struggle with compartmentalization
This paper investigates compartmentalization in LLMs, where models fail to share statistical strength across distinct representations of the same concept, leading to reduced sample efficiency and model capacity. The authors demonstrate this phenomenon in multilingual and multi-format settings and show that synthetic parallel data does not fully resolve it.
Rant: Stop saying LLMs are just “next token predictors.”
A critique of the oversimplified claim that LLMs are 'just next token predictors,' arguing that prediction at scale induces useful representations and capabilities, and that such dismissals confuse objective with learned system.
Positional Failures in Long-Context LLMs: A Blind Spot in Reasoning Benchmarks
This paper identifies a blind spot in long-context LLM reasoning benchmarks: they fail to control task position within the context, allowing positional failures to go undetected. The authors propose Context Rot Evaluation (CRE) to systematically vary task position, filler content, and context length, revealing severe accuracy drops for some models when reasoning tasks are placed in the middle of long contexts.
Position: Uncertainty Quantification in LLMs is Just Unsupervised Clustering
This position paper argues that current uncertainty quantification methods for large language models are essentially unsupervised clustering, measuring internal consistency rather than external correctness, and therefore fail to detect confident hallucinations. The authors advocate for a paradigm shift to ground uncertainty in objective truth.