Do LLMs Understand Sequential Structure? A Controlled Study of Inference and Generation
Summary
This paper investigates whether LLMs can recover latent sequential structure beyond surface action frequencies, using controlled Rock-Paper-Scissors interactions and stochastic n-gram continuation tasks. Findings show that longer context doesn't improve strategy identification, recognition doesn't guarantee faithful simulation, and higher-order dependencies degrade rule recovery — suggesting apparent behavioral fidelity can mask incorrect generative mechanisms.
View Cached Full Text
Cached at: 10/09/26, 06:23 AM
Paper page - Do LLMs Understand Sequential Structure? A Controlled Study of Inference and Generation
Source: https://huggingface.co/papers/2610.04977
Abstract
Largelanguagemodels(LLMs)areincreasinglyusedasinteractiveagentsandsimulators,yetitremainsunclearwhethertheycanrecoverlatentsequentialstructurebeyondsurfaceactionfrequencies.Thisdistinctioniscriticalforbehavioralsimulation,whereactionsareoftenshapedbypriorcontextratherthanmarginalfrequenciesalone.Westudythisquestionusingcontrolledtwo-playerRock--Paper--Scissorsinteractionsandaone-playerstochasticn-gramcontinuationtask.Acrosstheseexperiments,wetestwhetherLLMscanidentifylatentstrategies,followsimpleMarkovrules,andsustainhigher-orderconditionaldependencies.Ourframeworkseparatesdistributionmatchingfromconditionalrulefollowing.Resultsshowthatlongercontextdoesnotimproveidentification,correctrecognitiondoesnotensurefaithfulsimulation,andhigher-orderdependenciessubstantiallydegraderulerecovery.Apparentbehavioralfidelitycanthereforemaskincorrectgenerativemechanisms.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2610\.04977
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2610.04977 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2610.04977 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2610.04977 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Sequential statistical inference for Large Language Models: Representation, validity, and monitoring
This paper argues for a sequential inference framework to enhance LLM trustworthiness by modeling interactions as dependent stochastic processes, ensuring validity under repeated use, and enabling online monitoring for behavioral shifts.
Cross-LLM Consistency in Inference: Evidence from Shared Interactions
This paper investigates whether different LLMs share common inference patterns when predicting the same token, using interaction-based explanations. Results show that advanced LLMs exhibit consistent interaction patterns, suggesting implicit optimization toward shared inference mechanisms.
How Well Do LLMs Perform on the Simplest Long-Chain Reasoning Tasks: An Empirical Study on the Equivalence Class Problem
This empirical study evaluates LLMs on the Equivalence Class Problem to assess long-chain reasoning capabilities, finding that non-reasoning models fail while reasoning models struggle with specific structural difficulties.
When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions
This paper investigates when chain-of-thought reasoning is beneficial for LLMs, showing that early-stage entropy dynamics reliably indicate reasoning utility, and introduces EDRM, a lightweight, training-free framework that adaptively selects inference strategies to achieve significant token savings while maintaining or improving accuracy.
LLMs Learn Better In-Context from Rules than from Examples
This paper compares rule-based and example-based in-context learning in LLMs, finding that rules generally lead to more reliable learning across diverse tasks, with implications for model and task properties.