Training and Inference Dynamics of PLDR-LLMs: Row-Map Collapse, Renormalization, and Predictive Reduction

Hugging Face Daily Papers Papers
training-dynamics inference-dynamics pldr-llms row-map-collapse renormalization predictive-reduction theoretical-analysis

Summary

This monograph develops a unified account of training and inference dynamics in Power Law Decoder Representation language models (PLDR-LLMs), covering exact identities, renormalization, and experimental findings.

This monograph develops a unified account of training and inference in Power Law Decoder Representation language models (PLDR-LLMs). Exact finite work identities decompose changes in the absolute energy of the row-centered learned map into parameter contributions, signed interactions, and numerical observation defects. Positive affine blocking retains restarts at the row-constant face, while the augmented AdamW state supplies the complete dynamical description. Predictive renormalization acts on the complete conditional training law for a single pass over distinct corpus target blocks, retaining optimizer memory, remaining data, schedule, and numerical policy. Autonomous reductions require closure; approximate reductions carry successor and emission errors. Finite-population covariance, matched physical clocks, matrix fluxes, and signed temporal energy connect row dynamics to model-wide observations. Absolute row collapse, relative row concentration, operator stabilization, and predictive accuracy are distinguished. Experiments reveal observer and optimizer dependence, reject the tested autonomous row-state candidates, and support finite conditional prediction and state-specific operator reduction. Independent single-pass families exhibit moving finite fluctuation regions without establishing a thermodynamic critical class. Conditional symmetry, head limits, covariance flows, and readout error budgets specify assumptions needed to transfer scaling laws to inference. The theory separates exact identities, conditional dynamical claims, and finite empirical findings, with proofs, selected formal checks, and compact numerical evidence.
Original Article
View Cached Full Text

Cached at: 09/29/26, 08:15 AM

Paper page - Training and Inference Dynamics of PLDR-LLMs: Row-Map Collapse, Renormalization, and Predictive Reduction

Source: https://huggingface.co/papers/2609.34130

Abstract

ThismonographdevelopsaunifiedaccountoftrainingandinferenceinPowerLawDecoderRepresentationlanguagemodels(PLDR-LLMs).Exactfiniteworkidentitiesdecomposechangesintheabsoluteenergyoftherow-centeredlearnedmapintoparametercontributions,signedinteractions,andnumericalobservationdefects.Positiveaffineblockingretainsrestartsattherow-constantface,whiletheaugmentedAdamWstatesuppliesthecompletedynamicaldescription.Predictiverenormalizationactsonthecompleteconditionaltraininglawforasinglepassoverdistinctcorpustargetblocks,retainingoptimizermemory,remainingdata,schedule,andnumericalpolicy.Autonomousreductionsrequireclosure;approximatereductionscarrysuccessorandemissionerrors.Finite-populationcovariance,matchedphysicalclocks,matrixfluxes,andsignedtemporalenergyconnectrowdynamicstomodel-wideobservations.Absoluterowcollapse,relativerowconcentration,operatorstabilization,andpredictiveaccuracyaredistinguished.Experimentsrevealobserverandoptimizerdependence,rejectthetestedautonomousrow-statecandidates,andsupportfiniteconditionalpredictionandstate-specificoperatorreduction.Independentsingle-passfamiliesexhibitmovingfinitefluctuationregionswithoutestablishingathermodynamiccriticalclass.Conditionalsymmetry,headlimits,covarianceflows,andreadouterrorbudgetsspecifyassumptionsneededtotransferscalinglawstoinference.Thetheoryseparatesexactidentities,conditionaldynamicalclaims,andfiniteempiricalfindings,withproofs,selectedformalchecks,andcompactnumericalevidence.

View arXiv pageView PDFProject pageGitHub0Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.34130 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.34130 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.34130 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions

arXiv cs.LG

This paper investigates when chain-of-thought reasoning is beneficial for LLMs, showing that early-stage entropy dynamics reliably indicate reasoning utility, and introduces EDRM, a lightweight, training-free framework that adaptively selects inference strategies to achieve significant token savings while maintaining or improving accuracy.

Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key

Hugging Face Daily Papers

This paper introduces ScaleLogic, a framework demonstrating that RL training compute scales as a power law with reasoning depth in LLMs. It highlights that logical expressiveness is key to improving downstream transfer and training efficiency.

Learning to Refine Hidden States for Reliable LLM Reasoning

arXiv cs.LG

Proposes ReLAR, a reinforcement-guided latent refinement framework that iteratively updates hidden representations in LLMs before decoding, improving reasoning reliability and efficiency compared to chain-of-thought methods.