FLEET: From Logits Entropy to Enhanced Trajectories in Text Generation

Hugging Face Daily Papers Papers

Summary

FLEET introduces a memory mechanism to text generation in large language models, using logits entropy to enhance trajectories, resulting in improved accuracy and a 3x speedup, particularly on coding tasks.

Solutions based on large language models (LLMs) often rely on temperature sampling to improve accuracy and stability by aggregating multiple samples from the completion distribution. However, this memoryless approach is inherently suboptimal: because it lacks awareness of prior generations and their evaluations, it produces an increasing proportion of semantically duplicate answers as more samples are drawn, leading to diminishing returns. To address this limitation, we introduce FLEET, a novel method that integrates a memory mechanism into the generation process. FLEET represents each generation as a sparse trajectory through states whose entropy exceeds a predefined threshold and uses these trajectories to infer per-token utility scores that adjust the logits. Benchmark evaluations demonstrate that FLEET achieves the same accuracy as the repeated sampling baseline, with a 3x speedup, and substantially improves accuracy on complex coding tasks (LiveCodeBench Pass@32 increases from 59.9% to 66.2%) under the same budget. Furthermore, in the greedy-decoding configuration evaluated here, the approach is deterministic and uses a single calibration pass to derive its principal hyperparameters, requiring only minimal modifications to existing LLM pipelines.
Original Article
View Cached Full Text

Cached at: 09/24/26, 11:39 AM

Paper page - FLEET: From Logits Entropy to Enhanced Trajectories in Text Generation

Source: https://huggingface.co/papers/2609.27657

Abstract

Solutionsbasedonlargelanguagemodels(LLMs)oftenrelyontemperaturesamplingtoimproveaccuracyandstabilitybyaggregatingmultiplesamplesfromthecompletiondistribution.However,thismemorylessapproachisinherentlysuboptimal:becauseitlacksawarenessofpriorgenerationsandtheirevaluations,itproducesanincreasingproportionofsemanticallyduplicateanswersasmoresamplesaredrawn,leadingtodiminishingreturns.Toaddressthislimitation,weintroduceFLEET,anovelmethodthatintegratesamemorymechanismintothegenerationprocess.FLEETrepresentseachgenerationasasparsetrajectorythroughstateswhoseentropyexceedsapredefinedthresholdandusesthesetrajectoriestoinferper-tokenutilityscoresthatadjustthelogits.BenchmarkevaluationsdemonstratethatFLEETachievesthesameaccuracyastherepeatedsamplingbaseline,witha3xspeedup,andsubstantiallyimprovesaccuracyoncomplexcodingtasks([email protected]%to66.2%)underthesamebudget.Furthermore,inthegreedy-decodingconfigurationevaluatedhere,theapproachisdeterministicandusesasinglecalibrationpasstoderiveitsprincipalhyperparameters,requiringonlyminimalmodificationstoexistingLLMpipelines.

View arXiv pageView PDFGitHub0Add to collection

Get this paper in your agent:

hf papers read 2609\.27657

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.27657 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.27657 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.27657 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation

Hugging Face Daily Papers

This paper introduces Trajectory-Shaped Discrete Flow Matching (TS-DFM), which replaces blind stochastic jumps with guided navigation to significantly improve text generation efficiency and reduce computational costs. The method achieves superior perplexity and speed compared to traditional multi-step baselines while maintaining unchanged inference costs.

One-Step Text Generation by Seq-Drifting

arXiv cs.AI

This paper proposes Seq-Drifting, a framework that learns a map from Gaussian noise to text in a single forward pass by applying drifting to sequences of continuous token representations, achieving a generative perplexity of 36.01 on LM1B with large speedups over few-step and multi-step diffusion language models.

Adding memory to search instead of sampling in reward maximization tasks [R]

Reddit r/MachineLearning

A preprint introduces FLEET, an algorithm that makes Best-of-N sampling reward-aware by attributing external rewards to tokens, storing hidden states with reward metadata in a vector store, and using modified MCTS to reweight logits in subsequent iterations. Results show equivalent or better performance with far fewer sampling iterations on GSM8K and LiveCodeBench with Llama 3.2 3B, plus a reusable metadata store that can serve as a prior for other tasks or enrich SFT/RL.