If you’ve trained an LLM, you’ve may have seen doom loop, in which a LLM endlessly repeats the same token or sentence, wasting compute and breaking its reasoning process which why people not trusting small open source ai models Written an article on it https://open.substack.com/pub/cmdraman/p/why-your-llm-keeps-saying-the-same…

Why Your LLM Keeps Saying the Same Thing: Doom Loops

Source: https://cmdraman.substack.com/p/why-your-llm-keeps-saying-the-same?triedRedirect=true If you’ve trained an LLM, you’ve may have seendoom loop, in which a LLM endlessly repeats the same token or sentence, wasting compute and breaking its reasoning process which why people not trusting small open source ai models.

Source: Liquid AI Liquid AI, one of the leading AI research companies, recently introducedFinal Token Preference Optimization (FTPO), a simple approach for dealing with these doom loops.

Three key reasons behind these loops.

• **First, overtrained tokens.**During synthetic data generation, certain words or phrases appear far more frequently than others. Whenever model become uncertsin, it chooses those words, thus increasing the chances of repetitive loop.
• **Second, prior context reinforcement.**Once a loop begins, every repeated token becomes part of the model’s context. Since language models condition on previous tokens, those repeated tokens further increase their own probability, creating a positive feedback loop that is difficult to escape.
• **Third, greedy or low-temperature decoding.**Reasoning models uses very low temperatures to produce deterministic reasoning traces. While this improves consistency, it also makes the model repeatedly choose the highest-probability token. If that highest-probability token is part of a developing loop, the model can become trapped.

How did they fix it?

They first detectdoom loopsby finding repeated sequences (more than four repetitions or over 60 repeated characters). Once detected, they identify thefirst tokenthat starts the loop and replace it with one or more valid alternatives. This creates a preference dataset containing theprompt,generated prefix,rejected token, andchosen token(s). They then fine-tune the model usingFTPO, teaching it to make a better next-token decision exactly where the loop begins.

1. Final-token training:Instead of optimizing the entire completion like DPO, FTPO only trains on thedecision pointwhere the doom loop begins—the prompt, generated prefix, rejected token, and chosen token(s).

**2. Multiple chosen tokens:**Rather than rewarding a single replacement token, FTPO uses multiple valid alternatives. This preserves diversity and avoids simply replacing one overtrained token with another. 3. Logit-space regularization:Instead of applying KL divergence after softmax, FTPO computes a KL-like loss directly on thelogits. This keeps updates localized, preventing changes to unrelated tokens.

**4. Two-part regularization:**The chosen and rejected tokens are allowed to change more freely, while the rest of the vocabulary is kept close to the original model. This fixes doom loops without affecting the model’s overall behavior.

After training, the doom-looping rate on our early LFM2.5-2.6B checkpoint dropped from 10.2% to 1.4%. Eval scores improved across the board, attributable entirely to the reduction in looping. The training set teaches the model nothing new about math or code; it removes the failure mode that was preventing the model from reaching answers it could already produce.

We also train Qwen3.5-4B on the Antidoom pipeline, known to produce repetitive loops during reasoning. Its doom-looping rate dropped from 22.9% to 1% under greedy sampling, with eval scores increasing markedly

Antidoom repairs degenerate repetitive behavior commonly seen after training, especially with thinking models. It selectively targets the problematic tokens that begin loops, with minimal collateral damage to the remaining distribution. Results so far demonstrate near-complete elimination of repetitive loops in internal Liquid LFM checkpoints, and also on Qwen3.5-4B.

Discussion about this post

Ready for more?