Tag
The article argues that AI labs commit fraud by advertising high benchmark scores from ideal model versions while shipping heavily degraded versions (e.g., quantized, safety-stacked) that perform 50-60% worse, and proposes mandatory third-party re-benchmarking as a solution.
The article discusses how AI agent performance degrades over long sessions due to context window clutter from raw history, tool outputs, and repeated reasoning, and suggests solutions like summarizing old turns and trimming tool outputs to extend useful run length.
This paper introduces 'pigeonholing,' a phenomenon where bad prompts cause LLMs to collapse and repeat errors, leading to a 38-40% performance drop. Experiments across 10 tasks and 10 models show worsening with more conversation turns, and propose RLVR with synthetic errors as a mitigation.
This thread presents a comprehensive guide to context engineering for AI agents, explaining why context management is critical for agent performance and how to optimize token usage to avoid degradation.
The paper proposes a Shannon Scaling Law that models LLM training as information transmission over a noisy channel, explaining non-monotonic performance phenomena like catastrophic overtraining and quantization-induced degradation, and demonstrating superior predictive accuracy over traditional scaling laws.
This paper identifies 'library drift' as a silent failure mode in self-evolving LLM skill libraries, where unbounded skill accumulation causes retrieval degradation and performance stagnation. It provides trace-level diagnostics and a verified governance recipe that lifts pass@1 from 0.258 to 0.584 on MBPP+ hard-100.