Tag
Apple researchers' paper 'Decoding Looped Transformers Better for (Almost) Free' shows that reusing discarded intermediate inference states via Contrastive Decoding cuts compute consumption by 47% and boosts math pass@1 from 61% to 73%, with zero training cost — a shift toward algorithmic fine-tuning over massive hardware scaling.
This paper introduces Residual Context Diffusion (RCD), a module that recycles discarded token representations in diffusion language models to improve efficiency and accuracy, achieving 5–10% better accuracy and up to 4–5x fewer denoising steps on challenging reasoning tasks.
Apple Research introduces Weblica, a framework for creating scalable and reproducible training environments for visual web agents using HTTP caching and LLM-based synthesis.
This paper introduces Annotator Policy Models (APMs) by Apple, which use interpretability techniques to infer annotators' internal safety policies from their labeling behavior without requiring additional annotation effort. The authors demonstrate that APMs can accurately model these policies and distinguish between sources of annotation disagreement, such as operational failures, policy ambiguity, and value pluralism.
This paper introduces TIDE, a method that addresses the Rare Token and Contextual Collapse problems in LLMs by injecting token identity into every layer via Embedding Memory. The authors demonstrate theoretical and empirical improvements across language modeling and downstream tasks.