Tag
OpenAI announced ChatGPT 5.6 with three models (Sol, Terra, Luna), offering cost advantages over Anthropic's Claude, but benchmarks comparing Sol to Mythos are unconvincing. The analysis suggests subscription users should focus on model quality, where Claude still leads for ambitious tasks.
Pangram Labs explores the internal representations of its AI detection model Pangram 3.3.2, analyzing how the model distinguishes human vs AI text across layers using a balanced dataset of 5,000 documents from various sources.
The author maps the Kullback-Leibler divergence of KV cache quantization for the Qwen3.6-35B-A3B and Gemma4-E2B QAT models.
An analysis of how transparent Google's DiffusionGemma model release is, discussing the implications for AI safety and accountability.
This paper reveals that interleaved speech-text language models implicitly transcribe speech into text in intermediate layers, then predict in text space before converting back to speech, shedding light on internal modality interaction.
This paper analyzes on-policy distillation (OPD), finding that OPD updates are sparse, distributed across layers and FFN-heavy, and retain geometric properties distinct from dense parameter rewriting. The sparse structure is operationally useful, but sparsity-inducing SGD underperforms AdamW due to heterogeneous gradient scales.