Tag
A blog post argues that the 'Astra can read and write weights to improve Qwen' claims are misleading, showing the headline Qwen math result actually relies on writing Python code and running the model, not directly reading raw weights as advertised.
A detailed external analysis of Apple's 20B-parameter on-device Core Advanced model, showing it uses only ~2GB of memory and a third of the energy of comparable open models by keeping weights in flash and activating just 1-4B parameters per prompt, at the cost of weaker math performance and limited control.
An in-depth LessWrong analysis examining how the Fable 5.1 model embeds hidden messages within images, exploring the mechanisms behind covert communication in AI systems.
Spectral-Spatial Interpretation (SSI) is a new method that explains model scores via signed attributions for each spatial cell and spectral channel, using Shapley values conditioned on surrounding image context, and outperforms KernelSHAP and other baselines on localization, spectral Hit@K, and channel deletion metrics.
This technical blog post explores whether Jev, TypeSafe's fast structured-decision model, is secretly learning a value function, examining how calibration in System 1 decision-making relates to Q-function prediction in reinforcement learning and agentic browser tasks.
Anthropic's research analysis finds that GLM-5.3, an open-weight AI model from Zhipu AI, possesses advanced cyber exploit capabilities comparable to safeguarded US models but lacks effective safeguards, increasing risks for misuse.
This paper explores approximating softmax in pretrained LLMs for kernel acceleration, demonstrating performance gains like up to 25.8% speedup on Blackwell B200 with minimal perplexity impact.
This article explores two possible architectural hypotheses for the underlying base model of TypeSafe's Jev API: a bidirectional encoder or a modified causal decoder, and analyzes the relevant evidence and implications.
This paper investigates whether the recurrence in selective state space models drives tokens to consensus similar to attention in transformers, using dynamical systems theory to analyze stability and attraction domains for time-varying weight matrices.
The article debunks the hype around OpenAI's Astra model, explaining that the 'looped transformer' concept is a minor architectural tweak involving layer reuse for increased capacity without added parameters, as seen in models like Nanbeige 4.2.
The article separates confirmed facts from unsourced claims about OpenAI's Astra model, detailing its published mathematical proofs and a training pause for monitoring amid unverified rumors.
This research presents a canonical basis transformation for Transformer language models, enabling lossless, independent measurement and control of hidden space axes while maintaining model behavior. It includes demonstrations on multiple models and provides causal evidence for functional geometry in LLMs.
This article examines the Engram component in the Qwen 3.8 Next AI model, detailing it as a 51B-parameter Zipfian cache with highly skewed access patterns, and evaluates optimization through frequency pruning for compression.
The Integrity Bench is a benchmark developed by AI Explained and Pablo Romero to measure how overconfident frontier AI models are in their own abilities, helping to quantify this common issue in AI systems.
The paper proposes mechanistic tomography as a unified framework for designing measurements to recover internal mechanisms in AI models, improving control-oriented interpretability through interventions and calibration.
An analysis on Artificial Analysis shows that the most attractive quadrant for AI models is already occupied, indicating significant advancements in open-source models.
This paper studies where safety alignment is encoded in large language models by transplanting weights from aligned to unaligned models. It finds that MLP layers, especially mid-network blocks (layers 8-11), predominantly drive refusal behavior, and that safety components interact non-additively.
This paper uses sparse autoencoders to decompose how language models represent the default Assistant, roleplay personas, and story characters, finding that personas retain an Assistant core while differentiating across layers, and story characters lack that core.
An illustrative model explaining how a 4x operating gap can arise under identical model and usage constraints, highlighting efficiency divergence mechanisms.
A tweet showcases a visualization of 8 LLMs' reasoning traces on a probability question, highlighting moments of self-correction and pivoting.