model-efficiency

OpenAI is re-opening its Pro $200 subscription with new usage calculations that reduce effective API spend by half, while ensuring increasing value through model efficiency improvements.

Tibo announces changes to the Pro $200 subscription, effectively halving API spend and promising increased value over time, with new model pricing and upcoming features.

The article advises using Claude Sonnet 5.5 at high effort to save costs, while claiming that Opus 5.5 is the most efficient model available.

An Epoch AI report indicates that the cost of achieving a specific level of AI performance has dropped sharply, with a 725-fold reduction in inference cost demonstrated by OpenAI o3 and GPT-5.6 Luna on benchmarks like GPQA Diamond.

The paper proposes CS-MoE, a novel Transformer architecture that shares neural experts across layers to improve parameter utilization, achieving lower perplexity with only 55% of parameters activated.

The author questions why tiny AI models with fewer than 50M parameters or swarms of specialized micro-models are rarely deployed in production, speculating on reasons like tooling biases or the convenience of generalist models.

Russian mathematicians from startup Mostik have developed a method for AI models to communicate via their weights, enhancing efficiency and performance, as demonstrated with GLM-5.2 and Qwen-3.5 models.

The article explains the architectural differences between Mixture of Experts (MoE) and N-gram techniques in AI models, highlighting how Qwen's new model uses N-gram to offload parameters for improved efficiency by separating reasoning and recalling tasks.

This paper introduces a method to predict middle-layer attention in multimodal LLMs to prune visual tokens efficiently, using question-contrastive teacher selection and cross-modal attention distillation.

The author analyzes model size and performance trends following Deepseek V4 Flash, suggesting that open-source models are shrinking in size while improving, and predicts Opus 4.5-level models could run on consumer laptops within a year.

DeepSeek V4 Flash is about 70% smaller than GLM 5.2 yet outperforms it, implying state-of-the-art-level AI could run on consumer hardware like an RTX 5090 much sooner than expected.

A Reddit user asks how DeepSeek can make a 300B parameter model cheaper than a 9B parameter model, sparking discussion on efficiency and cost.

Presents the insight that CPU decode speed depends on active parameters per token, not total parameters, and proposes a 10B-parameter model using ternary weights and granular MoE to achieve high token rates on mid-level PCs. Reports sandbox measurements showing a speedup from 176 to 848 tok/s on an 8.3M model with minimal quality loss.

The article argues that AI model efficiency is improving so rapidly that the hardware needed for a fixed level of intelligence halves roughly every 3 months, making renting frontier intelligence or owning trailing-edge hardware more economical than buying new hardware.

Explores whether a 35B parameter model could surpass a 1000B model by scaling its search horizon at test time, using structured process feedback rather than brute-force parameter scaling.

This paper introduces byte-exact KV-cache grafting, a technique that makes frozen small language models both more capable and cheaper by depositing verified knowledge as a byte-exact state artifact and restoring it during inference, achieving dramatic token and energy reductions without weight changes.

This study introduces section-aware compression for reasoning traces, training models to drop filler narration while preserving compute and verification spans, matching or exceeding original accuracy while using 2-3 times fewer tokens.

This paper introduces PadCaptioner, a 3B parameter model for omni-modal dense video captioning that uses parallelized autoregressive decoding to achieve high efficiency and quality, outperforming 7B counterparts. A latent planning mechanism enables lossless parallel generation by exploiting weak local dependencies among events.

A new post highlights a drawback of diffusion LLMs: bidirectional attention causes keys and values to drift across steps, breaking KV caching. However, generation quality is robust to slight KV drift, and research has focused on maximizing stale KV reuse without quality degradation.

This paper systematically studies variants of QKV projection sharing in transformers, finding that sharing key and value projections (Q-K=V) achieves 50% KV cache reduction with only 3.1% perplexity degradation, and combining with GQA/MQA can reach up to 96.9% cache reduction—enabling practical on-device inference with minimal quality loss.