Tag
Introduces PAJAMA, a hybrid evaluation system that improves upon the LLM-as-a-judge approach by extracting rubrics and executing them programmatically, pushing the Pareto frontier of speed, cost, and transparency.
A site called MiniPCs.zip charts thousands of Mini PCs by benchmark and reveals the Pareto frontier to help users get the most compute per dollar, using Gemini to extract specs from listings.
SwiftCTS is a physics-informed surrogate framework that uses gradient-boosted ensembles and few-shot calibration to rapidly predict and Pareto-optimize clock tree metrics (power, wirelength, timing skew) across unseen designs, achieving high accuracy with minimal training data.
ATOM is a multi-agent framework that formulates molecular optimization as a tree-structured search with specialized agents along paths, enabling exploration of alternative molecular trajectories and improving Pareto coverage in multi-objective benchmarks.
This paper systematically studies hybrid multi-agent systems combining cloud-based LLMs and on-device SLMs, revealing task-dependent optimal architectures and challenging the assumption that more frontier compute always improves performance.
OpenBMB releases MiniCPM5-1B, a leading 1B open weights LLM that achieves the highest Artificial Analysis Intelligence Index score (17.9) in its size class, surpassing larger models like Qwen3.5 2B while using fewer parameters.
Mosaic is a probabilistic weather model that matches state-of-the-art skill while generating a 24-member, 10-day global forecast in under 12 seconds on a single H100.
Practical findings from auditing a production customer support RAG system reveal that heuristic evaluators give false signal, retrieval bugs often masquerade as LLM failures, and the Pareto frontier for cost and quality is often not where expected. Sweeping models showed that replacing the incumbent (Gemini Flash Lite Preview) with Gemma 4 26B achieved a 19% quality improvement at 79% lower cost.
GEPA is a prompt optimizer that uses natural language reflection to learn from trial and error, outperforming reinforcement learning methods like GRPO and MIPROv2 with up to 35x fewer rollouts across multiple tasks.