Tag
The article compares Claude Sonnet 4 and GPT-5.5 by generating code for a Flappy Bird game, demonstrating significant progress in LLM capabilities over 1.5 years.
This paper explores the impact of different candidate-generation schedules on the energy consumption and performance of large language models during test-time scaling, demonstrating that larger batch sizes reduce energy use and latency.
An AI agent using the Qwen3.8-27b model from Alibaba achieved a 56.7% success rate in the AndroidLife benchmark, running 60 real tasks on a OnePlus phone with detailed performance and efficiency metrics.
Compute:Arena is a community-submitted benchmarking platform for evaluating the performance of local AI models across various hardware, runtimes, and quantizations.
SoL-Pi introduces a method for recursively scaling auto-research loops in coding agents, achieving significant token and cost reductions while maintaining performance on benchmarks.
SiliconBench evaluates nine Apple Silicon LLM serving engines on speed, memory, and fidelity, finding that explicit memory budgets don't ensure headroom and only a few stacks meet all criteria for concurrency scaling and model coverage.
This article presents a ranking of AI models based on adjusted scores from Scale Labs' SEAL testing, with Fable-5.1 leading the list.
This article provides an in-depth review of ZCode, a desktop coding agent by Z.ai powered by the GLM-5.3 model, evaluating its features, workflow, cost, and performance on real projects.
This article presents detailed test results comparing the performance of Qwen3.8-Flash-Next-NVFP4 and Qwen3.8-27B-FP8 AI models across various tasks, highlighting that Flash-Next is faster with fewer failures but struggles with multi-step symbolic work.
This paper compares tensor parallelism and KV-cache compression techniques for memory-bound LLM serving, finding that compression is generally cheaper and discusses decision rules based on model size relative to device memory.
A comparison of dense and Mixture of Experts (MoE) models to assess if MoE can match dense model quality in agentic tasks, based on benchmark tests with various models showing performance metrics.
This study develops an ambiguity taxonomy to evaluate large language model performance on clinical registry abstraction from unprocessed EMR data, finding that LLM accuracy is significantly lower than human abstractors and declines as task ambiguity increases.
Gemini 3.7 Flash takes first place on the new AA-AnalystAgent benchmark, excelling in accuracy (60% pass^5), speed (1.32s per task), and cost efficiency across 80 real-world quantitative analysis tasks in multiple domains.
The study analyzes the performance and algorithmic design of methods created by LLM agents compared to human-designed methods, finding that agents can occasionally match or surpass human state-of-the-art but largely recombine existing algorithmic choices.
The article critiques the lack of systematic benchmarking for AI models in different quantization formats, highlighting discrepancies between benchmark results and real-world usage, and calls for more thorough evaluation.
The article compares the real-world performance of GLM-5.3, DeepSeek V4 Pro/Flash, and Gemini 3.7 Flash, recommending Kimi K3 for complex tasks, DeepSeek V4 Flash for general use, and others for specific roles like cybersecurity.
The article reports on AI model performances, indicating that Qwen3.8 (27b) outperforms GPT-5.6-Terra (Max) in agentic tasks based on the Agentic Index scores.
ASTRA is an end-to-end training simulator for air traffic control operators that automates sim pilot roles using locally adapted speech models, achieving a significant reduction in word error rates for Singaporean-accented aviation speech and incorporating AI-assisted performance evaluation.
A detailed evaluation of Deepseek V4's 1M token context window across production codebases reveals optimal performance at 150-250k tokens, with degradation past 300k and significant latency in reasoning mode. The model exhibits high hallucination rates on unknown tasks, requiring validation layers for production use.