Tag
In a simulated business challenge called Vending Bench, AI models were given $500 and a vending machine to operate for a year. GPT-6 Sol achieved $14,428 in profit, nearly matching Astra's performance at one-eighth the cost.
GLM-5.3 outperforms Space Bunny Alpha in a 5-scene physics test conducted by AI/ML API, highlighting strengths and weaknesses in AI physics simulation.
Elon Musk tweeted a response to user Auggie, who reported that Grok 4.7 scored 100% on a music error detection test, matching GPT-6 Astra's performance.
This article evaluates Jev, a System One model from TypeSafe AI, as a new agent evaluator, showing it outperforms LLM judges in consistency, speed, and cost.
Fusion, a new AI model from a Singapore lab, demonstrates strong performance in SVG generation with accurate structural connections and significant cost savings compared to Opus 5 from Claude.
A 7.9B MoE model runs at 152 tokens per second on an M4 Pro with 64GB unified memory, enabling offline processing of sensitive contract data and demonstrating the practical use of local AI.
This article benchmarks various engines for running the Qwen 3.8 27B model on macOS, comparing their speed and performance in agentic coding tasks, and recommends MTPLX or llama.cpp with MTP for best results.
RunAnywhere releases an on-device AI benchmark app that lets users measure model performance directly on their own phones, replacing posted numbers with real-time testing.