Tag
ENPIRE is a framework that enables coding agents to autonomously improve robot manipulation policies through a real-world feedback loop, achieving 99% success on dexterous tasks like pin insertion and zip tie cutting.
Discussion about the significant gap between Llama model benchmark scores and actual real-world performance, with the author seeking assistance.
ENPIRE is a framework that enables autonomous robot policy self-improvement in the real world through a closed-loop system of environment feedback, policy refinement, and evolutionary code optimization, achieving 99% success on dexterous manipulation tasks.
NVIDIA GEAR lab introduces ENPIRE, a framework for autonomous real-world robot policy self-improvement that achieves 99% success on dexterous manipulation tasks like GPU insertion and zip-tying, with multi-robot parallel learning and open-source release.
FrontierCode is a new benchmark for coding agents, human-verified with a continuous scoring model, designed to evaluate real-world performance.
This tweet promotes a free collection of over 300 real ML system case studies from top companies, arguing that toy projects are insufficient for building a strong portfolio.
A discussion about the most useful AI agents actually deployed in production, highlighting simple, single-problem solutions like lead qualification and support triage.
Agent Arena is a new leaderboard that evaluates AI models on real-world agentic tasks such as coding, research, and file analysis, using signals like task success, steerability, and recovery, with GPT-5.5 High leading.
Discussion on whether AI agents are transitioning from impressive demos to genuinely useful tools in research, coding, operations, and personal productivity.
A 6-week real-world experiment using an open-source desktop agent shell with a three-model split (Haiku triager, Sonnet reviewer, Opus executor) reports a 64% cost reduction and details failure modes like context bloat and runaway sub-agents.
The author, working at an AI infrastructure company, observes that running AI agents in production is less about the model and more about environment, access control, isolation, and safe state management, and asks if the community wants detailed architecture patterns.
An agency founder shares lessons from 50+ AI automation implementations, highlighting that most fail due to broken underlying processes, lack of internal ownership, and over-engineering, while the most successful automations are simple, focused, and backed by a named client-side owner.
Apex-Testing, a benchmark for evaluating agentic coding models using real private GitHub repositories, has been updated with recent models and detailed metrics including cost, time, and ELO-based leaderboard.
This paper introduces TerminalWorld, a benchmark for evaluating AI agents on real-world terminal tasks, derived from 80,870 terminal recordings. Current systems achieve at most 62.5% pass rate, highlighting challenges in authentic terminal workflows.
A reflection on the gap between impressive AI agent demos and dependable real-world execution, arguing that current agents excel at structured tasks but fail under unpredictable conditions, suggesting near-term AI roles will focus on narrow automation with human oversight.
A reflection on AI agents: impressive for narrow supervised tasks but fragile and unreliable in long-running, messy workflows due to issues like session expiration, context drift, and silent failures.
Anthropic's Agents team unveiled a production-grade four-layer framework for multi-agent systems during a 30-minute presentation, marking a shift from demo to real-world applications.
Mega-ASR proposes scaling up real-world acoustic simulation to improve automatic speech recognition in challenging, wild conditions, aiming to narrow the performance gap between lab and real-world settings.
DetectRL-X is a comprehensive multilingual benchmark for evaluating LLM-generated text detectors across 8 languages and 6 domains, including stress testing with AI-assisted writing operations and perturbations. It reveals strengths and limitations of current detectors in multilingual scenarios.