@tunguz: Here is one big reason why this matters. Time spent on non-LLM inference tasks is only going to increase. However, tool…
Summary
A post highlights that 42% of time in modern agentic coding is spent on CPU-based tool use, which is inefficient and presents a major opportunity to redesign these tools for AI agents.
View Cached Full Text
Cached at: 05/24/26, 12:13 AM
Here is one big reason why this matters. Time spent on non-LLM inference tasks is only going to increase. However, tools that these AI system use are very inefficient and have been built from the ground up for CPU and human use. There is a huge untapped opportunity there to significantly improve those processes with AI agents in mind from the ground up.
SemiAnalysis (@SemiAnalysis_): FACT ALERT 🚨 : In modern agentic coding, 42% of the time is spent on CPU doing tool use such as editing files, running Bash scripts, running lints, etc. The economy of traditional cloud computing charges at $ per cpu core. In the economy of agents, the business model is $ per
Similar Articles
LLMs and performative productivity
A developer reflects on using AI agents and questions whether the apparent productivity gains are genuine or merely performative, noting that while tasks are completed faster, deep understanding and real value may be lost.
LLM Agents Already Know When to Call Tools -- Even Without Reasoning
This paper introduces When2Tool, a benchmark to study when LLM agents actually need to call tools, and reveals that models already know tool necessity from hidden states but fail to act. The proposed Probe&Prefill method reduces unnecessary tool calls by 48% with minimal accuracy loss.
@bibryam: How I use LLMs as a staff engineer in 2026 https://seangoedecke.com/how-i-use-llms-in-2026… The biggest AI workflow cha…
A staff engineer describes how LLM agents have evolved by 2026 to become reliable collaborators for coding, debugging, and codebase research, while humans retain responsibility for judgment and review.
Beyond LLMs: Why Scalable Enterprise AI Adoption Depends on Agent Logic
IBM Research explores how agent logic—software primitives like knowledge graphs and program analysis—can guide LLM-based agents to efficiently handle complex enterprise workflows, reducing hallucinations and costs while improving outcomes.
Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems
This paper presents a method for compiling repeated standard operating procedure steps into validated, versioned tools before deployment, replacing inference-time code generation. In a fulfillment center alarm-triage system, this approach reduces p50 latency by 42% and end-to-end error rate by up to 53%.