Tag
TomasuLLM 是一个将乱序推测执行思想引入 LLM Agent 的运行时，让工具调用在隔离的写时复制沙箱中乱序执行并按序提交验证，从而在 SWE-bench Verified、Terminal-Bench 2.0 和 SWE-Marathon 上分别提升 1.31x、1.35x 和 1.27x 的性能，且在 4,010 条提交验证记录中零误报。
VUSec presents Branch Target Reuse (BTR), a new Spectre-v2 attack exploiting stale branch predictor entries in JIT engines (cBPF, GraalVM, SpiderMonkey) across Intel, AMD, and ARM CPUs, including an end-to-end Linux kernel exploit that bypasses existing mitigations to leak kernel memory.
This paper introduces Speculative Macro Commit (SMC), a two-tier agent system that reduces latency in tool-using LLM agents by speculatively executing future action chains and committing them when matched, achieving speedups over sequential execution.
The article proposes speculative programmatic tool calling (sPTC), a technique to optimize tool calls in AI harnesses by overlapping execution with token generation, reducing latency inspired by speculative execution in CPUs and LLMs.
Introducing Speculative Programmatic Tool Calling (sPTC), a technique to speculate on tool calls during code generation to overlap with token generation and execution time, improving efficiency in AI harnesses.
Uber open-sourced SubmitQueue, a high-performance speculative merge queue that validates multiple changes in parallel against predicted future states of HEAD to keep trunk green at scale.
A paper introducing predictive speculative KV replication to handle bursty LLM inference workloads, with code available on GitHub.
This paper introduces memory-augmented speculative execution for LLM agents, using three online memory systems to improve prediction accuracy by 19-39% on action prediction and up to 2.5x on observation prediction, all while being lossless with zero added wall-clock cost.
GPUHedge is an open-source tool that uses speculative execution to hedge between serverless GPU providers, reducing cold start p95 latency from 117s to 30s.
SpecHop is a continuous speculation framework that accelerates multi-hop retrieval agents by maintaining multiple speculative threads and verifying predictions asynchronously, achieving up to 40% latency reduction without changing final model outputs.
Accio is a speculative execution framework that reduces cost and latency for web agents by leveraging offline site-structure profiling and online selection of fast paths, achieving a 1.9x reduction in per-task cost and 33.4% latency reduction while maintaining accuracy.