We benchmarked MCP vs filesystem access across 20 production-agent scenarios. The filesystem setup cut LLM costs by 27% and latency by 32%

We gave the same agent 20 cross-application tasks using two different setups: Official Slack, Notion and Linear MCP integrations. The same application data synchronized and mounted as files using Locality, which I work on. We kept the agent harness, model, prompts and machines the same. We ran each scenario three times and conducted 180 blind, randomized comparisons of the resulting answers. Compared with MCP, the filesystem setup: Produced higher quality answer in 70% of the blind evaluations. Reduced LLM costs by 27%. Reduced latency by 32%. Required 61% fewer tool calls. Used roughly 40% fewer tokens. The traces suggest that most of these gains came from gathering context, not from differences in reasoning. In one scenario, the agent had to identify product-launch risks by comparing evidence across Slack, Linear, Notion and a Git repository. The filesystem agent searched across those sources using a small set of parallel rg and file operations. During one evidence-gathering stage, those operations took roughly 0.3 seconds. The MCP agent spent about a minute on the same stage, making 21 calls with roughly 30 seconds of tool-call time as it iteratively gathered context. The agents spent similar amounts of time reasoning. The main difference was how they found and inspected the evidence needed to reason. Our takeaway is that a filesystem gives agents one composable interface for searching, filtering and reading across sources. This enables ready discovery of context which can be retrieved at scale. MCP gives them multiple application-specific interfaces, which can create longer retrieval chains for context-heavy work. This benchmark focused on cross-application research and synthesis, so it doesn’t cover every MCP use case. MCP may still be the better interface for individual actions and lightweight integrations. Full analysis, traces and scenario-level results linked below. If you’re running agents in production, do you fetch application context through tools at runtime or synchronize it into the environment beforehand? Where have you seen either approach break down?