Knowing how LLM contexts work and how to work around context limitations – aka “context engineering” – is becoming so important. No better person to explain than @dexhorthy

Timestamps:

00:00 Intro 01:33 Dex’s path into tech 03:34 Early work in platform engineering 05:28 Replicated 11:24 Metalytics 12:36 12-factor agents 18:27 Context engineering 23:38 Harness engineering 26:11 Context overload 30:45 Loop engineering 44:34 Software factories before and after AI 50:33 Automation limits 55:18 Three options for automating 59:00 RPI framework 1:04:16 Intentional compaction 1:11:48 Token harder vs. token smarter 1:16:44 AI slop 1:19:15 HumanLayer 1:29:09 Book recommendation

Brought to you by:

• @AntithesisHQ — with Antithesis, you can use AI agents to work on critical systems without worrying about correctness. Teams like Jane Street, http://Fly.io, and the etcd community use Antithesis to ship better code, faster. https://antithesis.com/pragmatic

• @buildkite  — the CI orchestration platform built for reliable scale. Used by OpenAI, Anthropic, Cursor, Meta, Uber, Ramp, Nvidia, Airbnb and many more. https://buildkite.com/pragmatic

• @sentry — application monitoring software built by developers, for developers. Check out their AI agent, Seer AI, and Sentry MCP. http://sentry.io/pragmatic

Three interesting learnings from this episode:

1. Lesson learned: Shipping unread code spells disaster within months.

Dex experimented with having the model write the code and humans not reviewing anything in July 2025. Four months later, they shut things down and threw the whole system out. Production broke, and no matter how much the team prompted Opus 4.1, the model could not find the root cause. Once fixed, it took three weeks (!!) to re-onboard to a codebase no human had ever read

2. Context engineering 101: figure out where the “dumb zone” begins.

As a rule of thumb, the less of the context window that is used, the better the outcomes are. This is because the attention mechanism is quadratic: the more that goes into the context window, the more compute is required to process it all.

3. “You’re completely right!” or “you’re right to push back on that” are phrases that mean it’s time to start a new session.

These responses mean the LLM session is trajectory-poisoned, and you’re wasting time and tokens to continue. This is because models are autoregressive.

Build fast. Run any code fearlessly.

Source: https://fly.io/ The platform for devs who just want to ship. Powered by sandboxes that let you deploy any code with confidence.

Deploy your app

Modern Compute Without the Complexity

For builders who need compute that keeps up with their ideas, Fly Machines are hardware-virtualized containers that launch instantly and run only when you need them. Deploy an app in minutes or run untrusted code in isolated sandboxes.

Sandboxes That Feel Like a Superpower

Spin up hardware-isolated sandboxes for running AI-generated code in complete safety. Every Sprite is a self-contained environment, ready for action in under a second. Checkpoint your entire environment, run anything, and restore if you break something. Pay only for actual CPU and memory consumption, down to the second.

Check Em Out!

Storage That Keeps Up

Fast local NVMe for low-latency work, plus global, durable object storage for data that needs to stick around. Snapshot environments, persist state, and scale storage right alongside your sandboxes.

Built-In Private Networking

Private networks per sandbox, granular routing, and end-to-end encryption — all automatic.

• VMs That Do Everything You Need Everything runs on Machines: your apps, agents that scale to millions of users, and AI-generated code.

• Fork Off VMs Like They’re Processes Fly Machines start fast enough to handle HTTP requests, run only when you need them, and scale into tens of thousands of instances.

• Built for Distributed Systems Run clustered databases like Cockroach, globally-distributed Postgres, and modern RPC systems like Elixir FLAME. No Terraform required.

• Get Right in Your Users’ Faces Deploy in 18 regions, from Sydney to São Paulo. Sub-100ms response times give users that native-app feel, no matter where they are.

Agents Run Better on Sprites

You don’t have to be a cloud guru to build scalable agents. Run every instance on its own VM with dedicated CPU, memory, networking, and a private filesystem. Forget wrestling with shared runtimes or noisy neighbors. Get persistent brains when you need them, ephemeral workers when you don’t.

Build for scale from Day One

Enterprise-Ready

Apps running on Fly Machines are KVM hardware-isolated, built on a memory-safe stack and running directly on our own metal.

• Single Sign-On
• Guaranteed Support Response Times
• SOC2 Type 2 Attested
• Memory-safe Rust and Go stack
• CI/CD Integration

Build Whatever You Can Think Up (and Give Your Users The Same Power)

Fly.io is the platform that gets out of your way, running on the infrastructure that lets you run any code without fear. See how fast you can get started.

Spin Up a Sandbox