Karpathy’s Agentic Engineering finally has proper DevTools!

When an agent stops working, the model is only one possible cause.

The problem could be a failed tool, a lost connection, an interface update that never appeared, or something earlier in the conversation.

@CopilotKit has rebuilt its open-source Inspector around this problem.

It sits inside the application and watches the full interaction between the user, the interface, and the agent.

When something fails, the Inspector button turns red, names the failure, and opens the spot where it happened.

But even a much harder problem is reproducing that failure.

This is because agents do not always follow the same path twice. Running the same prompt again may trigger another tool, produce another response, or start with a different application state.

Inspector handles this through saved Threads and an isolated Agent Playground.

A developer can open the conversation where the problem occurred, choose “Try from here,” and copy everything up to that point into the Playground.

The original conversation remains unchanged while another response or tool path is tested.

Threads can open inside the live application, while assistant messages can jump back to their matching context in Inspector.

Everything runs underneath AG-UI, the protocol that carries messages, tool activity, state changes, and other events between the agent and the interface.

Those interactions can also feed CopilotKit Intelligence so that when the same issue or useful behavior appears repeatedly, it can become an evidence-backed Insight and a proposed SKILL(.)md file.

Developers can review, edit, and approve the improvement before the agent inherits it.

The tool also helps developers find a problem, recreate its surrounding context, test another path, and turn repeated lessons into agent improvements.

It is open source and included with CopilotKit development builds. Intelligence setup starts with one prompt.

Here are the docs: https://docs.copilotkit.ai/inspector

I am testing this extensively and will cover this in more detail soon with a hands-on demo.

Inspector

Source: https://docs.copilotkit.ai/inspector

What it shows#

The CopilotKit Inspector is a built-in debugging tool that overlays on your app. The first open lands onHome. Later opens return to the last pane you used.

FeatureDescriptionHomeProject, runtime, services, and CopilotKit news.ThreadsInspect real Threads and their messages, AG-UI events, and state.MemoryInspect long-term memory when Intelligence exposes it.AG-UI EventsView the raw AG-UI event stream between your frontend and agent in real time.Available AgentsSee which agents are connected and available to your app.Agent StateInspect your agent’s current state as it updates.Frontend ToolsSee what tools you’ve defined on the frontend and their parameter schemas.ContextView the context you’ve provided to the agent, including readables and document context.Project ContextSee trusted organization, project, plan, Thread usage, expiry, and license-aware actions when the Runtime supplies them.## Navigation and Threads#

The sidebar has three groups:Home,Workbench(Threads, Memory), andInspect(Agent, AG-UI Events, Frontend Tools, Context). Capabilities appears only when an A2UI catalog is present.Talk to an Engineerstays in the sidebar footer.

When Intelligence is connected, Home shows a smallIntelligence connectedchip on the right ofWhat’s going on. Open a real Thread to inspect itsMessages,AG-UI Events, andState. Metadata updates do not change the selected Thread.

When Threads has no real rows, or when Threads is locked, the Inspector keeps the overview video, three local example threads, their detail tabs, and the guided tour. The examples do not send real Thread requests. With reduced motion enabled, the video starts paused with a labelled Play control. If the media fails, fallback text appears while the examples, details, and tour stay usable.

Project context and usage#

An Intelligence-backed Runtime can supply optional trusted project metadata. Valid organization and project names and the plan label appear in the account strip. Each module is independent, so missing or invalid identity data does not hide valid usage or action data.

Trusted Thread usage and a compatible supplied action appear only in the Threads footer. They render independently: usage can appear without identity, plan, license, or action metadata, and action-only metadata does not invent usage.

Usage stateInspector behaviorFiniteShowsused / limit Threadsand a native progress bar. An overage showslimit\+ / limit Threadsand caps the bar at 100%.UnlimitedShows the trusted used count withUnlimitedand no progress bar.Unknown limitShows the trusted used count withLimit unavailable, but invents no numeric limit or progress.Known expiry countShows the count, including0 Expiring Soon.Missing or malformed expiryHides the expiry value without hiding valid base usage.Expiring Soondescribes a future retention-policy threshold in the next 24 hours. It does not prove that the thread culler ran or that the Inspector locked or deleted a Thread.

When Threads is locked, the Runtime’s license state controls the copy and any action:

License stateInspector behaviorActiveExplains that the Runtime has no Threads endpoints and shows no locked action.Not enabledShowsEnable Intelligenceonly for a matching trusted action.ExpiredShowsReneworManage Your Planonly for a matching trusted action.UnknownShows neutral unavailable copy without an action.A valid managed footer can showManage Your Planonly for a matching trusted action. Managed Enterprise has no manage-plan action, and Team Self-Hosted has no hosted action; either can keep valid usage visible.

The Inspector opens the exact trusted URL supplied by the Runtime. It does not build a URL from project details and has no hard-coded signup fallback. If metadata license state conflicts with the Runtime’s license status, the Runtime status controls the copy and the Inspector hides the action without hiding valid usage.

Older Runtime, Core, and Inspector versions remain compatible without a synchronized deployment. Older V1 producers leaveusage\.expiringSoonCountabsent, while pre-expiry Shared and Runtime versions ignore or remove the additive leaf from newer producers. Older Inspectors ignore metadata they do not render; newer Inspectors feature-detect older Core and Runtime support and use a safe missing-metadata fallback. ExplicitthreadEndpointsremain the authority in every mix, and metadata never enables Thread work.

Showing or hiding the Inspector#

By default, the Inspector appears only onlocalhost,127\.0\.0\.1, and0\.0\.0\.0. SetenableInspectortofalseto hide it on those hosts:

<CopilotKit
  publicLicenseKey={process.env.NEXT_PUBLIC_COPILOTKIT_LICENSE_KEY}
  enableInspector={false}
>
  {children}
</CopilotKit>

SetenableInspectortotrueto show it on another host, including in a production build. CopilotKit always uses an explicittrueorfalsevalue.

NEXT\_PUBLIC\_COPILOTKIT\_LICENSE\_KEYis a browser-visible publishable key and is adifferent credentialfrom the server-sideCPK\_INTELLIGENCE\_API\_KEYthatcopilotkit project selectwrites into your\.env. The server-side key is consumed by theCopilotKitIntelligenceclient described inRuntime endpoints. Do not substitute one for the other, and never expose the server-side key to the browser.