@Vtrivedy10: ok so it’s early but @mattpocockuk’s grill-me skill feels like great DX for iteratively building evals/environments wit…
Summary
A tweet thread discussing the iterative process of building evaluations and environments for AI agents, emphasizing human-agent collaboration and the importance of data and verifier design.
View Cached Full Text
Cached at: 07/15/26, 03:43 AM
ok so it’s early but @mattpocockuk’s grill-me skill feels like great DX for iteratively building evals/environments with humans + agents in the loop
understanding a repo + traces + definition of success is pretty much never a one shot thing, it’s many step collaboration between humans and agents
idk how to measure & improve every agent so getting grilled helps
from an agent definition + data dump, it’s impossible to know which axis of agent performance teams want to generate evals for measurement and hill-climbing
specifics matter a lot here:
- verifier design (do we have ground truth somewhere)
- what tools in the environment should be simulated vs run (semi)-live
- what characteristics do we care most about rn? cost, accuracy, etc?
- what data needs to be sourced or synthetically generated to produce a realistic production like environment? how do we do that?
data is the currency for building better agents
humans sharing expertise/priors & controlling the loop on data design helps a lot today
this looks great, thanks for the pointer!
yeah! + env/eval gen is one of the highest leverage areas humans can allocate their time and should! the quality of the envs dictate the agent behavior
works fine in codex & deepagents for me
need to give it a spin!
Similar Articles
@BraceSproul: I've been thinking a lot about the two different groups of evals you need in general agents/agents which handle broad t…
A Twitter thread discussing two distinct evaluation suites needed for general AI agents: a lightweight benchmark eval for quick iteration and a comprehensive test coverage eval for thorough validation across diverse user paths.
@mattpocockuk: Going live, doing a full feature build using: - /grill-with-docs - /handoff - /prototype
Matt Pocock live streams building a 'delivery calendar' feature for his course video manager using AI tools like Grill with Docs. He focuses on aligning terminology and data models with AI agents, and discusses the importance of coding standards for agent experience (AX).
@nurijanian: my favorite ways to "write requirements" with AI 1. /grill-me by @mattpocockuk https://github.com/mattpocock/skills… 2.…
Discusses favorite tools and methods for writing requirements with AI, highlighting the /grill-me skill by Matt Pocock and a new skill based on business analyst literature to improve alignment and reduce verbosity in AI coding agents.
@shao__meng: Seven Stages of AI-Driven Development 1. Grill 2. Research 3. Prototype 4. PRD 5. Issues 6. Implement 7. Review From Skills For Real Engineers by @…
A Twitter thread shares the seven stages of AI-driven development from the Skills For Real Engineers project by @mattpocockuk, including alignment techniques like grill sessions and tooling for coding agents.
@mattpocockuk: Reworded /grill-me so it can now truly be used anywhere - not just engineering. "design tree" -> "decision tree" "every…
Matt Pocock updated the /grill-me skill to be usable beyond engineering by rewording terms like 'design tree' to 'decision tree'. The skills repository provides composable skills for AI coding agents to improve alignment and reduce verbosity.