A Non-Exhaustive Inventory of exe’s Software Factory:

An agent that looks for security issues systematically. An agent that investigates alerts. An agent that investigates logs. Bots to fix flaky and slow tests. A status page. A system to page our phones (using the excellent and simple PushOver).

Read on:

A Non-Exhaustive Inventory of exe’s Software Factory

Source: https://blog.exe.dev/inventory

1. An agent that looks for security issues systematically. Fable refuses to help out, so we systematically look for security issues, with a bias toward recent changes.
2. An agent that investigates alerts. Sisyphus keeps track of our alerts. We have ones that page us and ones that merely make noise in Slack. Either way, Sisyphus looks through our logs and metrics and source code, as well as analyzing its own previous investigations, to tease out what’s going on. It gives a great head start when investigating an issue (or just a flaky alert!).
3. An agent that investigates logs. Every day, I get an email with interesting trends in our logs.
4. Bots to fix flaky and slow tests. A bot is continuously analyzing flaky or slow tests in our CI and suggesting changes.
5. A status page. status.exe.devisn’t hosted on exe.dev. We built it ourselves, though.
6. A system to page our phones (using the excellent and simple PushOver) When the aforementioned alerts fire, our phones beep very loudly. Traditionally you use PagerDuty for this, but PagerDuty’s durable asset is the entitlement for “Emergency Alerts” from Apple. Turns out PushOver has this as well, and a lovely API.
7. An agent that supervises deploys and rollouts. Athena helps do rollouts. Infrastructure deploys are not instantaneous, and even the most patient operators stop paying attention. It checks metrics and logs (and has looked at the source code for what changed in this deploy).
8. A blog CMS, with comments, collaborative text editing, embargoes, the whole nine yards If you’re reading this on blog.exe.dev, this ain’t Wordpress. Our blog started out as Markdown files in git, but now there’s a full-featured CMS, with collaborative editing, revision history, comments, embargoes, and a content calendar. A built-in agent (really, Shelley running on the same VM) can import a blog post from whatever you paste in.
9. UI tests described as textual paragraphs that lazily materialize into browser instructions but self-heal Who are we kidding? We’re not maintaining Playwright tests by hand anymore. Shelley’s UI tests are increasingly a paragraph of text asking for some behavior. There’s a cache file (checked into git) that makes the test cheap and fast. When it fails, the CI system “heals” it with an LLM, and either fails or checks in the new fixed test. Yes, the build queue modifies the commit on its way through if necessary. More on this in a future post.
10. Intrepid reporter bots that report on git commits, our help threads, and so on Every day, we get summaries in Slack about what’s happened in the past day, across git commits and such.

Please note: if you’re writing bots that read untrusted data, understand theLethal Trifecta: private data, untrusted content, and external communication. We happen to think thatexe.devVMs are a great place to isolate these bots, but we also make sure that the tools available to these agentic loops (an agent is just 11 lines of code:https://sketch.dev/blog/agent-loop) are limited in what they can do.