the accessibility tree gotchas that kept breaking my desktop agent

my desktop agent stopped failing the moment i stopped trusting the accessibility tree as a single source of truth. The dumbest one was cross-app handoff. agent clicks a link in mail, safari becomes frontmost, the agent keeps asking for the original pid's tree and operating on a frozen snapshot. fix is detecting when the frontmost app changes between actions and traversing the new one before the next step. Easy to miss because the previous pid is still alive, just no longer relevant. second one was sheets and dialogs overriding window viewport scope. an element shows up in the tree because it technically exists in the hierarchy, but it sits underneath an active modal sheet, so clicks pass to whatever is actually on top. Needed an explicit "is this element inside the current modal" check before every click. Multi-monitor coordinates were the third. on a 3 screen setup the left external sits at x around -3840 and the right around 3456. a naive "click at x:200" lands on whichever screen contains (200, y), which is almost never the one you mean. llm clicking the wrong button is rarely the model. it is the tree state being stale or scoped wrong, and the failure mode is silent until you diff before and after screenshots. written with s4lai