Where should the safety boundary live when agents can trigger physical actions?

Most agent discussions I see are still about software: browser tasks, code, internal tools, data workflows, tickets, email, and API calls. I am more interested in what happens when agents start touching local hardware. Not sci fi robots, just ordinary devices like cameras, microphones, sensors, relays, small motors, smart home systems, lab equipment, or access controls. Because once physical hardware is involved, the stakes and the failure modes change completely. A bad browser action is usually recoverable. A bad hardware action can physically move something, unlock a door, disable a safety protocol, or trigger a signal at the worst possible time. My current view is that the model should not have final authority. It can interpret intent and propose an action, but a separate layer should decide whether that action is allowed. Read only should be the default. State changing actions should need explicit approval. Anything involving access control should be treated as high risk. Every physical action should leave a log. The part I am still thinking through is where to enforce that boundary. Tool wrappers are convenient, middleware feels cleaner, device level permissions are harder to bypass, and human approval is safest but can make the system less useful. For people building agents that touch hardware, robotics, smart homes, or access systems: where do you draw the line?