How do people build batch agents now?

I recently talked with a few people who are building agents, and their use cases are showing some common patterns. A few examples of what they are building: 1. Spin up an agent read team to attack the agent under test to find the vulnerabilities. 2. Use agents to simulate different customer personas in order to test the website before launch. 3. Evaluate the end-to-end agent experience of their CLI + SKILL with many agent sessions that takes in different user queries. The pattern: 1. The entire task can be deferred. They usually don't expect the immediate response. Unlike the personal agent. 2. They are not interactive tasks. Most of the time, these tasks will be completed without human in the loop. 3. Typically in batch. The number of tasks is usually larger. I found this pattern to be interesting because it challenges the current infrastructure of agents. Take inference for example, most providers offer realtime & batch API. However, none of them seem to be a perfect fit: For deferred tasks, people are willing to trade the latency for cost savings. The realtime api doesn't offer that. The batch api on the other hand has lower price with higher latency, but it makes it almost impossible to hit the KV cache. Hitting the KV cache is essential to agentic workloads because of multi-turn inference. Therefore, batch api will not actually save any cost. Why there should be a solution? It is because only a small portion of compute resource is used to serve realtime traffic due to the unpredictability of realtime request. The batch jobs is more predictable, so it can use the computer resource more efficiently, which should lead to lower price. These are just theoretical thinkings. Let me know your thoughts:)