@akshay_pachaar: AI security goes far beyond AI. Adding an LLM call to a product puts security focus primarily on prompt filtering, outp…

AI security goes far beyond AI. Adding an LLM call to a product puts security focus primarily on prompt filtering, output guardrails, and input validation. That is crucial, but it only covers the application layer, and an equally important infra layer needs its own controls. To understand this, trace a single model call. The prompt leaves the app, hits a provider endpoint, and depending on their retention policy, it sits in external logs for days. If that prompt carries any regulated/private data, it moves beyond the app's boundary. This isn't a failure of prompt filtering since it only governs what goes into the model, and it does not enforce where data can travel/who can store it. Beyond application-layer controls, two infrastructure-level concerns need their own coverage: 1) The first is per-request. Every model call moves data outside the app boundary. Who can access it in transit? Is it scoped to the right tenant? 2) The second is posture drift. Do providers retain the data, and for how long? Can one tenant's prompts reach another tenant's context through unintended disclosure? And do the AI outputs stay inside the compliance boundary that enterprise deals require? Teams typically solve both in the application code, where each new AI feature gets its: - own scoping logic - own encryption - own logging Ten features mean ten separate implementations of the same controls. Moving both to the infrastructure layer simplifies this. - If IAM policies control which services can call which endpoints, every new model call will inherit that boundary automatically. - VPC isolation keeps tenant traffic apart at the network level. - Encryption covers every data flow by default. Every API call, including model calls, gets logged without the feature code handling any of it. @awscloud builds this as the managed infra and they have partnered with me today to explain how it works. Model calls through Bedrock or SageMaker inherit the same IAM, VPC, encryption, and CloudTrail boundaries as every other service. For instance, IDEMIA runs this pattern in production. The company handles identity data for 45+ US government agencies. After moving to containers on Amazon EKS in AWS GovCloud, they cut costs by 30% and sped up delivery by 4x, with the security coming from the infrastructure, not the application code. You can read more on how AWS supports ISVs building this way → https://fandf.co/4d23iNH Also, I wrote a detailed breakdown of the 11 components in a production agent harness, the full software layer that wraps an LLM. This post covers what sits underneath it. The harness is only as secure as the layer it runs on. Read it below.