The article compares confidential inference APIs from Privatemode, Tinfoil, NEAR AI, and Chutes, highlighting their security features like end-to-end encryption and trusted execution environments, along with tradeoffs in model selection and verification maturity.
I've been looking for inference APIs where the provider technically can't read the prompts, rather than just promising ZDR. The four that seem most interesting so far are Privatemode, Tinfoil, NEAR AI and Chutes. Privatemode probably has the cleanest security story to me right now. Their stack is open source, they use reproducible builds, remote attestation and E2EE into the confidential worker, with NVIDIA confidential GPUs. They're also EU hosted. The tradeoff seems to be a relatively small model selection, and you need to use their proxy if you want the full E2EE and verification setup. https://docs.privatemode.ai/security/pqc/ Tinfoil seems similarly strong, but takes a slightly different approach. Their enclave code is open source and they tie the running measurement back to their public build/Sigstore infrastructure. I like that the client can verify what enclave it's connecting to instead of just trusting an API endpoint. Compared to Privatemode, though, they rely more on build provenance and transparency rather than reproducible builds, and their model selection is also fairly limited. https://docs.tinfoil.sh/verification/verification-in-tinfoil NEAR AI looks like a good middle ground. Their E2EE flow encrypts requests for an attested inference instance using TDX + NVIDIA confidential computing, and they have a better model selection than Privatemode or Tinfoil. The downside for me is that their verification story feels less straightforward. I'm not yet sure how independently verifiable the complete chain from public source code to the workload I'm actually sending my prompt to is. https://docs.near.ai/cloud/guides/e2ee-chat-completions/ Chutes seems the most attractive if price and model selection matter. Their E2EE setup encrypts requests specifically for an attested confidential instance, while the normal API layer only handles ciphertext for routing and billing. They have a huge model selection, are very cheap, and have open sourced a good chunk of their TEE stack. The tradeoff is that the whole system has more moving parts, and I'm less confident about how mature and independently verifiable the source/build/measurement chain is. https://chutes.ai/news/end-to-end-encrypted-ai-inference-with-post-quantum-cryptography The thing I'm ultimately trying to figure out is how much of this is actually provable. A TEE on its own doesn't stop malicious code inside the TEE from decrypting the prompt and sending it somewhere else. Ideally, I'd want the source to be public, the build to be reproducible or otherwise verifiable, and the client to check the attested measurement before giving the workload access to the prompt. That way you're verifying what code actually receives the plaintext, rather than just verifying that something is running inside a TEE. Has anyone here looked deeply into these or used them in production? I'm especially curious whether I'm missing any important differences in their verification models, or have I missed a provider.
Apple announced integration of Google Gemini models with its Private Cloud Compute for Siri AI, aiming to use personal context while maintaining privacy, but the article argues that private inference still exposes private data during computation, raising concerns about true privacy.
The article discusses the concept of verifiable AI inference, exploring methods like trusted attestation and cryptographic proofs to ensure the authenticity and provenance of AI-generated outputs without rerunning the model.
Researchers discovered a method to extract hidden reasoning traces from frontier AI models via APIs, exposing personal-information leakage risks and enabling large-scale distillation attacks. The findings also suggest possible reasoning distillation from US models into Chinese model Kimi K3, though not conclusively.
NVIDIA's Confidential Computing, using Blackwell GPUs, is being adopted by Apple to expand its Private Cloud Compute to Google Cloud, enabling secure server-side inference for Apple Intelligence features while maintaining strong privacy guarantees.