Tag
A massive supply-chain attack on the open-source AI tool LiteLLM exposed terabytes of credentials from thousands of organizations, including Microsoft, Amazon, and Cisco, during a 40-minute window in March. Security firms CloudSEK and Hudson Rock disclosed the breach, attributing it to the TeamPCP gang.
The author shares experience building a human-in-the-loop approval system for an enterprise agent platform, emphasizing that the approval step must be a true blocking pause with editable parameters and first-class rejection/editing outcomes, and asks how others structure agent suspension.
LiteLLM has migrated from Python to Rust, achieving massive performance improvements: request overhead reduced by 150x to 0.05ms, throughput increased by 15x, memory usage reduced by 11x to 32MB.
An Android phone is repurposed as a portable GGUF inference server with Vulkan acceleration, exposing an OpenAI-compatible endpoint via LiteLLM and Tailscale mesh for integration into a self-hosted AI cluster.
A detailed comparison of OpenRouter, Concentrate.ai, Portkey, and LiteLLM as LLM gateways for a B2B SaaS product, covering features, costs, and governance.
kg-gen is an open-source Python package that uses language models (via LiteLLM and DSPy) to extract knowledge graphs from plain text or conversation messages, featuring chunking, clustering, and flexible provider routing.
A founder seeks validation for a service that configures production-grade LLM gateways to address common enterprise issues like cost visibility, provider lock-in, and PII leakage using open-source tools.
Major open-source release featuring RLM improvements, optimization chaining, and initial LiteLLM decoupling with 24 new contributors.
GoModel is a new open-source AI gateway written in Go that offers a unified OpenAI-compatible API for multiple providers and claims to be 44× lighter than LiteLLM.