Tag
A 2025 court order requiring OpenAI to preserve deleted chat and API logs highlights that agent builders using public LLMs lose control over data deletion, prompting discussion of private model deployments and retention policies.
SVPino highlights Parallel Web Systems' new Decisions API, which combines the speed and cost-efficiency of decision models like Jev with real-time web-grounded news context, available at $0.001 per request.
A roundup of changelogs from 10 AI agent tools over a 9-day period, covering releases like GitHub Copilot Fusion, GPT-6.1 Terra, Anthropic SDK breaking changes, and Cursor's Agent SDK GA — closing with discussion questions about upgrade strategy.
Alibaba quietly raised the output price of DeepSeek V4 Flash 0731 by 230% overnight, with Mistral and Qwen models seeing similar hikes, signaling the end of VC-subsidized LLM pricing for builders.
The author strongly recommends an article for anyone building AI applications, automation, and Forward Deployed Engineer roles, outlining the complete AI engineering stack: Python → FastAPI → LLM API → Tool Calling → RAG → Workflow → Agent → Eval → Deployment.
Inferock 在 Product Hunt 发布,提供每笔 LLM API 调用的独立收据、失败调用返额以及自带密钥的调用监控与成本追踪,支持 OpenAI、Claude、Gemini 等模型的无服务器按 token 计费推理。
Twigg is a stateful API for LLMs that manages conversation context, schema fitting, and routing to simplify application development.
This paper introduces the Hydration Proxy Pattern, a three-tier architecture to manage conversational state for stateless LLM APIs, ensuring data sovereignty, optimizing caching, and improving semantic grounding in enterprise systems.
Fast Inference is an LLM API service that offers fast and affordable access to top open-source and closed-source models for developers, with integrations for coding agents and tools like Claude Code and Codex.
The article introduces aimock, an open-source tool by CopilotKit that mocks API calls for AI applications during CI/CD, reducing costs and maintaining type correctness across multiple providers.
GitHub Models has been retired, causing disruptions for developers using its unified LLM API in GitHub Actions. Simon Willison reflects on the shutdown and his migration to OpenAI's API.
Analyzes how per-token LLM pricing creates incentives for verbose output, and proposes low-entropy prompt constraints (FAOA) to reduce cost and increase semantic density.
A detailed measurement study across Anthropic, OpenAI, Gemini, and DeepSeek finds that the conventional 30-second prompt cache keepalive is 8x too frequent; a 4-minute interval is optimal, and only Anthropic's cache saves money at long idle gaps.
Proposes Cache-Aware Prompt Compression (CAPC), a method that combines query-agnostic compression with caching to reduce LLM API costs, demonstrating significant savings over existing approaches on Anthropic's Sonnet API and production workloads.
Sakana AI launched Fugu, a multi-model orchestration API, after Anthropic suspended access to Claude Fable 5 and Mythos 5. Fugu Ultra scored 93.2 on LiveCodeBench, outperforming Fable 5, and starts at $5 per million input tokens.
Bash4LLM+ is a lightweight, dependency-free Bash wrapper for LLM APIs, offering secure and auditable interaction with Groq and other providers, with features like dynamic model lists, streaming, and extensible extras.
FreeLLMAPI is an open-source tool that aggregates the free quotas of 16 LLM providers into a single OpenAI-compatible endpoint, with automatic routing and usage tracking, totaling about 1.7 billion tokens per month.
The author notes a surprising lack of publicly available APIs for medical-oriented LLMs, despite models like MedGemma and BioMistral existing on Hugging Face, and seeks information on any available options.
The author explores encrypted reasoning blobs in LLM APIs from OpenAI and Anthropic, discussing how chain-of-thought data is encrypted and signed, and the security implications of tampering with those blocks.
A practical guide listing 10 strategies to reduce costs when using LLM APIs, including model selection, prompt caching, batch processing, and monitoring expenses.