Tag
MCP has received a major update. The core protocol is now fully stateless, removing sessions and long-lived connections, making it better suited for serverless, edge deployment, and horizontal scaling. It's being called the biggest update since its release.
camelAI rewrote their coding agent to run entirely in a Cloudflare Durable Object, using SQLite and R2 for filesystem and replacing bash with JavaScript. The migration from VMs reduced costs and latency, and the codebase is now open source.
This article compares serverless, on-prem, and edge deployment for AI models, highlighting inefficiencies in current multi-model serving. It introduces the Superlinked Inference Engine (SIE), an open-source tool that serves multiple models on a single GPU by dynamically loading and unloading weights, aiming to reduce costs and complexity.
A proposed SLA-constrained carbon-aware routing policy for geo-distributed serverless clouds achieves up to 46.8% carbon reduction while maintaining zero SLA violations, evaluated on real AWS deployments.
Modal is a serverless cloud platform designed for AI workloads, supporting inference, training, and sandboxes. It enables instant scaling from zero to thousands of GPUs with pure Python code, significantly reducing latency and accelerating time to market.
Explores different long-term memory architectures for AI agents, with a focus on an agent-as-memory-controller approach using neon postgres.
Runpod announces FlashBoot, a serverless approach for AI models that moves idle models to cheaper storage and pages them back to GPU, achieving cold starts under 200ms and cutting costs by 90% compared to traditional clouds.
This paper presents a dependency-aware autoscaling framework for serverless environments, integrating graph-based bottleneck identification, multi-model forecasting (MLP, LSTM, CNN) via a probabilistic ensemble, and cost-aware scaling control. Experiments show 99.88% prediction accuracy and reduced infrastructure costs.
Telegram launches a serverless platform for bots and Mini Apps, allowing developers to run JavaScript code on Telegram's infrastructure without managing servers.
Google publicly launches Cloud Run sandboxes, showcasing the ability to start, execute, and stop 1,000 sandboxes in 5 seconds with an average latency of 500ms.
Modal aims to be a global aggregator of compute and resource manager, acting as a very large global computer.
Charles wrote an article explaining what Modal is while on a flight to ICML in Seoul.
In a Twitter thread, the founder of Modal explains that the platform is best understood as a computer, drawing parallels between traditional computer architecture and Modal's serverless cloud infrastructure.
Cerebrium reduces GPU cold starts for AI workloads by checkpointing CPU and GPU memory, restoring fully initialized containers in seconds, cutting startup time by over 80%.
FlareMo is a personal note-taking system based on Cloudflare Workers. It uses the free tier to achieve zero server maintenance, supporting Flomo-style timeline notes, tags, attachments, search, and more.
Vercel now supports running any Dockerfile, allowing developers to deploy containerized HTTP services (Go, Rails, Spring Boot, etc.) directly on Vercel's Fluid compute platform with autoscaling, preview deployments, and pay-per-CPU usage.
Vercel announces Services, allowing users to collocate multiple backend and frontend services in one project with atomic deployment, rollback, preview URLs, and internal networking.
Fireworks AI announces Serverless 2.0, introducing three serving tiers (Standard, Priority, Fast) to handle traffic congestion without pre-provisioning GPUs, enabling per-request routing for reliability and cost efficiency.
Modal announces a new Auto Endpoints feature powered by Modal Servers, detailing the architecture using EnvoyProxy, Google Cloud Spanner, and Cloudflare Pingora.
Modal announces Auto Endpoints for effortless inference, praised by developer Anthony Corletti as a top-level abstraction over compute, storage, and networking.