s3

Shopify's CEO Tobi Lütke released walgit, a Rust-based Git server that stores repositories in object storage like S3 without a database, featuring various functionalities and over 1,300 GitHub stars.

PicoMQ is a system for durable real-time streams over HTTP, built on S3-compatible object storage, allowing for independent and scalable streams per use case.

Walgit is a Git server implementation that runs as a single binary, using object storage like S3 or GCS as the backend, enabling scalable hosting of Git repositories without local state.

celld is a self-hosted, distributed server that runs Cloudflare Workers and Durable Objects code unchanged, using your own S3 bucket for storage and coordination. It claims significantly lower costs and sub-millisecond warm latencies at scale, with data stored in SQLite/LTX format.

Docker shares a comprehensive guide by Nick Janetakis on backing up PostgreSQL in Docker, covering local storage, S3, and the open-source Plakar tool for reliable daily backups and restores.

Firn is an open-source, multi-tenant vector and full-text search engine backed by object storage like AWS S3, providing a tiered storage architecture with RAM and NVMe caching for performance. It achieves sub-second cold queries with IVF_PQ indexes and microsecond warm hits via result caching.

ZeroFS is a log-structured filesystem that serves S3-compatible buckets as POSIX filesystems over NFS and 9P, or as raw block devices, with compression, encryption, and local caching for fast reads. It passes extensive POSIX and stress test suites, including kernel builds and Jepsen failover testing.

Databricks introduces Lakebase LTAP architecture that stores Postgres data in Parquet on S3, enabling transactions and analytics on a single copy of data without CDC or mirroring.

The author explains how they built a compute platform capable of launching millions of sandboxes per second in constant time, focusing on decoupled scheduling and capacity aggregation using Cassandra and S3.

This project turns an ESP32-S3 development board into a USB drive that appears to contain every possible file, inspired by the Library of Babel. It's a fun hack that generates arbitrary file contents based on path names using MTP protocol.

Streambed is an open-source CDC engine that streams Postgres WAL changes to Iceberg tables on S3, with a built-in query server using DuckDB that speaks the Postgres wire protocol.

A tweet discusses the idea of building a transactional database with infinite throughput using S3-like object storage and content-addressing, where blocks are written in parallel and the root hash is updated periodically.

Almog Gavra announces that a performance optimization using HEAD requests instead of LIST calls has been deployed at Dropbox, reducing SlateDB startup time by 60x.