database

Man Group has open-sourced ArcticDB, a high-performance, serverless DataFrame database designed for Python data science and quantitative finance scenarios, supporting billions of rows of time-series data, S3/LMDB/Azure storage, time travel and snapshots, and C++ engine acceleration.

This article explains how to build a SQL optimizer using the Egg framework in Rust, covering techniques like constant folding, predicate pushdown, and join reordering with rule- and cost-based optimization, using under a thousand lines of code.

An academic paper arguing that SQL's use of nulls and bags is unnecessary, and presenting the fully normalized, null-free language Rel as evidence that set semantics can replace these features.

This article summarizes the key points of Richard Hipp's 2024 talk on SQLite design principles and philosophy, explaining SQLite's embedded, single-file, low-complexity features and how it works.

A developer reflects on a decade-old article about Postgres connection management, surveys popular managed Postgres providers, and argues that PgBouncer-style connection pooling is effectively a mandatory core feature.

The author describes using Antithesis with Claude to reproduce a long-standing SQLite WAL-reset bug in 15 minutes, and verifies the fix in version 3.51.3.

Simon Willison announces the alpha release of alchemy-utils, a database-agnostic Python library and CLI inspired by sqlite-utils, built with AI assistance and supporting PostgreSQL, SQLite, and DuckDB.

Tailscale recounts how they tracked down a 16-year-old SQLite bug causing database corruption and outages, eventually fixing it after months of investigation.

Tencent CloudBase adds PostgreSQL support, letting AI coding agents spin up isolated databases with a single prompt, with SQL-declared permissions and row-level security.

Explores the IBM i (OS/400) operating system, tracing its origins from the AS/400 platform and explaining its unique architecture where the Db2 relational database is integrated directly into the kernel.

Simon Willison experiments with storing SQLite text revision histories as compressed JSON arrays, comparing whole-blob and chunked storage approaches. A simulation of 1,000 edits compressed 20.4 MB of revision text into 80.3 KB using Zstandard.

A developer explains that AI coding agents keep raising the database pool max connection limit despite comments and instructions, and the only reliable guardrail has been a test that fails if the value changes.

A technical blog post explaining the FastLanes Unified Transport Layout, a SIMD-friendly data layout for parallel delta decoding, and how it enables data-parallel processing with wide virtual registers.

Shopify engineering describes replacing Redis with MySQL for inventory reservations, using MySQL's SKIP LOCKED feature to achieve high-throughput scaling during peak traffic and eliminate consistency issues between separate systems.

A tweet from @venkateshdotdev listing 10 challenging system design interview questions covering URL shorteners, scaling, consistency, rate limiting, fault tolerance, and handling traffic spikes.

pgrust 0.2 rebuilds the Postgres query engine with batching, operator fusion, and SIMD, achieving 300x faster analytical queries and 30% faster OLTP than Postgres.

This article explains why PostgreSQL disables parallel query for count(DISTINCT) aggregates, causing full-table sorts and disk spills, and compares it to the parallel-friendly count(*).

A deep-dive blog post walks through a Rust performance PR in GreptimeDB that makes Prometheus read conversion 10x faster, with benchmark reproduction and code analysis.

This paper introduces ISEE, an interactive system that uses LLM-based agents to assess and collaboratively enrich the semantic quality of database field descriptions, reducing cognitive load and improving downstream tasks like entity linking.

GreptimeDB v1.2.0-beta.1 is released with major features including JSON2 as a structured column type, Prometheus Remote Write v2 with native histograms, dictionary-encoded series keys for faster queries, and additional hardening/breaking changes.