@almoggavra: this is now deployed at Dropbox and their SlateDB startup time dropped 60x LIST calls are expensive, folk
Summary
Almog Gavra announces that a performance optimization using HEAD requests instead of LIST calls has been deployed at Dropbox, reducing SlateDB startup time by 60x.
View Cached Full Text
Cached at: 05/30/26, 06:31 AM
this is now deployed at Dropbox and their SlateDB startup time dropped 60x
LIST calls are expensive, folk
Almog Gavra (@almoggavra): Know your S3 op latency/costs. Sometimes it’s better to probe for a file using HEAD requests instead of LIST.
Just merged in Slate: https://t.co/q65sgsTmkB uses exponential probes + binary search on sequential files
Similar Articles
@Greptime: GreptimeDB v1.1.0 is out PromQL rate/increase up to 97% faster, 20–40% lower query time overall Up to 4.5x faster on TS…
GreptimeDB v1.1.0 is released, offering up to 97% faster PromQL queries, 20-40% lower overall query times, and up to 4.5x improvement on TSBS scan-heavy queries, along with online repartitioning for existing tables.
@Greptime: Most of GreptimeDB's June work came down to one idea: a filter is no use if it can't reach the data. In a distributed q…
GreptimeDB improved distributed query performance by enabling remote dynamic filters to push down to datanode scans at runtime and optimizing the optimizer to run before MergeScan wraps remote plans, ensuring filters reach the data. JSON v2 columns now support type hints.
@_avichawla: How Cloudflare cut query times by 35x without leaving Postgres: Their Postgres tables hit billions of rows, and every t…
Cloudflare improved query performance by 35x on billion-row Postgres tables using TimescaleDB (Tiger Cloud), and built a real-time earthquake dashboard with Claude Code and Tiger CLI.
@Greptime: GreptimeDB's flat-format queries can now prefilter on any column — tags, fields, timestamps — not just primary keys. Wh…
GreptimeDB's flat-format queries now support prefiltering on any column (tags, fields, timestamps), not just primary keys, delivering up to 4.5x faster performance. Additionally, the mito2 storage engine removed its legacy scan path, cleaning up about 1,800 lines of code.
DuckDB – Data power tools for your laptop, now in Clojure (2023)
TechAscent illustrates how DuckDB's high-performance vectorized SQL engine can now be leveraged from Clojure via tech.ml.dataset (TMD), enabling large out-of-memory joins and 50GB CSV ingestion that compresses to 18GB in about two minutes.