parquet

Pydantic 的技术博客介绍 Logfire 通过 dynamic shredding（动态拆解半结构化 JSON 属性数据）技术实现部分查询提速 1000 倍，原本 30 秒超时的查询现在不到一秒即可完成，将于 2026 年 2 月向所有客户推出。

MotherDuck 的技术博客详细解析 DuckDB 2.0 alpha 版本为何更快，重点介绍了通过独立下载线程池实现的异步 I/O，使 S3 上的 Parquet 查询性能显著提升，并分享了实际基准测试数据。

DuckLake 是一个基于 SQL 和 Parquet 构建的开放 Lakehouse 格式，其 DuckDB 扩展允许直接读写 DuckLake 数据，支持时间旅行、Schema 演进和变更数据流（CDC）等功能。

Amazon Aurora PostgreSQL now embeds DuckDB to let users directly query live operational data alongside historical data in Apache Iceberg and Parquet formats stored in S3 data lakes, eliminating the need for ETL pipelines and data duplication.

This article details the implementation of a Parquet file writer in Haskell for the DataHaskell/Dataframe library, enabling efficient data serialization and interoperability with the data science ecosystem.

GreptimeDB Enterprise integrates with Apache Iceberg to make tables queryable from Spark, Trino, DuckDB, and other engines without copying data by publishing Iceberg metadata over existing Parquet files.

The article demonstrates a method for building fast drilldown dashboards from a single Parquet data cube using the Hyparquet JavaScript reader and HTTP range requests against object storage, eliminating the need for a traditional database.

DuckDB is introducing asynchronous I/O for Parquet and CSV files in v2.0, improving remote storage query performance by avoiding blocking worker threads during data fetches.

DuckDB's file_row_number option for paging through Parquet files is 2.53x faster than using LIMIT/OFFSET, as it leverages row group metadata to skip irrelevant blocks, but the benefit depends on having many row groups.

This blog post describes an optimization for Apache Parquet to efficiently store and decode fixed-length lists like vector embeddings, achieving decoding performance comparable to flat columns by bypassing Dremel reconstruction for fixed-size data pages.

Databricks introduces Lakebase LTAP architecture that stores Postgres data in Parquet on S3, enabling transactions and analytics on a single copy of data without CDC or mirroring.

A developer shares their experience building a points-of-interest feature for a running app using Geonames data, DuckDB, and AI assistance from Claude, encountering challenges with biases and hallucination.

GreptimeDB's Mito2 storage engine uses an LSM-tree design with columnar Parquet SSTs, three-level scan pruning, and TWCS compaction. The blog post provides a full walkthrough of its architecture.