@Greptime: Your logs keep growing. Query traffic comes in peaks. Why scale storage and compute together? In 3 minutes: how Greptim…

Your logs keep growing. Query traffic comes in peaks. Why scale storage and compute together?

In 3 minutes: how GreptimeDB separates them with shared object storage in distributed deployments, and reuses data files during repartitioning.

https://t.co/3hCde9iAY8

TL;DR: When logs keep growing and query traffic spikes, you don’t need to scale compute and storage in lockstep — GreptimeDB separates compute from persistent storage so you can scale compute by query load and scale storage by retention needs.

Logs Are Growing, Queries Have Peaks: Why Decouple

Your logs keep growing, and query traffic comes in peaks. The key question: does adding compute capacity mean migrating all historical data to new machines?

GreptimeDB’s answer: separate compute from persistent storage. That way, you scale compute based on query traffic and storage based on retention requirements — rather than growing both as a single unit.

What Is GreptimeDB

GreptimeDB is an open-source observability database. Metrics, logs, and traces share the same columnar engine, but can use independent tables, indexes, and retention policies.

The benefit: all three signals use one database backend, without forcing them into a single data model.

Joining Three Signals with SQL

When identifiers are aligned, you can query and correlate these signals with SQL — metrics, logs, and traces in the same query for cross-signal analysis.

Ingestion Path and Query Capabilities

Data ingestion supports:

• Prometheus remote write
• OpenTelemetry over HTTP
• Loki push endpoint

This lets you keep using familiar collectors while analyzing stored signals with SQL and querying metrics with PromQL.

Note the boundary: ingestion compatibility doesn’t mean full feature parity — supporting Loki push doesn’t mean supporting LogQL.

Architecture: Division of Labor on Shared Object Storage

In a distributed deployment using shared object storage, each component has a distinct role:

Hotspot Partitions: Changing References, Not Moving Data

A Region is a logical data shard, and its Manifest records references to persistent files.

During repartitioning, the rules, file references, and routing all change. The key point: existing SST files are reused rather than wholesale-copied between nodes. This avoids migrating all old files just to adjust partition boundaries.

Note: the files themselves stay in place — only the references change.

This Isn’t Zero I/O

Flushing and compaction may still produce writes. But this lets you adjust partition layout without rebuilding entire tables.

Prerequisites for Manual Repartitioning

Manual repartitioning requires:

1. A distributed cluster
2. Shared object storage
3. Garbage collection (GC) enabled on Meta SRV and each data node

Additionally, writes may be briefly affected during the switchover, so clients need retry mechanisms.

Enterprise Edition Feature Boundaries

Automatic repartitioning and automatic region balancing are enterprise features, subject to their own configuration and triggering conditions.

Comparison with ClickHouse

ClickHouse offers real-time analytics and the ClickStack observability platform, and ClickHouse Cloud also implements compute-storage separation.

GreptimeDB’s differentiator is its telemetry-native database foundation:

• Native ingestion, SQL, and PromQL
• A unified backend for different signal models
• Shared object storage and manual repartitioning included in the open-source core, runnable on your own infrastructure

This combination lets you build around familiar ingestion paths, correlate stored signals, and plan compute capacity and data retention independently.

Advantages Worth Verifying

In summary, the advantages worth verifying are clear:

• Unified backend: Supports different signal models
• Familiar ingestion paths: Prometheus remote write, OpenTelemetry, Loki push
• Compute capacity independent of data retention growth

Validate this combination against your ingest rates, retention periods, and typical queries.

Docs and code references are in the video description. Next, the author will apply a troubleshooting workflow to this architecture.

Source: @Greptime — How Compute-Storage Separation Avoids Migrating Historical Data When Scaling (https://www.youtube.com/watch?v=Sl_Q3zH_aFI)