@Greptime: Repartitioning a big table is normally a migration project: new table, dual-write, backfill TB of history, cut over. Gr…

X AI KOLs Timeline Products

Summary

GreptimeDB v1.1 introduces online repartitioning for existing tables via a single ALTER TABLE statement, eliminating the need for data migration, dual-writes, or application changes. It leverages shared object storage and logical shards to update manifests and routing without moving data between nodes.

Repartitioning a big table is normally a migration project: new table, dual-write, backfill TB of history, cut over. GreptimeDB v1.1 makes it one online DDL: ALTER TABLE sensor_readings PARTITION ON COLUMNS (device_id, area) ( device_id < 100 AND area < 'South', device_id < 100 AND area >= 'South', device_id >= 100 AND area <= 'East', device_id >= 100 AND area > 'East' ); Why it isn't a data migration: data lives in shared object storage, not node-local disks. A Region is a logical shard that references files through manifests. Repartitioning updates manifests + routing — zero bytes move between nodes. Then SPLIT / MERGE PARTITION to tune hotspots as load shifts. https://greptime.com/blogs/2026-07-09-greptimedb-v1.1-partition-existing-table…
Original Article
View Cached Full Text

Cached at: 07/14/26, 02:26 PM

Repartitioning a big table is normally a migration project: new table, dual-write, backfill TB of history, cut over.

GreptimeDB v1.1 makes it one online DDL:

ALTER TABLE sensor_readings PARTITION ON COLUMNS (device_id, area) ( device_id < 100 AND area < ‘South’, device_id < 100 AND area >= ‘South’, device_id >= 100 AND area <= ‘East’, device_id >= 100 AND area > ‘East’ );

Why it isn’t a data migration: data lives in shared object storage, not node-local disks. A Region is a logical shard that references files through manifests. Repartitioning updates manifests + routing — zero bytes move between nodes.

Then SPLIT / MERGE PARTITION to tune hotspots as load shifts.

https://greptime.com/blogs/2026-07-09-greptimedb-v1.1-partition-existing-table…


Adding Partitions to an Existing Table: Online Repartitioning in GreptimeDB v1.1

Source: https://greptime.com/blogs/2026-07-09-greptimedb-v1.1-partition-existing-table

You want to add partitions to a table after it’s created. In the past, the only option was to rebuild it: create a new partitioned table, dual-write, backfill historical data, then cut over traffic. GreptimeDB v1.1 lets you run a singleALTER TABLEto partition an existing, unpartitioned table, with no rebuild required.

Partition rules are usually fixed at table creation​

In most databases that support partitioning, the partition rules have to be decided when the table is created, and they’re hard to change afterward. When data volume is small, one table with a single shard is enough, so the limitation doesn’t show. It surfaces once the data grows.

Take asensor\_readingstable created without partitioning. As the number of devices grows from dozens to thousands, and writes climb from a few thousand rows per second to 50,000, everything lands on the same Region and the same node. That node becomes the bottleneck: write latency rises, queries slow down, and CPU and disk approach saturation, while the other nodes in the cluster sit idle.

At this point you want to add partitions and spread the load across nodes, but the partition rules can no longer be changed. The traditional path is the only one available:

  1. Create a new table with the new partition rules.
  2. Change the application side to dual-write to both the old and new tables.
  3. Backfill historical data from the old table into the new one.
  4. Verify consistency, cut over read traffic, and finally retire the old table.

Migrating TB-scale historical data takes a long time, consumes bandwidth, and affects the online workload, all while requiring application changes. So the partition scheme usually has to be planned up front, and adjusting it later is expensive.

This is exactly the gap GreptimeDB v1.1 closes: you can add partitions online to a table that already exists and isn’t partitioned.

v1.1: online partitioning in a single DDL​

Run the following statement:

sql

ALTER TABLE sensor_readings PARTITION ON COLUMNS (device_id, area) (
  device_id < 100 AND area < 'South',
  device_id < 100 AND area >= 'South',
  device_id >= 100 AND area <= 'East',
  device_id >= 100 AND area > 'East'
);

After it runs, thesensor\_readingstable, previously concentrated in a single Region, is split into four partitions that are scheduled onto four nodes. The load spreads out, and write throughput scales with the number of nodes. There’s no new table, no dual-write, no historical backfill, and no change to application code.

GreptimeDB online repartitioning overviewOnline repartitioning at a glance: from an overloaded single Region, to a singleALTER TABLEthat spreads across nodes, to continuous tuning with SPLIT / MERGE.## How it works: decoupled compute and storage​

This capability doesn’t come from a piece of SQL syntax. It comes from the underlying architecture: the decoupled compute and storage design.

In a traditional shared-nothing architecture, each node’s data sits on its own local disk. Repartitioning means physically moving data between nodes, copying TB-scale files from one machine to another. That process is slow and resource-heavy, which is a big reason many systems simply don’t support changing partitions after a table is created.

GreptimeDB stores its data in shared object storage (S3 and compatible stores), not bound to any single node’s local disk. A Region is a logical shard: it references data files in object storage through manifest files rather than holding the data itself. Repartitioning is therefore a metadata operation. It updates the manifest file references for each Region, switches to the new partition layout, and recomputes routing. The data itself never moves between nodes.

GreptimeDB storage-compute separationIn a coupled architecture, data is bound to nodes and repartitioning means moving it; with GreptimeDB’s decoupled storage, repartitioning is just a metadata switch.This is the elasticity that decoupled compute and storage brings: compute and storage scale independently, and the partition layout can follow the workload instead of being fixed at table creation. On a local-disk architecture, repartitioning a large table is an hours-long data migration. In GreptimeDB, the same task is an online metadata switch.

Writes may fluctuate briefly during repartitioning. The official guidance is to enable client-side retries, which smooths the transition.

After partitioning: splitting and merging​

Turning an unpartitioned table into a partitioned one is only the first step. Data distribution changes over time, and a layout that’s balanced today may develop hotspots or cold regions tomorrow. GreptimeDB offers two operations to adjust existing partitions:SPLIT PARTITIONsplits an overheated partition to increase write concurrency, andMERGE PARTITIONmerges partitions that have grown cold and small to reclaim resources.

sql

-- Split a hot partition
ALTER TABLE sensor_readings SPLIT PARTITION (
  device_id < 100 AND area < 'South'
) INTO (
  device_id < 50  AND area < 'South',
  device_id >= 50 AND device_id < 100 AND area < 'South'
);

-- Merge cold partitions
ALTER TABLE sensor_readings MERGE PARTITION (
  device_id >= 100 AND area <= 'East',
  device_id >= 100 AND area > 'East'
);

Beyond splitting on existing partition columns,SPLIT PARTITION \.\.\. ON COLUMNScan introduce a new partition column as part of the split. For example, a table initially partitioned only bydevice\_idcan later be divided further byarea:

sql

-- Initially partitioned by device_id only
ALTER TABLE sensor_readings PARTITION ON COLUMNS (device_id) (
  device_id < 100,
  device_id >= 100
);

-- Introduce a new partition column, area, during the split
ALTER TABLE sensor_readings SPLIT PARTITION (
  device_id < 100
) ON COLUMNS (device_id, area) INTO (
  device_id < 100 AND area < 'South',
  device_id < 100 AND area >= 'South'
);

After the split, the table’s partition columns become\(device\_id, area\). So the partitioning dimensions themselves can evolve with the business, not just the boundaries within existing dimensions.

For a complete step-by-step walkthrough of splitting and merging on an already-partitioned table, see the companion tutorial:How to Split and Merge Partitions Online in GreptimeDB.

Together with the initial partitioning fromPARTITION ON COLUMNS, this forms a complete flow: from unpartitioned, to partitioned, to continuously adjusted as the load changes.

The operational loop: diagnose, execute, iterate​

Online repartitioning turns capacity management into an observable, iterable loop.

Start by diagnosing which Region is under heavy load. Join Region-level statistics with the partition rules:

sql

SELECT
  t.table_name,
  r.region_id,
  r.region_number,
  p.partition_name,
  p.partition_description,
  r.written_bytes_since_open,
  r.region_rows
FROM information_schema.region_statistics r
JOIN information_schema.tables t
  ON r.table_id = t.table_id
JOIN information_schema.partitions p
  ON p.table_schema = t.table_schema
  AND p.table_name = t.table_name
  AND p.greptime_partition_id = r.region_id
WHERE t.table_schema = 'public'
  AND t.table_name = 'sensor_readings'
ORDER BY r.written_bytes_since_open DESC
LIMIT 10;

A partition whosewritten\_bytes\_since\_openstays consistently high is a good candidate for splitting. At the same time, queryinformation\_schema\.region\_peersto confirm the corresponding nodes are healthy, so you don’t mistake node instability for a hotspot.

Beyond SQL, GreptimeDB’s built-inGrafana dashboardsnow include a Hotspot section (Hotspot Regions, Datanode write load, and data distribution) that surfaces hot Regions and write-skewed Datanodes visually, seegreptimedb#8098.

Next, execute: run the matchingALTER TABLEonline. This updates the metadata, and writes may fluctuate briefly.

Then iterate: watch the new load distribution and keep splitting hot partitions or merging cold ones as needed.

Things to keep in mind​

Online repartitioning depends on the decoupled compute and storage architecture, so there are a few prerequisites:

  • It’s supported only in distributed clusters. Standalone deployments can’t repartition.
  • You must use shared object storage (such as AWS S3) so all Datanodes can access the same copy of data.
  • GC must be enabled on Metasrv and all Datanodes. Object storage holds the Region files, and GC reclaims a file only after its old references are released, which prevents accidentally deleting data still in use during repartitioning.

A few practical details:

  • The partition arguments toSPLITandMERGEmust exactly match an existing partition rule. The common cases are 1-to-2 splits and 2-to-1 merges; more complex changes can be done in several split and merge steps.
  • Repartitioning statements support asynchronous execution. AddingWITH \(WAIT = false\)returns aprocedure\_idimmediately, which you can track withADMIN procedure\_state\(procedure\_id\).TIMEOUTcontrols the overall time limit and applies whetherWAITis true or false.

sql

ALTER TABLE sensor_readings SPLIT PARTITION (
  device_id < 100 AND area < 'South'
) INTO (
  device_id < 50  AND area < 'South',
  device_id >= 50 AND device_id < 100 AND area < 'South'
) WITH (
  TIMEOUT = '5m',
  WAIT = false
);

What this means if you’re evaluating GreptimeDB​

If you’re evaluating GreptimeDB’s scalability and operational cost, online repartitioning changes two things.

The first is scalability. A table’s capacity is no longer constrained by the partition decision made at creation time. An online table can be repartitioned and scaled out as data grows, with no downtime and no rebuild.

The second is operational cost. Thanks to decoupled compute and storage, repartitioning is a lightweight metadata switch rather than a TB-scale data migration. The partition plan doesn’t have to be fixed once at table creation; it can be adjusted continuously as the load changes.

Enterprise: automatic load balancing with Autopilot​

Everything above is manual repartitioning: you diagnose the hotspots, decide whether to split or merge, and run the statements yourself. GreptimeDB Enterprise’s Autopilot automates this workflow. It runs inside Metasrv, continuously collecting write statistics from Datanodes and Regions, smoothing out short-term fluctuations, and submitting scheduling actions when the configured conditions are met.

Autopilot includes two complementary strategies:

  • Auto Repartitionautomatically splits large Regions that may become bottlenecks into smaller ones. It works on tables that already have partition rules. Once you’ve partitioned a table withPARTITION ON COLUMNS, the ongoing splitting can be handed off to it; it won’t generate partitions for a table that has no partition rules at all.
  • Region Balancerautomatically migrates hot Regions across Datanodes to balance the write load among nodes.

The two share the same runtime and cluster statistics, and together they keep the partition distribution and node load evenly balanced, taking the manual diagnosis and decision-making off your plate. You can tune their behavior through configuration options such asplugins\.auto\_repartitionandplugins\.region\_balancer, including the split trigger ratio, the maximum number of parts per split, and cooldown periods.

Further reading​

Similar Articles