ALTER CLUSTER

Use ALTER CLUSTER to:

• Change configuration of a cluster, such as the SIZE or REPLICATON FACTOR.
• Rename a cluster.
• Change owner of a cluster.

For completeness, the syntax for SWAP WITH operation is provided. However, in general, you will not need to manually perform this operation.

Syntax

ALTER CLUSTER has the following syntax variations:

ALTER CLUSTER <cluster_name>
SET (
    [SIZE = <text>]
    [, REPLICATION FACTOR = <int>]
    [, MANAGED = <bool>]
    [, AUTO SCALING STRATEGY = (
        ON HYDRATION (
            HYDRATION SIZE = <text>
            [, LINGER DURATION = <interval>]
        )
    )]
)
[WITH ( <with_option>[,...])]
;

ALTER CLUSTER <cluster_name>
RESET (
    REPLICATION FACTOR | MANAGED | AUTO SCALING STRATEGY,
    ...
)
;

ALTER CLUSTER <cluster_name> RENAME TO <new_cluster_name>;

ALTER CLUSTER <cluster_name> OWNER TO <new_owner_role>;

ALTER CLUSTER <cluster1> SWAP WITH <cluster2>;

Considerations

Resizing

Available sizes

See also:

• cc to M.1 size mapping.

• Materialize service consumption table.

• Blog:Scaling Beyond Memory: How Materialize Uses Swap for Larger Workloads.

Resource allocation

To determine the specific resource allocation for a given cluster size, query the mz_cluster_replica_sizes system catalog table.

Downtime considerations for v26.35 or after

Starting in v26.35, ALTER CLUSTER SET (SIZE = …) by default resizes the cluster gracefully and without downtime. For example:

ALTER CLUSTER c1 SET (SIZE = '100cc');

Resizing process

The resize proceeds in the background, allowing the command to return immediately.

During a graceful resize, Materialize:

1. Provisions new replicas at the target size, alongside the current replicas.
2. Waits for the new replicas to hydrate.
3. Retires the old replicas.

Throughout, the cluster keeps serving queries, first from the old replicas, then from both sets as the new replicas come up, so the resize incurs no downtime.

If the new replicas do not hydrate within the reconfiguration timeout (24 hours by default), Materialize rolls back the resize and the cluster keeps its current size. To customize the timeout behavior, use the WAIT UNTIL READY or WAIT FOR options. The resize still proceeds in the background.

• WAIT UNTIL READY (TIMEOUT = ..., ON TIMEOUT = ...) sets the timeout for the resize. On timeout, ON TIMEOUT selects whether to COMMIT (retire the old replicas and proceed with the not-yet-hydrated new ones, which can cause downtime) or ROLLBACK (keep the current size). Default: ROLLBACK.

  ALTER CLUSTER c1
  SET (SIZE = '100cc') WITH (WAIT UNTIL READY (TIMEOUT = '10m'));
  

• WAIT FOR '<duration>' sets the timeout and commits when it expires, regardless of hydration status, which can cause downtime. Prefer WAIT UNTIL READY.

See Monitoring a resize to track progress and cancel an in-flight resize.

Monitoring a resize

You can monitor a resize through the following:

• The activity column of SHOW CLUSTERS, which summarizes any in-flight reconfiguration or hydration burst, and is NULL when the cluster is steady.

• mz_internal.mz_cluster_reconfigurations, which shows the target shape, deadline, timeout action, and lifecycle status of the latest reconfiguration.

• mz_internal.mz_cluster_auto_scaling_strategies, which shows any in-flight hydration burst.

• mz_internal.mz_hydration_statuses, which shows per-object hydration status.

• The audit log (mz_catalog.mz_audit_events), which records each reconfiguration transition.

Cancel a resize

To cancel an in-flight resize, reissue ALTER CLUSTER with the cluster’s current size. Materialize drops the pending replicas and keeps the current configuration.

Downtime considerations for v26.34 or before

You can use the WAIT UNTIL READY option to perform a zero-downtime resizing, which incurs no downtime. Instead of restarting the cluster, this approach spins up an additional cluster replica under the covers with the desired new size, waits for the replica to be hydrated, and then replaces the original replica.

ALTER CLUSTER c1
SET (SIZE '100cc') WITH (WAIT UNTIL READY (TIMEOUT = '10m', ON TIMEOUT = 'COMMIT'));

The ALTER statement is blocking and will return only when the new replica becomes ready. This could take as long as the specified timeout. During this operation, any other reconfiguration command issued against this cluster will fail. Additionally, any connection interruption or statement cancelation will cause a rollback — no size change will take effect in that case.

Speed up hydration by autoscaling to a larger size

Beyond a one-off resize, you can configure a standing autoscaling strategy so the cluster provisions a burst replica at a larger size on its own whenever it has un-hydrated objects. You can set the strategy when you first create the cluster with CREATE CLUSTER ... (AUTO SCALING STRATEGY = ...), or add it to an existing cluster with ALTER CLUSTER ... SET (AUTO SCALING STRATEGY = ...). The example below uses CREATE CLUSTER; see Configure autoscaling for the ALTER CLUSTER form.

When you create an index, materialized view, or Kafka upsert source, or when a cluster restarts, the cluster must hydrate the affected objects before they can serve results. Hydration reads the input data and rebuilds in-memory state, and its speed scales with the cluster size.

The AUTO SCALING STRATEGY (ON HYDRATION) option lets a cluster automatically provision an extra burst replica at the configured HYDRATION SIZE while it has un-hydrated objects. This speeds up hydration without manually scaling the cluster up before hydration and back down afterward. The steady-size replicas continue hydrating in parallel, and once one of them catches up with the burst, the burst replica lingers for the LINGER DURATION and is then removed. The burst replica is an ordinary cluster replica, billed only for the time it is provisioned. See Usage & billing for details.

AUTO SCALING STRATEGY (ON HYDRATION) is particularly useful for blue/green deployments, where a new cluster must hydrate before the cutover. It is only available on managed clusters, and cannot be combined with a cluster SCHEDULE other than the default MANUAL.

For example, the following cluster can provision a burst replica of size 800cc:

CREATE CLUSTER fast_start (
    SIZE = '100cc',
    AUTO SCALING STRATEGY = (
        ON HYDRATION (
            HYDRATION SIZE = '800cc',
            LINGER DURATION = '15s'
        )
    )
);

You can specify the following options:

Provisioning the burst replica requires enough compute capacity to run it. In Materialize Self-Managed, this means your Kubernetes cluster must have enough spare resources (for example, available nodes) to schedule the burst replica.

The burst is best-effort and never blocks the cluster: if the burst replica cannot be provisioned, the steady-size replicas still come up and hydrate as usual, as long as there are enough resources for them.

To remove the autoscaling strategy from a cluster, use ALTER CLUSTER ... RESET (AUTO SCALING STRATEGY) or set an empty strategy with AUTO SCALING STRATEGY = ().

You can inspect the configured strategy and any in-flight burst in the mz_internal.mz_cluster_auto_scaling_strategies catalog view.

Replication factor

The REPLICATION FACTOR option determines the number of replicas provisioned for the cluster. Each replica of the cluster provisions a new pool of compute resources to perform exactly the same computations on exactly the same data. Each replica incurs cost, calculated as cluster size * replication factor per second. See Usage & billing for more details.

Replication factor and fault tolerance

Provisioning more than one replica provides fault tolerance. Clusters with multiple replicas can tolerate failures of the underlying hardware that cause a replica to become unreachable. As long as one replica of the cluster remains available, the cluster can continue to maintain dataflows and serve queries.

Materialize automatically assigns names to replicas (e.g., r1, r2). You can view information about individual replicas in the Materialize console and the system catalog.

Availability guarantees

When provisioning replicas,

• For clusters sized under 3200cc, Materialize guarantees that all provisioned replicas in a cluster are spread across the underlying cloud provider’s availability zones.

• For clusters sized at 3200cc and above, even distribution of replicas across availability zones cannot be guaranteed.

Required privileges

To execute the ALTER CLUSTER command, you need:

• Ownership of the cluster.

• To rename a cluster, you must also have membership in the <new_owner_role>.

• To swap names with another cluster, you must also have ownership of the other cluster.

See also:

• Access control (Materialize Cloud)
• Access control (Materialize Self-Managed)

Rename restrictions

You cannot rename system clusters, such as mz_system and mz_catalog_server.

Examples

Replication factor

The following example uses ALTER CLUSTER to update the REPLICATION FACTOR of cluster c1 to 2:

ALTER CLUSTER c1 SET (REPLICATION FACTOR 2);

Increasing the REPLICATION FACTOR increases the cluster’s fault tolerance, not its work capacity.

Resizing

By default, altering the cluster size is graceful and incurs no downtime. The command returns immediately and the resize proceeds in the background. See Resizing process and Monitoring a resize.

ALTER CLUSTER c1 SET (SIZE = '100cc');

To customize the timeout and what happens when it expires, use the WAIT UNTIL READY option:

ALTER CLUSTER c1
SET (SIZE = '100cc') WITH (WAIT UNTIL READY (TIMEOUT = '10m', ON TIMEOUT = 'ROLLBACK'));

Configure autoscaling

To speed up hydration, configure an autoscaling strategy that provisions a burst replica at a larger size while the cluster has un-hydrated objects:

ALTER CLUSTER c1 SET (
    AUTO SCALING STRATEGY = (
        ON HYDRATION (HYDRATION SIZE = '800cc', LINGER DURATION = '15s')
    )
);

To remove the strategy:

ALTER CLUSTER c1 RESET (AUTO SCALING STRATEGY);

To inspect the configured strategy and any in-flight burst, query mz_internal.mz_cluster_auto_scaling_strategies. The strategy column holds the configured policy, and the state column holds the in-flight burst details, or NULL when no burst is running:

SELECT
    c.name AS cluster,
    s.strategy->'on_hydration'->>'hydration_size' AS hydration_size,
    (s.strategy->'on_hydration'->'linger_duration'->>'secs')::int AS linger_seconds,
    s.state->'burst'->>'burst_size' AS inflight_burst_size
FROM mz_internal.mz_cluster_auto_scaling_strategies AS s
JOIN mz_clusters AS c ON c.id = s.cluster_id;

 cluster | hydration_size | linger_seconds | inflight_burst_size
---------+----------------+----------------+---------------------
 c1      | 800cc          |             15 |

Here, c1 is configured to provision an 800cc burst replica that lingers 15 seconds, and no burst is currently running (inflight_burst_size is NULL). While a burst is in flight, inflight_burst_size reports the burst replica’s size.

SHOW CLUSTERS also summarizes any in-flight hydration burst in its activity column.

Converting unmanaged to managed clusters

Alter the managed status of a cluster to managed:

ALTER CLUSTER c1 SET (MANAGED);

Materialize permits converting an unmanged cluster to a managed cluster if the following conditions are met:

• The cluster replica names are r1, r2, …, rN.
• All replicas have the same size.
• If there are no replicas, SIZE needs to be specified.
• If specified, the replication factor must match the number of replicas.

Note that the cluster will not have settings for the availability zones, and compute-specific settings. If needed, these can be set explicitly.

See also

• CREATE CLUSTER
• SHOW CLUSTERS
• DROP CLUSTER