Cluster sizes

Default Cluster Sizes

For Self-Managed Materialize, the cluster sizes are configured with the following default resource allocations:

Size	Scale	CPU Limit	Disk Limit	Memory Limit
`25cc`	`1`	`0.5`	`7762MiB`	`3881MiB`
`50cc`	`1`	`1`	`15525MiB`	`7762MiB`
`100cc`	`1`	`2`	`31050MiB`	`15525MiB`
`200cc`	`1`	`4`	`62100MiB`	`31050MiB`
`300cc`	`1`	`6`	`93150MiB`	`46575MiB`
`400cc`	`1`	`8`	`124201MiB`	`62100MiB`
`600cc`	`1`	`12`	`186301MiB`	`93150MiB`
`800cc`	`1`	`16`	`248402MiB`	`124201MiB`
`1200cc`	`1`	`24`	`372603MiB`	`186301MiB`
`1600cc`	`1`	`31`	`481280MiB`	`240640MiB`
`3200cc`	`1`	`62`	`962560MiB`	`481280MiB`
`6400cc`	`2`	`62`	`962560MiB`	`481280MiB`

Custom Cluster Sizes

When installing the Materialize Helm chart, you can override the default cluster sizes and resource allocations. These cluster sizes are used for both internal clusters, such as the system_cluster, as well as user clusters.

💡 Tip: In general, you should not have to override the defaults. At minimum, we recommend that you keep the 25-200cc cluster sizes.

operator:
  clusters:
    sizes:
      <size>:
        workers: <int>
        scale: 1                  # Generally, should be set to 1.
        cpu_exclusive: <bool>
        cpu_limit: <float>         # e.g., 6
        cpu_request: <float>       # e.g., 4 (optional, defaults to cpu_limit, may not be higher than cpu_limit)
        credits_per_hour: "0.0"    # N/A for self-managed.
        disk_limit: <string>       # e.g., "93150MiB"
        memory_limit: <string>     # e.g., "46575MiB"
        swap_enabled: <bool>       # optional, defaults to the cluster-level swap_enabled
        selectors: <map>           # k8s label selectors
        # ex: kubernetes.io/arch: amd64

Field	Type	Description	Recommendation
workers	int	The number of timely workers in your cluster replica.	Use 1 worker per CPU core, with a minimum of 1 worker.
scale	int	The number of pods (i.e., processes) to use in a cluster replica; used to scale out replicas horizontally. Each pod will be provisioned using the settings defined in the size definition.	Generally, this should be set to 1. This should only be greater than 1 when a replica needs to take on limits that are greater than the maximum limits permitted on a single node.
cpu_exclusive	bool	The flag that determines if the workers should attempt to pin to a particular CPU core.	Set to true if and only if the `cpu_limit` is a whole number and the CPU management policy in the k8s cluster is set to static.
cpu_limit	float	The Kubernetes CPU limit for a replica pod, in cores.	Prefer whole number values to enable CPU affinity. Kubernetes only allows CPU Affinity for pods taking a whole number of cores. If the value is not a whole number, set `cpu_exclusive` to false.
cpu_request	float	The Kubernetes CPU request for a replica pod, in cores. If not set, defaults to the value of `cpu_limit`.	In most cases, you do not need to set this. It is useful when you want to allow CPU bursting by setting a request lower than the limit.
memory_limit	string	The Kubernetes memory limit for a replica pod (e.g., `"46575MiB"`).	For most workloads, use an approximate 1:8 CPU-to-memory ratio (1 core 8 GiB). This can vary depending on your workload characteristics.
disk_limit	string	The size of the persistent volume to provision for a replica pod (e.g., `"93150MiB"`).	When spill-to-disk is enabled, use a 1:2 memory-to-disk ratio. Materialize spills data to disk when memory is insufficient, which can impact performance. When `swap_enabled` is true, this field is automatically set to `"0"` by the Helm chart.
credits_per_hour	string	This is a cloud attribute that should be set to “0.00” in self-managed.	Set to “0.00” for self-managed deployments.
swap_enabled	bool	Enables swap as the spill-to-disk mechanism for this size. When enabled, the replica uses swap instead of a provisioned persistent volume for spilling data. This also causes `disk_limit` to be set to `"0"`.	This defaults to the global `swap_enabled` value if not specified per size. Swap generally performs better than spill-to-disk via persistent volumes.
selectors	map	A map of Kubernetes label selector keys to values used to schedule pods for this cluster size on specific nodes.	It is generally not required to set this.

NOTE: If you have modified the default cluster size configurations, you can query the mz_cluster_replica_sizes system catalog table for the specific resource allocations.