Appendix: Alternative cluster architectures

If the recommended three-tier architecture is infeasible or unnecessary due to low volume or a non-production setup, a two-tier or a one-tier architecture may suffice.

Two-tier architecture

If the recommended three-tier architecture is infeasible or unnecessary due to low volume or a non-production setup, a two-tier architecture may suffice. A two-tier architecture consists of:

Image of the 2-cluster architecture: Source cluster, Compute/Transform +
Serving cluster

Tier	Description
Source cluster(s)	A dedicated cluster(s) for sources. In addition, for upsert sources: Consider separating upsert sources from your other sources. Upsert sources have higher resource requirements (since, for upsert sources, Materialize maintains each key and associated last value for the key as well as to perform deduplication). As such, if possible, use a separate source cluster for upsert sources. Consider using a larger cluster size during snapshotting for upsert sources. Once the snapshotting operation is complete, you can downsize the cluster to align with the steady-state ingestion.
Compute/Transform + Serving cluster	A dedicated cluster for both compute/transformation and serving queries: Views that define the transformations. Indexes on views to maintain up-to-date results in memory and serve queries. With a two-tier architecture, compute and queries compete for the same cluster resources. 💡 Tip: Except for when used with a sink, subscribe, or temporal filters, avoid creating materialized views on a shared cluster used for both compute/transformat operations and serving queries. Use indexed views instead.

Tier

Description

Source cluster(s)

A dedicated cluster(s) for sources.

In addition, for upsert sources:

Consider separating upsert sources from your other sources. Upsert sources have higher resource requirements (since, for upsert sources, Materialize maintains each key and associated last value for the key as well as to perform deduplication). As such, if possible, use a separate source cluster for upsert sources.
Consider using a larger cluster size during snapshotting for upsert sources. Once the snapshotting operation is complete, you can downsize the cluster to align with the steady-state ingestion.

Compute/Transform + Serving cluster

A dedicated cluster for both compute/transformation and serving queries:

Views that define the transformations.
Indexes on views to maintain up-to-date results in memory and serve queries.

With a two-tier architecture, compute and queries compete for the same cluster resources.

💡 Tip: Except for when used with a sink, subscribe, or temporal filters, avoid creating materialized views on a shared cluster used for both compute/transformat operations and serving queries. Use indexed views instead.

Benefits of a two-tier architecture include:

Support for blue/green deployments
More cost effective than a three-tier architecture.

However, with a two-tier architecture:

Compute/transform operations and queries compete for the same cluster resources.
Cluster restarts require rehydration of the indexes on views.

One-tier architecture

If the recommended three-tier architecture is infeasible or unnecessary due to low volume or a non-production setup, a one-tier architecture may suffice for your sources, compute objects, and query serving needs.

Image of the 1-cluster-architecture
architecture

Tier	Description
All-in-one cluster	A cluster for sources, compute/transformation and serving queries: Sources to ingest data. Views that define the transformations. Indexes on views to maintain up-to-date results in memory and serve queries. With a 1-tier single-cluster architecture, sources, compute, and queries compete for the same cluster resources. 💡 Tip: Except for when used with a sink, subscribe, or temporal filters, avoid creating materialized views on a shared cluster used for both compute/transformat operations and serving queries. Use indexed views instead.

Tier

Description

All-in-one cluster

A cluster for sources, compute/transformation and serving queries:

Sources to ingest data.
Views that define the transformations.
Indexes on views to maintain up-to-date results in memory and serve queries.

With a 1-tier single-cluster architecture, sources, compute, and queries compete for the same cluster resources.

Benefits of a one-tier architecture include:

Cost effective

Limitations of a one-tier architecture include:

Sources, compute objects, and queries compete for cluster resources.
Blue/green deployment is unsupported since sources would need to be dropped and recreated, putting strain on your upstream system during source recreation.

To support blue/green deployments, use a two-tier architecture by moving compute objects to a new cluster (i.e., recreating compute objects in a new cluster).
Cluster restarts require rehydration of the indexes on views.