Pricing

Resource demands of individual workloads is highly dependent on three factors below, so we recommend finding the right size by initially running on medium clusters and then checking the resource utilization metrics. Since each size has 2x the resources, if your workload is using less than 50% of resources you can downsize one level, 25% two levels, etc… Up and downscaling can be done without disruption by creating additional replicas of different sizes. The three factors that most heavily influence the resource demands of a Materialize workload are:

1. Size and type of dataset - Streaming input Sources for larger datasets may need to run on larger cluster replicas, and high-cardinality data kept in storage will require larger cluster replicas to process and index.
2. Throughput of changes - Changes (Updates/Inserts/Deletes) are what trigger work in Materialize, so the more often data is changing, the more computation work Materialize does.
3. Quantity and complexity of transformations - Like in a traditional database, the amount of resources needed to compute a specific SQL query can vary dramatically based on the joins, aggregations, window queries, CTE’s, subqueries, and computations needed.

Clusters can be used strategically to isolate your workloads from failure in several ways.

• Separation of Source ingest responsibilities - Streaming input Sources run on their own cluster (either one source per cluster, or multiple sources on a cluster.)
• Separation of use cases - Teams can isolate separate use cases (e.g. Reporting, Feature Serving, Alerting) to ensure changes to one don’t affect the others.
• Separation of dev/stage/prod - Teams can have smaller separate clusters for development and staging work.
• Separation of compute stages - Teams can use materialized views to write outputs of a SQL transformation down to storage, and then pull the results back up into new clusters for more computation or serving. This allows for creation of pipelines that can be architected to continue to serve stale results in the case of an upstream failure. Further reading: Clusters Explained

Materialize uses active replication: there is no primary/secondary, all of the computation assigned to a cluster is run on all replicas.

Benefits of continuous active replication:

• Increased Availability - Running with 2+ replicas means if one suffers a temporary outage due to hardware failure, other replicas are still operating and end-users will not experience disruption.
• Lower P99 time - The SQL control layer routes requests to all replicas and sends the user the first response it receives. Each replica is competing to answer queries fastest, so increasing the number of replicas decreases the standard deviation of response times.

Benefits of temporary active replication:

• Upscaling/Downscaling - If a S replica has high resource utilization, create a second M in parallel, wait for it to come online and when ready remove the old S for zero-downtime upscaling. The same process works for downscaling. Further reading: Clusters Explained

Storage is roughly proportional to the size of your source datasets plus the size of materialized views, with some overhead from uncompacted previous values of data and system metrics. But more broadly, two factors lead to storage costs making up a very small percentage of real-world Materialize invoices:

1. Materialize uses cheap, scalable object storage (currently S3 on AWS) as a storage layer, and primarily passes the cost through to the customer, At a rate of $0.0000411 per GB/hr, 1TB stored for one month (730hrs) equates to $30 USD.
2. With the exception of Append-only sources, most data in Materialize is continually compacted, so the total state stored in Materialize tends to grow at a rate more similar to OLTP databases than traditional cloud data warehouses.

See Free Trial FAQs

If you have already committed spend in AWS, you can put it towards Materialize credits through the AWS marketplace. Get in touch here and we can help you through the process.