Strong Consistency Guarantees for Streaming Data
Materialize guarantees strict serializable consistency that is standard in databases, but unprecendented in distributed stream processors.
Consistency in Streaming →The Cloud Database for
|
We put a streaming engine in a database, so your team can build with real-time data faster, under budget, in the same SQL workflows as a cloud data warehouse.
Trusted by data and engineering teams
View Customer Stories.
What is Materialize?
A Streaming Database
Materialize is a Postgres wire-compatible database with an operating model identical to cloud data warehouses like Snowflake. But, computation is done at the moment of data arrival, rather than query time, making results available almost instantly.
Connect Inputs
Stream updates directly from your Postgres DB, or any other OLTP database via change-data-capture data in Kafka.
Write SQL
Write complex joins, aggregations, transformations of your data in standard SQL.
Serve Results
Work is done at the moment of data arrival, rather than query time, so that maintained results are available almost instantly.
Why is it useful?
Built from the ground up to solve the hard problems in data.
Streamlines solutions to historically challenging problems
Integrates Seamlessly
Works with your existing stacks, pipelines and workflows
Speaks SQL
Replaces complex, bespoke architectures with standard SQL
Scales Effortlessly
Makes developing on data more accessible across an organization
Re-Introducing Materialized Views:
The Missing Building Block in Databases
Materialized Views are as old as databases. They're supposed to be SQL queries whose results are updated as source data changes. But they've always been broken.
It turns out when they work they're quite useful. So we fixed Materialized Views.
Problem
Complex queries on OLTP DB are too slow
Elaborate on problem
Problem
New Consumers from Warehouse demand fresher data
Think about the growing category of use cases that demand access to fresh analytical-style data: Audience Segmentation and Personalization, Real-Time Analytics, Sales and Marketing Automation, Fraud Detection, ML in Production.
Problem
Stream Processing is too complicated
Elaborate on problem
Use Cases
What Can You Build with Materialize?
Data Engineers and Developers use Materialize as a better engine for cutting-edge data products.
Real-Time Analytics
Use the same ANSI SQL from data warehouses to build real-time views that serve internal and customer-facing dashboards, APIs and apps.
Automation and Alerting
Build user-facing notifications, fraud and risk models, and automated services using event-driven SQL primitives in a streaming database.
Segmentation and Personalization
Build engaging experiences with customer data aggregations that are always up-to-date: personalization, recommendations, dynamic pricing and more.
ML Ops
Power online feature stores with continually updated data, monitor and react to changes in ML effectiveness - all in standard SQL.
Key Features
Presents as PostgreSQL
Manage and query Materialize using any Postgres driver or tool.
Built for JOINs
Multi-way, complex join support across real-time streams - all in standard SQL.
Separate Storage & Compute
Data is stored cheaply, compute scales independently and without limits.
Incremental Compute Engine
Instead of re-computing on every query, results are updated as data changes.
Active Replication
Use replication to increase availability, reduce downtime, scale seamlessly.
Low-Latency Reads
Results can be maintained in memory, making read latency similar to Redis.
Managed in standard SQL
Give your team streaming data capabilities without changing their language or workflow.
Incrementally Maintained Views
Write complex SQL transformations as materialized views that efficiently update themselves as inputs change.
Sliding Windows
Write queries that filter to a window of time anchored to the present, Materialize will update results as time advances.
SQL Alerting
Write alerts as SQL queries with filters and subscribe to new rows as they appear.
1CREATE MATERIALIZED VIEW my_view AS
2 SELECT userid, COUNT(api.id), COUNT(pageviews.id)
3 FROM users
4 JOIN pageviews on users.id = pageviews.userid
5 JOIN api ON users.id = api.userId
6 GROUP BY userid;$
| userID | api_calls | pageviews |
|---|---|---|
| VPLaKV | 400 | 20 |
| MN37Mt | 60 | 9 |
| 1fT4KY | 72 | 42 |
| sT4QY | 10 | 342 |
Incrementally Maintained Views
Write complex SQL transformations as materialized views that efficiently update themselves as inputs change.
Learn More1CREATE MATERIALIZED VIEW my_view AS
2 SELECT userid, COUNT(api.id), COUNT(pageviews.id)
3 FROM users
4 JOIN pageviews on users.id = pageviews.userid
5 JOIN api ON users.id = api.userId
6 GROUP BY userid;$
| userID | api_calls | pageviews |
|---|---|---|
| VPLaKV | 400 | 20 |
| MN37Mt | 60 | 9 |
| 1fT4KY | 72 | 42 |
| sT4QY | 10 | 342 |
Sliding Windows
Write queries that filter to a window of time anchored to the present, Materialize will update results as time advances.
Learn More1CREATE MATERIALIZED VIEW my_window AS
2 SELECT date_trunc('minute', received_at),
3 COUNT(*) as order_ct, SUM(amount) as revenue
4 FROM orders
5 WHERE mz_now() < received_at + interval '5 minutes'
6 GROUP BY 1;$
| minute | order_ct | revenue |
|---|
SQL Alerting
Write alerts as SQL queries with filters and subscribe to new rows as they appear.
Learn More1SELECT userID, email, MAX(orders.id) as last_order
2FROM users
3JOIN orders ON orders.userID = users.id
4GROUP BY userId, email
5-- Use a filter to surface users with a high % of fraud
6HAVING SUM(is_fraud) / COUNT(orders.id)::FLOAT > 0.5;$
| userID | last_order | |
|---|---|---|
| REOtIb | a@gmail.com | 13/12/2022 |
| Y5KBE8 | b@yahoo.com | 9/12/2022 |
| Wj7JQ0 | c@hotmail.com | 13/12/2022 |
| tPCQ0 | d@xyz.com | 13/11/2022 |
Why Materialize?
Built from the ground up to solve the hard problems in data.
Multi-way and Cross-Stream Joins
Write the same kinds of complex, multi-way SQL joins you would use on a traditional data warehouse. Materialize maintains the joins efficiently in memory and serves real-time results.
Joins in Materialize →
Event-Driven Primitives: Sink and Subscribe
Push updates out of the database without performance limits using two new primitives:
PostgreSQL Up Front, Timely Dataflow Underneath
Materialize expands access to powerful stream processing capabilities of Timely and Differential Dataflow by wrapping them in a familiar and accessible SQL layer that is wire-compatible with Postgres.
Postgres Compatibility Explainer →Works with Your Existing Data Stack
PostgreSQL
Streaming infrastructure is not required to use Materialize for real-time computation. Connect directly to any Postgres database via CDC and continually ingest data as it changes.
dbt
Run your existing dbt models on top of streaming data in Materialize, and dbt persists a materialized view. No matter how much or how frequently your data arrives, your model will stay up to date.
Kafka
Connect multiple Kafka topics to Materialize and easily explore, transform, and join streaming datasets - and sink maintained SQL query results downstream to new, enriched Kafka topics.
Trusted By Data Teams
Emily Hawkins
Data Infrastructure Lead, Drizly
We can write real-time SQL, exactly the same way as we already are in Snowflake with batch.
Ryan Gaus
Staff Engineer and Tech Lead, Density
Materialize has saved us I-don’t-know-how-many untold quarters of trying to build our own thing.
Jean-Francois Perreton
Head of Algo Quant, Kepler Chevreaux
Materialize directly integrates with our third-party applications, BI tools, you name it. It’s really SQL.
Tyler Richie
Cofounder and CTO, Sproutfi
With Materialize we don't have to worry about avoiding complex joins with streaming data; we can just do them very easily.
Johan Stuyts
Data Architect, Maqqie
Materialize is correct, and not just eventually consistent. The alternatives simply don’t support consistency, and you end up wasting a lot of time troubleshooting.
Think Declaratively, Act Incrementally
Materialize helps you get access to the power of a stream processing engine, with the simplicity of a PostgreSQL-compatible developer interface.
Fast
Millisecond-level latency through incrementally-updated views.
Familiar
Control it with ANSI-standard SQL. Connect with Postgres drivers.
Fully-Featured
Support for multi-way joins, subqueries, upserts, deletes, CTEs.
Sign Up for Access
Ready to see if Materialize works for your use case? Register for access today!
Footer
Our Product
Developer
Join the Materialize Community
Join hundreds of other Materialize users and connect directly with our engineers.
Join the Community© 2023 Materialize, Inc. Terms of Service