Blog
Featured
What is a Streaming Database?
Get an overview of how streaming databases differ from traditional DBs. What are the tradeoffs? How are they used?
Jan 4, 2023

Product Announcement
May 18, 2023
With major updates to the streaming replication connection to PostgreSQL, users can now set up Materialize as a drop-in enabler of real-time, incrementally updated, materialized views for their PostgreSQL database.
Read More →
Conceptual Article
May 11, 2023
When to use Materialize vs a Stream Processor

If you're already familiar with stream processors you may wonder: When is it better to use Materialize vs a Stream Processor? And why?
Read More →
May 11, 2023

Webinar
Data Products and Decentralized Data Teams with Materialize
Challenges scaling data teams are real, so why does it seem like real examples of “data mesh” and “data products” are so rare in the wild? In this talk, we’re going to try to tease out the good ideas in these patterns and figure out how we can move incrementally towards solving our data team bottlenecks, and how a platform such as Materialize can help us to move towards solutions without a massive upheaval.
Watch ReplayProduct Update
Apr 25, 2023
A Terraform Provider for Materialize

Materialize maintains an official Terraform Provider you can use to manage your clusters, replicas, connections and secrets as code.
Read More →
Deep Dive
Apr 20, 2023
Everything you need to know to be a Materialize power-user

You could just write SQL, get continually updated results on Materialize. But if you want to get more scale, performance, power, here is a gentle introduction to key internals that will help.
Read More →
Key Concept
Apr 5, 2023
The Four ACID Questions

Four questions, and their answers, to explain ACID transactions and how they are handled within Materialize.
Read More →
Key Concept
Mar 30, 2023
CDC in Production: An Operating Guide

We spent hours interviewing engineers running change data capture in production, and compiled a list of their planning tips, design patterns to use and footguns to avoid here.
Read More →
Mar 23, 2023

Tech Demo
CI/CD workflows for dbt+Materialize
Join Marta Paes (Head of Developer Experience) for a demo on setting up CI/CD workflows to deploy dbt projects to Materialize.
Watch ReplayConceptual Article
Mar 9, 2023
Towards Real-Time dbt

Here's a framework for thinking about reducing the time between when raw data is available to transform with dbt, and when it is delivering value to your customers.
Read More →
Technical Article
Feb 23, 2023
The Software Architecture of Materialize

Materialize aims to be usable by anyone who knows SQL, but for those interested in going deeper and understanding the architecture powering Materialize, this post is for you!
Read More →
Conceptual Article
Feb 16, 2023
When to Use Indexes and Materialized Views

If you are familiar with materialized views and indexes from other databases, this article will help you apply that understanding to Materialize.
Read More →
Technical Article
Feb 9, 2023
Building Differential Dataflow from scratch

Let's build (in Python) the Differential Dataflow framework at the heart of Materialize, and explain what it's doing along the way.
Read More →
Conceptual Article
Jan 31, 2023
Clusters, explained with Data Warehouses

If you're familiar with data warehouses, this article will help you understand Materialize Clusters in relation to well-known components in Snowflake.
Read More →
Technical Article
Jan 18, 2023
Understand how to optimize joins with indexes and late materialization.
Read More →
Technical Article
Jan 11, 2023
Recursion in Materialize

Differential Dataflow is capable of incrementally updated iterative computation (recursion) but we haven't yet wired it up to SQL. Let's talk about what recursion could look like in Materialize, and why it's important.
Read More →
Key Concept
Jan 4, 2023
What is a Streaming Database?

Get an overview of how streaming databases differ from traditional DBs. What are the tradeoffs? How are they used?
Read More →
Conceptual Article
Dec 6, 2022
Our experience with Rust!

Materialize is written in Rust. Why did we make that decision and how has it turned out for the project?
Read More →
Technical Article
Oct 19, 2022
Real-Time Customer Data Platform Views on Materialize

Let's demonstrate the unique features of Materialize by building the core functionality of a customer data platform.
Read More →
Key Concept
Oct 18, 2022
How and why is Materialize compatible with PostgreSQL?

As a streaming database, Materialize is fundamentally different on the inside, but it's compatible with PostgreSQL in a few important ways.
Read More →
Oct 13, 2022

Community Meetup
The next generation of Materialize , an Overview
Join Andy Hattemer and Joaquin Colacci from our Developer Experience team for a technical overview on the major updates to Materialize.
Watch ReplayProduct Update
Oct 3, 2022
Announcing the next generation of Materialize


Today, we’re excited to announce a product that we feel is transformational: a persistent, scalable, cloud-native Materialize.
Read More →
Integration Guide
Sep 7, 2022
How to Connect Segment to Materialize

Connect a real-time stream of user-centric events from Segment to Materialize, using Kafka as the intermediary.
Read More →
Technical Article
Jul 27, 2022
Indexes: A Silent Frenemy

Even in traditional databases, indexes can at different times be the problem and the solution when it comes to scaling. In this article we discuss how indexes change in streaming databases.
Read More →
Jul 21, 2022

Community Meetup
Materialize + dbt: From Streaming Analytics to Continuous Testing
Join Anna Glander from the Developer Experience team for a live session and Q&A on using Materialize + dbt to do continuous testing.
Watch ReplayTechnical Article
Jul 14, 2022
In traditional databases, a SQL query used as a test runs as a point-in-time check. In streaming, the same query can run continually as data changes, creating a SQL-based data monitoring primitive.
Read More →
Ecosystem & Integrations
Jun 15, 2022
Let's explore a hands-on example where we use dbt (data build tool) to manage and document a streaming analytics workflow from a message broker to Metabase.
Read More →
Technical Article
Jun 14, 2022
Virtual Time: The Secret to Strong Consistency and Scalable Performance in Materialize

The key to Materialize's ability to separate compute from storage and scale horizontally without sacrificing consistency is a concept called virtual time.
Read More →
Technical Article
Jun 9, 2022
Let’s talk about Data Apps

What is a Data Application? How do they help our customers? What new challenges do we face when building Data Apps? Here's our perspective.
Read More →
Ecosystem & Integrations
May 13, 2022
Announcing the Materialize Integration with Cube


Connect headless BI tool Cube.js to the read-side of Materialize to get Rest/GraphQL API's, Authentication, metrics modelling, and more out of the box.
Read More →
Product Update
May 6, 2022
Materialize's unbundled cloud architecture

The `materialized` binary is stable and performant, the time has come to break it apart into separate services to enable the next phase: unbounded scale in a cloud architecture.
Read More →
Technical Article
Apr 25, 2022
Creating a Real-Time Feature Store with Materialize

Let's use Materialize to deliver a feature store that continuously updates dimensions as new data becomes available without compromising on correctness or speed.
Read More →
Apr 22, 2022

Technical Overview of Materialize's New Unbundled Architecture
Join Materialize cofounder and chief scientist Frank McSherry on June 22nd at 2pm ET for a live session and Q&A on the upcoming unbundled architecture of Materialize.
Watch ReplayTechnical Article
Mar 3, 2022
Subscribe to changes in a view with Materialize

Developers have long wished for the ability to subscribe to changes in a SQL query or a view in a database. Materialize has a SUBSCRIBE primitive that makes it possible.
Read More →
Product Update
Mar 1, 2022
What's new in Materialize? Vol. 2


Changelog: AWS roles for S3/Kinesis, PostgreSQL source improvements, Schema Registry SSL, SELECT statements in Tail queries, jsonb subscripting, DBeaver support, & Tailscale in cloud.
Read More →
Community Meetup
Materialize + dbt + Redpanda Virtual Hack Day 2022
Come hang out with us and build a streaming project alongside folks from all over the world! We’ll help you get the inspiration flowing with a seed project and some ideas for real-time data sources.
Watch ReplayTechnical Article
Feb 16, 2022
Connecting Materialize directly to PostgreSQL via the Replication stream


The PostgreSQL write-ahead log (WAL) was originally created to enable hot standby's and multi-node setups, but it also works great as a source of data for Materialize. This article discusses how and why.
Read More →
Technical Article
Feb 1, 2022
Taming the beast that is a SQL database

In this article, we will talk about one of the ways we approach the testing of the SQL engine of the product at Materialize. We hope to cover other modules and interesting angles in the future.
Read More →
Ecosystem & Integrations
Jan 19, 2022
Introducing: Tailscale + Materialize

Materialize Cloud works with Tailscale, a VPN solution based on the state-of-the-art WireGuard protocol, to help customers connect their Materialize clusters with services on their private networks.
Read More →
Product Update
Dec 20, 2021
What's new in Materialize? Volume 1


Changelog: Kafka source metadata, protobuf+schema registry for Redpanda, Time bucketing with date_bin, Metabase integration, cloud metrics and monitoring, and new availability region.
Read More →
Ecosystem & Integrations
Oct 19, 2021
Redpanda is a source-available, Kafka-compatible streaming data framework that works both as an upstream data source for Materialize and downstream data sink. Read on to learn how to start building with Redpanda and Materialize.
Read More →
Oct 19, 2021

Redpanda + Materialize
Join us to talk about how Redpanda and Materialize fit together, and how they can be used to build powerful streaming architectures previously only accessible to large enterprise teams.
Watch ReplayCompany News
Sep 30, 2021
Materialize raises another round of funding to help build a cloud-native streaming data warehouse.
Read More →
Ecosystem & Integrations
Sep 21, 2021
Change Data Capture is having a moment. Why?

Change Data Capture (CDC) is finally gaining widespread adoption as a architectural primitive. Why now?
Read More →
Technical Article
Jun 2, 2021
Maintaining Joins using Few Resources

Streaming joins must maintain the pre-join datasets in memory, making them potentially costly operations. Materialize uses shared arrangements to allow multiple join statements to share the same pre-join index.
Read More →
Technical Article
Apr 29, 2021
Generalizing linear operators in differential dataflow

Differential dataflow uses simple linear operators: `map`, `filter`, `flat_map` and complex: `explode` and temporal filter operators. But, with some thinking, we can generalize them all to a restricted form of join.
Read More →
Technical Article
Apr 27, 2021
Debezium and Materialize can be used as powerful tools for joining high-volume streams of data from Kafka and tables from databases.
Read More →
Technical Article
Apr 21, 2021
Build a real-time A/B testing stack with Segment, Kinesis and Materialize.
Read More →
Ecosystem & Integrations
Mar 24, 2021
dbt + Materialize demo: Running dbt’s jaffle_shop with Materialize

Let's demonstrate how to manage streaming SQL in Materialize with dbt by porting the classic dbt jaffle-shop demo scenario to the world of streaming.
Read More →
Technical Article
Mar 1, 2021
Subquery optimization is a high-complexity, high-impact task in databases. This post gives a rough map of existing approaches to optimizing subqueries and also describes how Materialize differs from them..
Read More →
Ecosystem & Integrations
Mar 1, 2021
Introducing: dbt + Materialize

dbt is a tool for managing SQL data transformations. Materialize is a streaming database. When used together, analytics works the way it always should have: Define transforms in SQL, get results in real-time.
Read More →
Technical Article
Feb 16, 2021
Temporal Filters: Enabling Windowed Queries in Materialize

Temporal filters give you a powerful SQL primitive for defining time-windowed computations over temporal data.
Read More →
Technical Article
Jan 20, 2021
Let's build a python application to demonstrate how developers can create real-time, event-driven experiences for their users, powered by Materialize.
Read More →
Technical Article
Jan 14, 2021
Slicing up Temporal Aggregates in Materialize

Technical Article
Jan 11, 2021
Life in Differential Dataflow

Let's use Conway's Game of Life to illustrate how to write algorithms in differential dataflow.
Read More →
Technical Article
Dec 14, 2020
Joins in Materialize

Joins in streaming systems are one of the harder things to do both correctly and efficiently. Let's talk about ways that Materialize maintains them, starting with basic binary joins and working our way up to delta joins.
Read More →
Key Concept
Dec 8, 2020
Kafka is not a Database


In principle, it is possible to use Kafka as a database. But in doing so you will confront every hard problem that database management systems have faced for decades.
Read More →
Technical Article
Dec 2, 2020
Materialize can be used to quickly build scalable backends for real-time apps! In this blog post, we describe two apps that you can try out at home that run on actual, live data.
Read More →
Technical Article
Aug 18, 2020
Lateral Joins and Demand-Driven Queries

In today's post we are going to show off Materialize's `LATERAL` join, and how you can use it to implement some pretty neat query patterns in an incremental view maintenance engine!
Read More →
Technical Article
Aug 13, 2020
Change Data Capture (part 1)

Here we set the context for and propose a change data capture protocol: a means of writing down and reading back changes to data.
Read More →
Technical Article
Aug 11, 2020
Why Use a Materialized View?

Querying materialized views, unlike querying tables or logical views, can reduce query costs by maintaining results in memory that are only updated when necessary. Read on to learn more!
Read More →
Technical Article
Aug 6, 2020
Why not RocksDB for streaming storage?

An explanation of our rationale for why Materialize chose not to use RocksDB as its underlying storage engine.
Read More →
Technical Article
Jul 24, 2020
Streaming TAIL to the Browser - A One Day Project

Technical Article
Jul 14, 2020
Eventual Consistency isn't for Streaming

Eventual consistency is common for key-value stores, where the trade-off is well understood and manageable. But in a streaming system, eventual consistency creates unboundedly large and systematic errors.
Read More →
Technical Article
Jun 11, 2020
How do you build a streaming database from scratch? Here is a roadmap: Start with a streaming framework, build a performant single binary, then break it up into a scalable distributed database platform.
Read More →
Technical Article
Jun 8, 2020
CMU DB Talk: Building Materialize

Arjun Narayan introduces the CMU DB group to streaming databases, the problems they solve, and specific architectural decisions in Materialize.
Read More →
Technical Article
May 5, 2020
Managing memory with differential dataflow

Frameworks that process unbounded streams of data need to be diligent about not also using unbounded amounts of memory. This post discusses some of the tricks used by Differential Dataflow to manage and limit memory use.
Read More →
Technical Article
Mar 31, 2020
In-order reliable message delivery is not enough. Showing views over streams of data requires thinking through additional consistency semantics to deliver correct results.
Read More →
Technical
Mar 24, 2020
What’s inside Materialize? An architecture overview


Let's review the internal architecture of Materialize, starting with the some context of how it's different than other databases.
Read More →
Key Concept
Mar 24, 2020
Incremental Computation in the Database

Incremental computation systems are used to make frequent compute-intensive tasks faster and more efficient in compilers, front-ends, and IDEs, but what does it look like at the data layer?
Read More →
Key Concept
Mar 24, 2020
What is a Materialized View?

What is a materialized view, where can you use them, and how are they useful?
Read More →
Technical Article
Mar 18, 2020
Taking Materialize for a spin on NYC taxi data

Key Concept
Feb 26, 2020
What is Streaming SQL?

Get a high-level overview of what Streaming SQL means, why it's useful, how it's being used in the real world, and how you can start using it yourself.
Read More →
Technical Article
Feb 24, 2020
View Maintenance: A New Approach to Data Processing

Company News
Feb 18, 2020
Introducing Materialize: the Streaming Data Warehouse


Despite substantial progress, data still moves too slowly. The solution is a different paradigm: Streaming. Materialize is a streaming data warehouse built on principles of interoperability and consistency at millisecond latency.
Read More →