Blog

Oct 16, 2023
Materialize extension for VS Code
Get a schema explorer, SQL validation, and query execution directly in VS Code so you can develop and maintain your operational work in Materialize without ever leaving your IDE.

Oct 12, 2023
Freshness and Operational Autonomy
At the heart of freshness in Materialize is autonomous proactive work, done in response to the arrival of data rather than waiting for a user command.

Sep 28, 2023
Announcing Webhook Sources
Today Materialize customers can create webhook sources, making it much easier to pipe in events from a long tail of SaaS platforms, services, and tools.

Sep 26, 2023
Consistency and Operational Confidence
Learn about Materialize's consistency guarantees, and how these guarantees are the foundation of confidence in an operational data warehouse. See these guarantees in action, using tests that you can also use to explore the properties of other solutions.

Sep 22, 2023
A guided tour through Materialize's product principles
Take a guided tour through Materialize's three pillars of product value, and see how we think about providing value for your operational workloads.

Sep 12, 2023
Materialize: An Operational Data Warehouse
We've built Materialize as a new kind of data warehouse, optimized to handle operational data work with the same familiar process from analytical warehouses.

Aug 31, 2023
RBAC now available for all customers
Role Based Access Control (RBAC) can now be enabled for any customer environment, giving Materialize users important security controls for production-grade workloads.

Aug 29, 2023
How Materialize can lower the cost of freshness for data teams
Materialize has a subtly different cost model that is a huge advantage for operational workloads that need fresh data.

Aug 1, 2023
Capturing Change Data Capture (CDC) Data
An illustration of the unexpectedly high downstream cost of clever optimizations to change data capture.

Jul 27, 2023
The uses and abuses of Cloud Data Warehouses
Data Warehouses are great for many things but often misused for operational workloads.

Jul 18, 2023
Materialize and Confluent partner to expand the streaming ecosystem
Confluent Cloud customers can now quickly and seamlessly integrate with Materialize via an officially-supported integration, bringing performant and fully-featured SQL on Kafka capabilities within reach of all data teams.

Jul 12, 2023
Incrementally Maintained Recursive SQL Queries in Materialize
Support for recursive SQL queries in Materialize is now available in public preview.

Jun 2, 2023
When and Why to move work from a Data Warehouse to Materialize
A framework for understanding why and when to shift a workload from traditional cloud data warehouses to Materialize.

May 18, 2023
Postgres Source Updates: Unlocking real-time materialized views in PostgreSQL
With major updates to the streaming replication connection to PostgreSQL, users can now set up Materialize as a drop-in enabler of real-time, incrementally updated, materialized views for their PostgreSQL database.

May 11, 2023
When to use Materialize vs a Stream Processor
If you're already familiar with stream processors you may wonder: When is it better to use Materialize vs a Stream Processor? And why?

Apr 25, 2023
A Terraform Provider for Materialize
Materialize maintains an official Terraform Provider you can use to manage your clusters, replicas, connections and secrets as code.

Apr 20, 2023
Everything you need to know to be a Materialize power-user
You could just write SQL, get continually updated results on Materialize. But if you want to get more scale, performance, power, here is a gentle introduction to key internals that will help.

Apr 5, 2023
The Four ACID Questions
Four questions, and their answers, to explain ACID transactions and how they are handled within Materialize.

Mar 9, 2023
Towards Real-Time dbt
Here's a framework for thinking about reducing the time between when raw data is available to transform with dbt, and when it is delivering value to your customers.

Feb 23, 2023
The Software Architecture of Materialize
Materialize aims to be usable by anyone who knows SQL, but for those interested in going deeper and understanding the architecture powering Materialize, this post is for you!

Feb 16, 2023
When to Use Indexes and Materialized Views
If you are familiar with materialized views and indexes from other databases, this article will help you apply that understanding to Materialize.

Jan 31, 2023
Clusters, explained with Data Warehouses
If you're familiar with data warehouses, this article will help you understand Materialize Clusters in relation to well-known components in Snowflake.

Jan 18, 2023
Delta Joins and Late Materialization
Understand how to optimize joins with indexes and late materialization.

Jan 11, 2023
Recursion in Materialize
Differential Dataflow is capable of incrementally updated iterative computation (recursion) but we haven't yet wired it up to SQL. Let's talk about what recursion could look like in Materialize, and why it's important.

Oct 19, 2022
Real-Time Customer Data Platform Views on Materialize
Let's demonstrate the unique features of Materialize by building the core functionality of a customer data platform.

Oct 18, 2022
How and why is Materialize compatible with PostgreSQL?
As an operational data warehouse, Materialize is fundamentally different on the inside, but it's compatible with PostgreSQL in a few important ways.

Oct 3, 2022
Announcing the next generation of Materialize
Today, we’re excited to announce a product that we feel is transformational: a persistent, scalable, cloud-native Materialize.

Jul 27, 2022
Indexes: A Silent Frenemy
Even in traditional databases, indexes can at different times be the problem and the solution when it comes to scaling. In this article we discuss how indexes change in streaming-first data warehouses.

Jul 14, 2022
Real-time data quality tests using dbt and Materialize
In traditional databases, a SQL query used as a test runs as a point-in-time check. In streaming, the same query can run continually as data changes, creating a SQL-based data monitoring primitive.

Jun 15, 2022
Managing streaming analytics pipelines with dbt
Let's explore a hands-on example where we use dbt (data build tool) to manage and document a streaming analytics workflow from a message broker to Metabase.

Jun 14, 2022
Virtual Time: The Secret to Strong Consistency and Scalable Performance in Materialize
The key to Materialize's ability to separate compute from storage and scale horizontally without sacrificing consistency is a concept called virtual time.

Jun 9, 2022
Let’s talk about Data Apps
What is a Data Application? How do they help our customers? What new challenges do we face when building Data Apps? Here's our perspective.

May 13, 2022
Announcing the Materialize Integration with Cube
Connect headless BI tool Cube.js to the read-side of Materialize to get Rest/GraphQL API's, Authentication, metrics modelling, and more out of the box.

May 6, 2022
Materialize's unbundled cloud architecture
The materialized
binary is stable and performant, the time has come to break it apart into separate services to enable the next phase: unbounded scale in a cloud architecture.

Apr 25, 2022
Creating a Real-Time Feature Store with Materialize
Let's use Materialize to deliver a feature store that continuously updates dimensions as new data becomes available without compromising on correctness or speed.

Mar 3, 2022
Subscribe to changes in a view with Materialize
Developers have long wished for the ability to subscribe to changes in a SQL query or a view in a database. Materialize has a SUBSCRIBE primitive that makes it possible.

Mar 1, 2022
What's new in Materialize? Vol. 2
Changelog: AWS roles for S3/Kinesis, PostgreSQL source improvements, Schema Registry SSL, SELECT statements in Tail queries, jsonb subscripting, DBeaver support, & Tailscale in cloud.

Jan 19, 2022
Introducing: Tailscale + Materialize
Materialize Cloud works with Tailscale, a VPN solution based on the state-of-the-art WireGuard protocol, to help customers connect their Materialize clusters with services on their private networks.

Dec 20, 2021
What's new in Materialize? Volume 1
Changelog: Kafka source metadata, protobuf+schema registry for Redpanda, Time bucketing with date_bin, Metabase integration, cloud metrics and monitoring, and new availability region.

Oct 19, 2021
Taking streaming analytics further faster with Redpanda + Materialize
Redpanda is a source-available, Kafka-compatible streaming data framework that works both as an upstream data source for Materialize and downstream data sink. Read on to learn how to start building with Redpanda and Materialize.

Sep 30, 2021
Materialize raises a $60M Series C, bringing total funding to over $100M
Materialize raises another round of funding to help build a cloud-native streaming data warehouse.

Sep 21, 2021
Change Data Capture is having a moment. Why?
Change Data Capture (CDC) is finally gaining widespread adoption as a architectural primitive. Why now?

Sep 13, 2021
Materialize Cloud Enters Open Beta

Aug 27, 2021
Release: 0.9

Aug 5, 2021
Materialize & Datalot: Real-time Application Development

Jun 14, 2021
Release: 0.8

Jun 2, 2021
Maintaining Joins using Few Resources
Streaming joins must maintain the pre-join datasets in memory, making them potentially costly operations. Materialize uses shared arrangements to allow multiple join statements to share the same pre-join index.

Apr 27, 2021
Join Kafka with a Database using Debezium and Materialize
Debezium and Materialize can be used as powerful tools for joining high-volume streams of data from Kafka and tables from databases.

Apr 21, 2021
Real-time A/B test results with Segment, Kinesis, and Materialize
Build a real-time A/B testing stack with Segment, Kinesis and Materialize.

Mar 24, 2021
dbt + Materialize demo: Running dbt’s jaffle_shop with Materialize
Let's demonstrate how to manage streaming SQL in Materialize with dbt by porting the classic dbt jaffle-shop demo scenario to the world of streaming.

Mar 9, 2021
Release: 0.7

Mar 1, 2021
How Materialize and other databases optimize SQL subqueries
Subquery optimization is a high-complexity, high-impact task in databases. This post gives a rough map of existing approaches to optimizing subqueries and also describes how Materialize differs from them..

Mar 1, 2021
Introducing: dbt + Materialize
dbt is a tool for managing SQL data transformations. Materialize is a operational data warehouse. When used together, analytics works the way it always should have: Define transforms in SQL, get results in real-time.

Feb 16, 2021
Temporal Filters: Enabling Windowed Queries in Materialize
Temporal filters give you a powerful SQL primitive for defining time-windowed computations over temporal data.

Jan 20, 2021
A Simple and Efficient Real Time Application Powered by Materialize's TAIL Command
Let's build a python application to demonstrate how developers can create real-time, event-driven experiences for their users, powered by Materialize.

Jan 14, 2021
Slicing up Temporal Aggregates in Materialize

Jan 7, 2021
Release: 0.6

Dec 14, 2020
Joins in Materialize
Joins in streaming systems are one of the harder things to do both correctly and efficiently. Let's talk about ways that Materialize maintains them, starting with basic binary joins and working our way up to delta joins.

Dec 8, 2020
Kafka is not a Database
In principle, it is possible to use Kafka as a database. But in doing so you will confront every hard problem that database management systems have faced for decades.

Dec 2, 2020
Live Maintained Views on Boston Transit to Run at Home
Materialize can be used to quickly build scalable backends for real-time apps! In this blog post, we describe two apps that you can try out at home that run on actual, live data.

Nov 30, 2020
Materialize Raises a Series B

Nov 24, 2020
Release: Materialize 0.5

Sep 30, 2020
Materialize under the Hood

Aug 18, 2020
Lateral Joins and Demand-Driven Queries
In today's post we are going to show off Materialize's LATERAL
join, and how you can use it to implement some pretty neat query patterns in an incremental view maintenance engine!

Aug 13, 2020
Change Data Capture (part 1)
Here we set the context for and propose a change data capture protocol: a means of writing down and reading back changes to data.

Aug 11, 2020
Why Use a Materialized View?
Querying materialized views, unlike querying tables or logical views, can reduce query costs by maintaining results in memory that are only updated when necessary. Read on to learn more!

Aug 6, 2020
Why not RocksDB for streaming storage?
An explanation of our rationale for why Materialize chose not to use RocksDB as its underlying storage engine.

Aug 4, 2020
Robust Reductions in Materialize

Jul 28, 2020
Release: Materialize 0.4

Jul 14, 2020
Eventual Consistency isn't for Streaming
Eventual consistency is common for key-value stores, where the trade-off is well understood and manageable. But in a streaming system, eventual consistency creates unboundedly large and systematic errors.

Jun 11, 2020
Materialize: Roadmap to Building a Streaming Database on Timely Dataflow
How do you build a streaming database from scratch? Here is a roadmap: Start with a streaming framework, build a performant single binary, then break it up into a scalable distributed database platform.

Jun 8, 2020
CMU DB Talk: Building Materialize
Arjun Narayan introduces the CMU DB group to streaming databases, the problems they solve, and specific architectural decisions in Materialize.

Jun 1, 2020
Release: Materialize 0.3

May 5, 2020
Managing memory with differential dataflow
Frameworks that process unbounded streams of data need to be diligent about not also using unbounded amounts of memory. This post discusses some of the tricks used by Differential Dataflow to manage and limit memory use.

Mar 31, 2020
What consistency guarantees should you expect from your streaming data platform?
In-order reliable message delivery is not enough. Showing views over streams of data requires thinking through additional consistency semantics to deliver correct results.

Mar 27, 2020
Upserts in Differential Dataflow

Mar 24, 2020
What’s inside Materialize? An architecture overview
Let's review the internal architecture of Materialize, starting with the some context of how it's different than other databases.

Mar 18, 2020
Taking Materialize for a spin on NYC taxi data

Feb 24, 2020
View Maintenance: A New Approach to Data Processing

Feb 20, 2020
Materialize Beta: The Details

Feb 18, 2020
Introducing Materialize: the Streaming Data Warehouse
Despite substantial progress, data still moves too slowly. The solution is a different paradigm: Streaming. Materialize is a streaming data warehouse built on principles of interoperability and consistency at millisecond latency.