Our Blog

Change Data Capture is having a moment. Why?
Ecosystem

Change Data Capture is having a moment. Why?

Andy Hattemer

Change Data Capture (CDC) concepts have been around for 20+ years, but recently we’ve seen a step-change increase in discussion, companies, and tool...

Materialize Cloud Enters Open Beta
Product

Materialize Cloud Enters Open Beta

Sin Jae (Sin) Kim

Since the very beginning the Materialize team has been focused on delivering a new approach to real-time data processing building on Timely Dataflow. ...

Release: 0.9
Release

Release: 0.9

Amanda Aguilera

We’re excited to announce the release of Materialize v0.9! This version of Materialize has been in development for nearly two months, and is focused...

Materialize & Datalot: Real-time Application Development
Case Study

Materialize & Datalot: Real-time Application Development

Andy Hattemer

About Datalot Founded in 2009, Datalot provides digital marketing and analytics solutions for insurance policy sales at scale. Their SaaS product prov...

Release: 0.8
Release

Release: 0.8

Amanda Aguilera

Materialize v0.8 was released on 9 June 2021 with significant improvements and features, including temporal filters, upserts, PubNub sources, S3 sourc...

Maintaining Joins using Few Resources
Deep-dive

Maintaining Joins using Few Resources

Frank McSherry

Today’s post is on a topic that a lot of folks have asked for, once they dive a bit into Materialize. One of our join implementation strategies ...

Generalizing linear operators in differential dataflow
Deep-dive

Generalizing linear operators in differential dataflow

Frank McSherry

Differential dataflows contain many operators, some of which are very complicated, but many of which are relatively simple. The map operator applies a...

Join Kafka with a Database using Debezium and Materialize
Deep-dive

Join Kafka with a Database using Debezium and Materialize

Andy Hattemer

The Problem We need to provide (internal or end-user) access to a view of data that combines a fast-changing stream of events from Kafka with a table ...

Real-time A/B test results with Segment, Kinesis, and Materialize
Use Case

Real-time A/B test results with Segment, Kinesis, and Materialize

Andy Hattemer

Introduction This is meant primarily to demonstrate how the Segment + Kinesis + Materialize stack can create new capabilities around querying, joining...

dbt + Materialize demo: Running dbt’s jaffle_shop with Materialize
Ecosystem

dbt + Materialize demo: Running dbt’s jaffle_shop with Materialize

Jessica Laughlin

Introduction We’ve recently partnered with dbt and community members to release a beta Materialize dbt adapter that allows you to transform your...

Release: 0.7
Release

Release: 0.7

Andy Hattemer

Materialize 0.7 was released on 08 February 2021 with significant improvements around getting data into Materialize. Key change: Source data from Amaz...

Introducing: dbt + Materialize
Ecosystem

Introducing: dbt + Materialize

Jessica Laughlin

Managing data is hard. Managing data pipelines is even harder. The meaning of individual tables or values in your data warehouse gets lost in translat...

How Materialize and other databases optimize SQL subqueries
Deep-dive

How Materialize and other databases optimize SQL subqueries

Jamie Brandon

Subqueries are a SQL feature that allow writing queries nested inside a scalar expression in an outer query. Using subqueries is often the most natura...

Streaming SQL: What is it, why is it useful?
Streaming

Streaming SQL: What is it, why is it useful?

Andy Hattemer

Summary Streaming SQL is about taking the same declarative SQL used to write database queries, and instead running it on streams of fast-changing data...

Temporal Filters: Enabling Windowed Queries in Materialize
Deep-dive

Temporal Filters: Enabling Windowed Queries in Materialize

Frank McSherry

Materialize provides a SQL interface to work with continually changing data. You write SQL queries as if against static data, and then as your data ch...

A Simple and Efficient Real Time Application Powered by Materialize’s TAIL Command
Deep-dive

A Simple and Efficient Real Time Application Powered by Materialize’s TAIL Command

Chris Golden

Within the web development community, there has been a clear shift towards frameworks that implement incremental view maintenance and for good reason....

Slicing up Temporal Aggregates in Materialize
Deep-dive

Slicing up Temporal Aggregates in Materialize

Frank McSherry

Materialize computes and maintains SQL queries as your underlying data change. This makes it especially well-suited to tracking the current state of v...

Life in Differential Dataflow
Deep-dive

Life in Differential Dataflow

Ruchir Khaitan

I’ve been working at Materialize for almost a year now, and I have really enjoyed learning about and using Differential Dataflow (hereafter just...

Release: 0.6
News

Release: 0.6

Albert Wang

Materialize 0.6 makes it easier to consume streams and build streaming applications. We’ve also made a number of changes that improve our SQL compat...

Joins in Materialize
Deep-dive

Joins in Materialize

Frank McSherry

This post is also available at my personal blog. Materialize allows you to maintain declarative, relational SQL queries over continually changing data...

Kafka is not a Database
Streaming

Kafka is not a Database

Arjun Narayan

This post is co-authored by George Fraser, the CEO of Fivetran, and Arjun Narayan, the CEO of Materialize. This blog post is cross-posted on the Fivet...

Live Maintained Views on Boston Transit to Run at Home
Use Case

Live Maintained Views on Boston Transit to Run at Home

Andi Wang

Materialize can be used to quickly build scalable backends for real-time apps! In this blog post, we describe two apps that you can try out at home th...

Materialize Raises a Series B
News

Materialize Raises a Series B

Arjun Narayan

Today we announced that we raised a $32M Series B round of funding led by Kleiner Perkins. This follows a $8.5m Series A last year led by Lightspeed V...

Release: Materialize 0.5
News

Release: Materialize 0.5

Albert Wang

We recently released Materialize 0.5!  Here’s what’s new and improved. What’s changed in Materialize 0.5 Version 0.5 includes a number of impro...

Materialize under the Hood
Product

Materialize under the Hood

Frank McSherry

Today we will take a bit of a tour of the moving parts that make up Materialize. This tour isn’t meant to be exhaustive, but rather to show off ...

Lateral Joins and Demand-Driven Queries
Deep-dive

Lateral Joins and Demand-Driven Queries

Frank McSherry

In today’s post we are going to show off Materialize’s LATERAL join (courtesy @benesch), and how you can use it to implement some pretty n...

Change Data Capture (part 1)
Streaming

Change Data Capture (part 1)

Frank McSherry

At Materialize we traffic in computation over data that change. As a consequence, it is important to have a way to write down and read back changes to...

Why Use a Materialized View?
Use Case

Why Use a Materialized View?

Jessica Laughlin

TL;DR: Querying materialized views, unlike querying tables or logical views, can reduce query costs by maintaining results in memory that are only upd...

Why not RocksDB for streaming storage?
Deep-dive

Why not RocksDB for streaming storage?

Arjun Narayan

A roadmap for a storage engine for Materialize...

Robust Reductions in Materialize
Deep-dive

Robust Reductions in Materialize

Frank McSherry

Materialize is an incremental view maintenance engine, one which takes your SQL queries expressed as views and continually maintains them as your data...

Release: Materialize 0.4
News

Release: Materialize 0.4

Albert Wang

We’re proud to announce that we’ve just released Materialize version 0.4. Here is a quick overview of the main features. What’s changed in M...

Streaming TAIL to the Browser – A One Day Project
Deep-dive

Streaming TAIL to the Browser – A One Day Project

Chris Golden

Last week concluded up my first week at Materialize, with Friday being my first Skunkworks Friday. Skunkworks Friday is a Materialize sponsored day of...

Eventual Consistency isn’t for Streaming
Deep-dive

Eventual Consistency isn’t for Streaming

Frank McSherry

Streaming systems consume inputs and produce outputs asyncronously: the output of a system at any moment may not reflect all of the inputs seen so far...

Rust for Data-Intensive Computation
Deep-dive

Rust for Data-Intensive Computation

Frank McSherry

I have some thoughts on the use of Rust for data-intensive computations. Specifically, I’ve found several of Rust’s key idioms line up ver...

Materialize: Roadmap to Building a Streaming Database on Timely Dataflow
Deep-dive

Materialize: Roadmap to Building a Streaming Database on Timely Dataflow

Arjun Narayan

How do you build a streaming database from scratch?...

CMU DB Talk: Building Materialize
Deep-dive

CMU DB Talk: Building Materialize

Arjun Narayan

This is an edited transcript and video of a talk that I gave at Carnegie Mellon’s Database Group Seminar on June 1st, 2020, hosted by Andy Pavlo. Yo...

Release: Materialize 0.3
News

Release: Materialize 0.3

Albert Wang

We recently announced Materialize, a real-time streaming SQL database that powers production applications. The latest release of Materialize, version ...

Managing memory with differential dataflow
Deep-dive

Managing memory with differential dataflow

Frank McSherry

Self-compacting dataflows Those of you familiar with dataflow processing are likely also familiar with the constant attendant anxiety: won’t my ...

What consistency guarantees should you expect from your streaming data platform?
Deep-dive

What consistency guarantees should you expect from your streaming data platform?

Natacha Crooks

In-order reliable message delivery is not enough. Showing views over streams of data requires thinking through additional consistency semantics to del...

Upserts in Differential Dataflow
Deep-dive

Upserts in Differential Dataflow

Frank McSherry

“Upserts” are a common way to express streams of changing data, especially in relational settings with primary keys. However, they aren...

What’s inside Materialize? An architecture overview
Deep-dive

What’s inside Materialize? An architecture overview

Sean Loiselle Arjun Narayan

Materialize lets you ask questions about your streaming data and get fresh answers with incredibly low latency. Since announcing Materialize’s v0.1,...

Taking Materialize for a spin on NYC taxi data
Deep-dive

Taking Materialize for a spin on NYC taxi data

Frank McSherry

Trying out Materialize This post will also be available at my personal blog. We all at Materialize are working from home, and while this is all a bit ...

View Maintenance: A New Approach to Data Processing
Deep-dive

View Maintenance: A New Approach to Data Processing

Frank McSherry

It’s with a great deal of excitement, and some trepidation, that we are now able to show off what we have been working on at Materialize. The Ma...

Materialize Beta: The Details
News

Materialize Beta: The Details

Cuong Do

We recently announced Materialize, the Streaming Data Warehouse. Our first beta release of Materialize, version 0.1, is available now. As a streaming ...

Introducing Materialize: the Streaming Data Warehouse
News

Introducing Materialize: the Streaming Data Warehouse

Arjun Narayan Frank McSherry

Databases, and data infrastructure generally, have made substantial progress over the years. We now have access to cloud-native infrastructure that al...

Categories

Follow Along

Get Materialize updates in your inbox:

Blog Newsletter

This field is for validation purposes and should be left unchanged.