Want to connect with Materialize? Join our growing community on Slack!
ProductMaterialize under the Hood

Today we will take a bit of a tour of the moving parts that make up Materialize. This tour isn’t meant to be exhaustive, but rather to show off some of the moments where things might be different from what you expect, and to give you a sense for why Materialize is relatively better at […]

Deep-diveLateral Joins and Demand-Driven Queries

In today’s post we are going to show off Materialize’s LATERAL join (courtesy @benesch), and how you can use it to implement some pretty neat query patterns in an incremental view maintenance engine! In particular, in the streaming SQL setting, lateral joins automatically turn your SQL prepared statement queries into what is essentially a streaming, […]

StreamingChange Data Capture (part 1)

At Materialize we traffic in computation over data that change. As a consequence, it is important to have a way to write down and read back changes to data. An unambiguous, robust, and performant way to write down and read back changes to data. What makes this challenging? Why not just write out a log […]

Deep-diveRobust Reductions in Materialize

Materialize is an incremental view maintenance engine, one which takes your SQL queries expressed as views and continually maintains them as your data change. Surely there are a lot of ways one could do this, ranging from the very na├»ve (just recompute from scratch) to the more sophisticated end of the spectrum (what we do). […]

Deep-diveEventual Consistency isn’t for Streaming

Streaming systems consume inputs and produce outputs asyncronously: the output of a system at any moment may not reflect all of the inputs seen so far. These systems provide various guarantees about how their outputs relate to their input. Among the weaker (but not unpopular) guarantees is eventual consistency. Informally, eventual consistency means if the […]

Deep-diveRust for Data-Intensive Computation

I have some thoughts on the use of Rust for data-intensive computations. Specifically, I’ve found several of Rust’s key idioms line up very well with the performance and correctness needs of data-intensive computing. If you want a tl;dr for the post: I’ve built multiple high-performance, distributed data processing platforms in Rust, and I never learned […]

Deep-diveManaging memory with differential dataflow

Self-compacting dataflows Those of you familiar with dataflow processing are likely also familiar with the constant attendant anxiety: won’t my constant stream of input data accumulate and eventually overwhelm my system? That’s a great worry! In many cases tools like differential dataflow work hard to maintain a compact representation for your data. At the same […]

Deep-diveUpserts in Differential Dataflow

“Upserts” are a common way to express streams of changing data, especially in relational settings with primary keys. However, they aren’t the best format for working with incremental computation. We’re about to learn why that is, how we deal with this in differential dataflow and Materialize, and what doors this opens up! This post is […]

Deep-diveTaking Materialize for a spin on NYC taxi data

Trying out Materialize This post will also be available at my personal blog. We all at Materialize are working from home, and while this is all a bit weird and different, it gives me some time to write a bit more and try and show off some of what we have been up to. I […]

Deep-diveView Maintenance: A New Approach to Data Processing

It’s with a great deal of excitement, and some trepidation, that we are now able to show off what we have been working on at Materialize. The Materialize product is a “streaming data warehouse”, which is a class of product you might not be familiar with as, to the best of our knowledge, there wasn’t […]

About This Blog

Welcome! On our blog, you’ll hear more about the inner workings of Materialize – what we’ve built, what we plan to build, and how it all works together.

New here? Read these