Engineering Blog

Recently, I've felt the pain of long Rust compile times at Materialize, and so was motived to improve them a bit. Here's how I did it.

Let's build (in Python) the Differential Dataflow framework at the heart of Materialize, and explain what it's doing along the way.

Materialize is written in Rust. Why did we make that decision and how has it turned out for the project?

The PostgreSQL write-ahead log (WAL) was originally created to enable hot standby's and multi-node setups, but it also works great as a source of data for Materialize. This article discusses how and why.

In this article, we will talk about one of the ways we approach the testing of the SQL engine of the product at Materialize. We hope to cover other modules and interesting angles in the future.

Differential dataflow uses simple linear operators: map, filter, flat_map and complex: explode and temporal filter operators. But, with some thinking, we can generalize them all to a restricted form of join.

Let's use Conway's Game of Life to illustrate how to write algorithms in differential dataflow.