Materialize + Novu: Real-Time Alerting Powered by a Cloud Operational Data Store

Real-time access to critical data can be the difference between a quick response and a missed opportunity. Materialize, as a cloud operational data store (ODS), operationalizes data to power crucial business processes like real-time alerting, ensuring teams can respond instantly when it matters the most.

In the following blog, we’ll show you how to create real-time alerts using Materialize’s integration with Novu, an open source notifications infrastructure for managing notifications across various platforms.

Real-Time Alerting: Automate Work & Prompt Teams to Act

Real-time alerts can serve a crucial role in business automation, acting as triggers for workflows and other automated processes. These alerts can turn on manufacturing equipment at just the right time, execute stock trades as market conditions shift, or display custom messages to website visitors.

But alerts don’t just automate processes — they also keep your team informed. Sales teams are notified of new leads. Engineers learn about faulty equipment in the field. And customer support reps are instantly alerted to new support tickets.

And importantly, alerts are sent where people already work, such as SMS or Slack. This allows teams to react quickly, making them faster, more efficient, and more productive.

Why Materialize is the Solution for Real-Time Alerting

Unlike traditional databases that rely on executing queries to process data, Materialize continuously updates your views as new data arrives. This incremental view maintenance enables real-time insight into your data, making it ideal for alerting systems. 

With Materialize, you never miss a key metric update because it detects and reacts to every data change in real-time. Best of all, you can define these alerts using the same SQL you already rely on for your business data, keeping things simple and efficient. 

Materialize’s SUBSCRIBE feature powers real-time alerting by pushing notifications whenever your alert view changes. It creates a long-lived Postgres connection to Materialize and sends a push notification to the client each time the results of your view change. 

Knowing what to alert on is only half the battle — you still need to ensure the notification reaches the right people, in the right way, and at the right time. Many Materialize users pair our system with partner apps like Novu. 

Novu, an open-source notification platform, takes care of configuring and delivering alerts across over 100 endpoints, including email, SMS, Slack, so teams can act quickly on critical events.

Step-by-Step Walkthrough: Deploy Real-Time Alerts with Materialize & Novu

The following section is a step-by-step walkthrough on how to set up real-time alerting with Materialize and Novu. 

In this walkthrough, the SUBSCRIBE feature in Materialize will play a prominent role. SUBSCRIBE updates your views in real-time, and when they change, it pushes the data to your downstream systems.  

We will use a Python Docker container that implements a “durable” subscribe to an indexed or materialized view. Every time that view updates, the container will trigger the Novu event API with the configured payload. If the payload passes a certain threshold, the Novu workflow will trigger an instant alert.

Here are some requirements for the walkthrough: 

• A free Novu cloud account with a configured workflow for the desired alert  

• A Materialize account with a view configured for the alert, and adequate rights to subscribe to the view 

• A system that can continually run the Docker container (such as AWS ECS, or Kubernetes)

• Access to the GitHub repo for the Novu Materialize integration

The walkthrough is based on a simple proof of concept use case. We’ll leverage an iteration of our auction house Quickstart use case in the walkthrough. 

Step 1: Build Views with Materialize

As a first step, you will build the views that drive real-time alerting. This guide offers two options: the simple view, or the lateral view. For the purposes of this walkthrough, we’ll implement the lateral view. 

The lateral view uses a lateral join pattern that separates the threshold values into a Materialize table, distinct from the query itself. 

This makes changes to the thresholds of the alert, or creates multiple instances of the alert, without needing to push an update to your dataflow. 

You can find the code for the lateral view below:  

CREATE TABLE 
	materialize.auction.auction_alerts 
	(
		alert_name VARCHAR,  
		price_above INT, 
		item_name VARCHAR
	);

INSERT INTO 
	materialize.auction.auction_alerts 
VALUES 
	('expensive pizza', 90, 'Best Pizza in Town' ), 
	('all art', 0, 'Custom Art');

CREATE VIEW active_alerts AS
	SELECT 
		alert_name,
        id as auction_id,
		item_name,
        amount as price 
	FROM 
	(
		SELECT 
            id,
			item, 
			amount 
		FROM 
			materialize.auction.winning_bids
	) p,
	LATERAL (
		SELECT 
			price_above, 
			item_name, 
			alert_name 
		FROM 
			materialize.auction.auction_alerts a
		WHERE 
			a.item_name = p.item
		AND 
			a.price_above <= p.amount
);

CREATE INDEX 
	active_alerts_idx 
ON 
	active_alerts (alert_name,alert_name)
WITH 
	(RETAIN HISTORY FOR '1hr');

Here’s the active_alerts view in the Materialize platform:

A lateral view shows a slightly more sophisticated pattern that you might use if you have multiple alerts of a similar kind. You can populate your WHERE clause from a LATERAL JOIN to an auction_alerts table. 

Each row of the auction_alerts table will create its own named alert with its own thresholds. You can filter them out by alert name on the Novu side by using step filters if you want to handle each alert differently. 

This example creates two alerts, the first is expensive_pizza, which alerts every time an auction closes for “Best Pizza In Town” above $90. The second is all art, which alerts on all auctions that close for Custom Art as long as the price is above zero.

The important detail here is that we can modify, delete, or add different alerts in real-time without re-deploying to Materialize or operational interruption by modifying the rows of the auction_alerts table.

In this example, MTZ_ALERT_VIEW is materialize.auction.auction_alerts and MTZ_ALERT_PAYLOAD is alert_name,auction_id,item_name,price. On the Novu side, you could have different step conditions based on whether the alert_name in the payload is “expensive pizza” or “all art”.

Step 2: Create Alerting Workflow with Novu

Now you’ll need to set up a Novu workflow to send out your alert. Sign up for an account with Novu Cloud. Navigate to the ‘Workflows’ section. Click ‘Add a workflow’.

Then select ‘Blank workflow’. 

Choose the ‘Workflow trigger’ step.

In the ‘To’ box, you will see your ‘subscriberId’. 

Navigate down to the ‘Payload’ box, enter the following:

This Payload is sent by the Docker container. Now navigate back to the Novu workflow. Click on the ‘+’ sign and select ‘Chat’.

In the chat box, type in the message: “Some expensive pizza just got bought!”

Click ‘Update’. Go back to the Novu workflow and press the “+” button. Choose ‘In-App’. Enter the following into the ‘Editor’ section:

Hey! Someone bought some art over here!
Auction Id: {{auction_id}}
Price: {{price}}

This will look like the following:

Then click ‘Update’. Now you can use the alert in-app. For instance, you can use this as a library that you include in your React app to send alerts. 

Read the documentation on how to add a Novu powered In-App notification center to your React app. 

Step 3: Build the Docker Container

As a next step, you’ll need to build a Docker container to integrate a Materialize subscription with Novu. 

Clone the GitHub repo. The Docker container can be built locally from the Python folder using your command line shell: 

docker build -t novu-materialize-integration .

Now that the Docker container is built, you’re ready to configure and run it. 

Step 4: Configure + Run the Docker Container

Configure the container using environment variables. They can be set using the .env file in the repo. You’ll need to configure the following variables:

Optionally, you can also configure these variables:

Open the .env file and fill in the variables. Place the .env file in the Python folder and run the following:

docker run --env-file env.simple novu-materialize-integration

With that, your configured container is now running.

Step 5: Receive Real-Time Alerts

Now, every time the view updates, you will receive an instant Slack alert if item = ‘Best Pizza in Town’ and amount is greater than 90. See the example Slack image below:

You will also receive an in-app alert from Novu in your React app whenever an art auction closes, as seen below:

Materialize + Novu: Powering Alerting Across Your Systems

The example in this blog is a simple proof of concept, but it demonstrates the power of the Materialize and Novu integration. By combining the SUBSCRIBE functionality in Materialize with Novu’s 100 messaging endpoints, you can alert teams and initiate workflows as soon as certain thresholds are met. 

Now you can deliver instant alerting to any team in many different systems, including email, SMS, Slack, in-app messages. You can also automate trigger-based workflows to power critical business processes. We’ve already seen customers use the integration for a number of use cases, including to monitor financial transactions, activate manufacturing processes, and alert sales teams to new leads.

Get started for free with Materialize and sign up for Novu to build an instant alerting system that powers your business.