Home  /  Manage Materialize  /  Monitoring and alerting  /  Self-Managed

Grafana using Prometheus

View as Markdown
WARNING! The metrics scraped are unstable and may change across releases.

This guide walks you through the steps required to monitor the performance and overall health of your Materialize instance using Prometheus and Grafana.

! Important: Every monitoring setup is unique, so be sure to adapt the steps in this guide to your specific environment and requirements.

Before you begin

Ensure you have:

  • A self-managed instance of Materialize installed with helm values observability.enabled=true, observability.podMetrics.enabled=true, and prometheus.scrapeAnnotations.enabled=true
  • Helm version 3.2.0+ installed
  • kubectl installed and configured
  • metrics-server installed in your cluster
  • kube-state-metrics installed in your cluster
! Important: This guide assumes you have administrative access to your Kubernetes cluster and the necessary permissions to install Prometheus.

1. Download our Prometheus scrape configurations (prometheus.yml)

Download the Prometheus scrape configurations that we’ll use to configure Prometheus to collect metrics from Materialize:

curl -o prometheus_scrape_configs.yml https://materializeinc.github.io/materialize-monitoring/prometheus-scrapers/classic/scrape_config.yaml
NOTE: If you are using prometheus-operator or do not know your prometheus distribution, visit materialize-monitoring/Scraping for additional scraping configurations.

2. Install Prometheus to your Kubernetes cluster

NOTE:

This guide uses the prometheus-community Helm chart to install Prometheus.

Many more production-ready Prometheus Helm charts are available which have different ways to configure Prometheus and its scrape configurations. Be sure to consult the documentation of the Prometheus Helm chart you are using to ensure that you are configuring Prometheus correctly.

  1. Download the prometheus-community default chart values (values.yaml):

    curl -O https://raw.githubusercontent.com/prometheus-community/helm-charts/refs/heads/main/charts/prometheus/values.yaml

  2. Within values.yaml, replace serverFiles > prometheus.yml > scrape_configs with our scrape configurations (prometheus_scrape_configs.yml).

  3. Install the helm chart with the updated values.yaml:

    kubectl create namespace prometheus
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install --namespace prometheus prometheus prometheus-community/prometheus \
--values values.yaml

3. Optional. Visualize through Grafana

  1. Install the Grafana helm chart following this guide.

  2. Set up port forwarding to access the Grafana UI:

    kubectl port-forward svc/grafana 3000:80 -n monitoring
    WARNING! The port forwarding method is for testing purposes only. For production environments, configure an ingress controller to securely expose the Grafana UI.

  3. Open the Grafana UI on http://localhost:3000 in a browser.

  4. Add a Prometheus data source. In the Grafana UI, under Connection > Data sources,

    • Click Add data source and select prometheus
    • In the Connection section, set Prometheus server URL to http://<prometheus server name>.<namespace>.svc:<port>(e.g. http://prometheus-server.prometheus.svc:80).

    Image of Grafana datasource setup

  5. Download dashboards from our Grafana dashboards repository

  6. Import the dashboards using the Prometheus data source

    Image of Grafana

Back to top ↑