Understand the lifecycle of a source

A source and the tables created from it move through a sequence of states before they continuously serve up-to-date data. Knowing which state an object is in tells you whether it is making progress or is stuck.

This page covers sources created with the CREATE SOURCE and CREATE TABLE ... FROM SOURCE syntax. All source types report through the same mz_source_statuses view and move through the same states, so the queries below apply whether you ingest from Kafka, PostgreSQL, MySQL, SQL Server, or a load generator.

The source and each of its tables report their own status. The source tracks the connection to the upstream system, while each table tracks the ingestion of one upstream relation.

States

stalled covers both errors that Materialize retries on its own and errors that do not clear until you act. See Stalled.

The examples below use a PostgreSQL source and a Kafka source on a dedicated cluster. Substitute your own object names.

CREATE CLUSTER ingest_demo SIZE '25cc';

CREATE SOURCE pg_src IN CLUSTER ingest_demo
  FROM POSTGRES CONNECTION pg_conn (PUBLICATION 'mz_orders');

CREATE SOURCE kafka_src IN CLUSTER ingest_demo
  FROM KAFKA CONNECTION kafka_conn (TOPIC 'clicks');

Created

A source that has no tables yet reports created. At this point Materialize has recorded the source and its upstream connection, but it is not ingesting anything and consumes no cluster resources:

SELECT name, type, status
FROM mz_internal.mz_source_statuses
ORDER BY name;

   name    |   type   | status
-----------+----------+---------
 kafka_src | kafka    | created
 pg_src    | postgres | created
(2 rows)

A source stays in created for as long as it has no tables. Ingestion begins only when you attach a table with CREATE TABLE ... FROM SOURCE:

CREATE TABLE orders FROM SOURCE pg_src (REFERENCE orders);
CREATE TABLE clicks FROM SOURCE kafka_src (REFERENCE clicks) FORMAT JSON;

Starting and running

Once a table is attached, the source and the table connect to the upstream system (starting), then begin ingesting (running):

SELECT o.name, s.status, s.error
FROM mz_internal.mz_source_statuses s
JOIN mz_objects o ON o.id = s.id
ORDER BY o.name;

   name    | status  | error
-----------+---------+-------
 clicks    | running |
 kafka_src | running |
 orders    | running |
 pg_src    | running |
(4 rows)

starting is usually brief. If an object stays in starting for more than a few minutes, see Troubleshooting: Why isn’t my source ingesting data?.

Snapshotting

running covers both the initial snapshot and steady-state ingestion, so the status alone does not tell you whether the initial snapshot is still in progress. Use mz_source_statistics instead:

SELECT o.name, s.snapshot_records_known, s.snapshot_records_staged,
       s.snapshot_committed
FROM mz_internal.mz_source_statistics s
JOIN mz_objects o ON o.id = s.id
WHERE o.name IN ('orders', 'clicks')
ORDER BY o.name;

  name  | snapshot_records_known | snapshot_records_staged | snapshot_committed
--------+------------------------+-------------------------+--------------------
 clicks | 200                    | 200                     | t
 orders | 501                    | 501                     | t
(2 rows)

While the snapshot is in progress, snapshot_records_staged climbs toward snapshot_records_known and snapshot_committed is f. A table cannot serve queries until its snapshot completes: queries against it block until then.

Nothing is committed until the whole snapshot has been read, because Materialize ingests it at a single timestamp. So offset_committed does not advance for the duration, and on upsert sources even updates_staged sits at 0, because the source buffers the snapshot while it builds its in-memory index. In those statistics a snapshot that is progressing normally is indistinguishable from a stuck one, so read progress from snapshot_records_staged, messages_received, and bytes_received.

mz_hydration_statuses answers the same question one level up, per source and replica rather than per table:

SELECT o.name, h.hydrated
FROM mz_internal.mz_hydration_statuses h
JOIN mz_objects o ON o.id = h.object_id
WHERE o.name IN ('pg_src', 'kafka_src')
ORDER BY o.name;

   name    | hydrated
-----------+----------
 kafka_src | t
 pg_src    | t
(2 rows)

hydrated is false while any table attached to the source is still snapshotting, whatever the source type, which makes it a cheap source-level check. It is not specific to the initial snapshot: a replica restart resets it too, after which it tracks the source rebuilding its in-memory state.

Snapshot duration and upstream impact vary by source type. CDC sources (PostgreSQL, MySQL, SQL Server) require the upstream system to retain its change log until the snapshot completes, so a long snapshot increases upstream disk usage. Kafka sources have no equivalent retention requirement. See Snapshotting and Monitoring the snapshotting progress.

Steady state

Once the snapshot is committed, the object continually ingests upstream changes and status stays running. To confirm it is keeping up, compare the offset Materialize has committed against the offset it knows about upstream:

SELECT o.name, s.offset_known, s.offset_committed,
       s.offset_known - s.offset_committed AS offset_delta
FROM mz_internal.mz_source_statistics s
JOIN mz_objects o ON o.id = s.id
WHERE o.name IN ('orders', 'clicks')
ORDER BY o.name;

  name  | offset_known | offset_committed | offset_delta
--------+--------------+------------------+--------------
 clicks | 200          | 200              | 0
 orders | 22526904     | 22526904         | 0
(2 rows)

You want offset_delta close to 0. The unit depends on the source type: for Kafka sources an offset is a Kafka offset, while for PostgreSQL sources it is a log sequence number (LSN), which is why the two rows above differ by orders of magnitude. Compare each object against itself over time rather than against other objects. See Monitoring data lag.

Paused

An object whose cluster has no replicas reports paused and makes no progress. The details column reports why:

SELECT o.name, s.status, s.details
FROM mz_internal.mz_source_statuses s
JOIN mz_objects o ON o.id = s.id
ORDER BY o.name;

   name    | status |                             details
-----------+--------+-----------------------------------------------------------------
 clicks    | paused | {"hints":["There is currently no replica running this source"]}
 kafka_src | paused | {"hints":["There is currently no replica running this source"]}
 orders    | paused | {"hints":["There is currently no replica running this source"]}
 pg_src    | paused | {"hints":["There is currently no replica running this source"]}
(4 rows)

A cluster that had a replica and lost it reports a different hint, The replica running this source has been dropped. Either way, ingestion resumes when the cluster has a replica again, so increase the replication factor of the cluster hosting the source.

Stalled

An object that hits an error reports stalled, with the cause in error:

SELECT o.name, s.status, s.error
FROM mz_internal.mz_source_statuses s
JOIN mz_objects o ON o.id = s.id
ORDER BY o.name;

In the output below, the publication backing the PostgreSQL source was dropped upstream. Both pg_src and its table orders stall, because neither can make progress without it, while the unrelated Kafka source keeps running:

   name    | status  |                      error
-----------+---------+--------------------------------------------------
 clicks    | running |
 kafka_src | running |
 orders    | stalled | postgres: publication "mz_orders" does not exist
 pg_src    | stalled | postgres: publication "mz_orders" does not exist
(4 rows)

Sources stall independently of one another, so a stall is scoped to the source that hit the error and the tables that depend on it.

The details column carries the same error tagged with the subsystem that reported it, which tells you which part of the pipeline failed:

{"namespaced":{"postgres":"publication \"mz_orders\" does not exist"}}

Which stalls clear on their own

Errors unrelated to the ingested data, such as a connection failure, an authentication failure, or an upstream restart, are retried. Materialize restarts the ingestion dataflow when it needs to, and the object moves back through starting to running once the upstream problem clears. Repeated stalled and starting transitions in mz_source_status_history are the signature of an error being retried.

Other errors are definite: the upstream system changed in a way that invalidates what Materialize has already ingested. Dropping the publication a PostgreSQL source replicates from, dropping or truncating an upstream table, invalidating a replication slot, and an incompatible upstream schema change all land here. Materialize records a definite error durably against the affected table, so reads of that table return it and restarting the dataflow does not clear it. Recovery means fixing the upstream cause and recreating the affected tables, as in Absorbing upstream schema changes. The stall above is one of these, so orders cannot be repaired in place.

For causes and fixes, see Troubleshooting data ingestion for any source type, and the CDC-specific guides for PostgreSQL and MySQL, which cover replication slot, WAL, and GTID errors unique to those connectors.

Dropped

Dropping an object is terminal. Once dropped, it no longer appears in mz_source_statuses, but its final dropped status remains in mz_source_status_history.

Reviewing the full history

mz_source_statuses reports only the current state. To see every transition an object has gone through, which is the fastest way to understand how it reached its current state, query the history:

SELECT o.name, h.occurred_at, h.status
FROM mz_internal.mz_source_status_history h
JOIN mz_objects o ON o.id = h.source_id
WHERE o.name IN ('orders', 'clicks')
ORDER BY h.occurred_at;

  name  |        occurred_at         |  status
--------+----------------------------+----------
 orders | 2026-09-11 15:04:24.384+00 | starting
 orders | 2026-09-11 15:04:24.385+00 | running
 clicks | 2026-09-11 15:04:24.515+00 | starting
 clicks | 2026-09-11 15:04:24.56+00  | running
(4 rows)

created is not a recorded transition, so it never appears in the history. Here both tables moved from starting to running within milliseconds, since each had only a few hundred rows to snapshot.

Related pages

• Sources
• Snapshotting
• Monitoring data ingestion
• Troubleshooting data ingestion