mz_internal

The following sections describe the available objects in the mz_internal schema.

WARNING! The objects in the mz_internal schema are not part of Materialize’s stable interface. Backwards-incompatible changes to these tables may be made at any time.
WARNING! SELECT statements may reference these objects, but creating views that reference these objects is not allowed.

System Relations

mz_cluster_replica_metrics

The mz_cluster_replica_metrics table gives the last known CPU and RAM utilization statistics for all processes of all extant cluster replicas.

At this time, we do not make any guarantees about the exactness or freshness of these numbers.

Field Type Meaning
replica_id text The ID of a cluster replica.
process_id uint8 An identifier of a compute process within a replica.
cpu_nano_cores uint8 Approximate CPU usage, in billionths of a vCPU core.
memory_bytes uint8 Approximate RAM usage, in bytes.
disk_bytes uint8 Approximate disk usage in bytes, if the replica has a disk attached. NULL otherwise.

mz_cluster_replica_sizes

The mz_cluster_replica_sizes table contains a mapping of logical sizes (e.g. “xlarge”) to physical sizes (number of processes, and CPU and memory allocations per process).

WARNING! The values in this table may change at any time. You should not rely on them for any kind of capacity planning.
Field Type Meaning
size text The human-readable replica size.
processes uint8 The number of processes in the replica.
workers uint8 The number of Timely Dataflow workers per process.
cpu_nano_cores uint8 The CPU allocation per process, in billionths of a vCPU core.
memory_bytes uint8 The RAM allocation per process, in billionths of a vCPU core.
disk_bytes uint8 The disk allocation per process, if the replica has a disk attached. NULL otherwise.
credits_per_hour numeric The number of compute credits consumed per hour.

The mz_cluster_links table contains a row for each cluster that is linked to a source or sink. When present, the lifetime of the specified cluster is tied to the lifetime of the specified source or sink: the cluster cannot be dropped without dropping the linked source or sink, and dropping the linked source or sink will also drop the cluster. There is at most one row per cluster.

NOTE: The concept of a linked cluster is not user-facing, and is intentionally undocumented. Linked clusters are meant to preserve the soon-to-be legacy interface for sizing sources and sinks.
Field Type Meaning
cluster_id text The ID of the cluster. Corresponds to mz_clusters.id.
object_id text The ID of the source or sink. Corresponds to mz_objects.id.

mz_cluster_replica_statuses

The mz_cluster_replica_statuses table contains a row describing the status of each process in each cluster replica in the system.

Field Type Meaning
replica_id text Materialize’s unique ID for the cluster replica.
process_id uint8 The ID of the process within the cluster replica.
status text The status of the cluster replica: ready or not-ready.
reason text If the cluster replica is in a not-ready state, the reason (if available). For example, oom-killed.
updated_at timestamp with time zone The time at which the status was last updated.

mz_cluster_replica_utilization

The mz_cluster_replica_utilization view gives the last known CPU and RAM utilization statistics for all processes of all extant cluster replicas, as a percentage of the total resource allocation.

At this time, we do not make any guarantees about the exactness or freshness of these numbers.

Field Type Meaning
replica_id text The ID of a cluster replica.
process_id uint8 An identifier of a compute process within a replica.
cpu_percent double precision Approximate CPU usage in percent of the total allocation.
memory_percent double precision Approximate RAM usage in percent of the total allocation.
disk_percent double precision Approximate disk usage in percent of the total allocation, if the replica has a disk attached. NULL otherwise.

mz_cluster_replica_heartbeats

The mz_cluster_replica_heartbeats table gives the last known heartbeat of all extant cluster replicas.

Field Type Meaning
replica_id text The ID of a cluster replica.
last_heartbeat timestamp with time zone The time of the replica’s last heartbeat.

mz_cluster_replica_history

The mz_cluster_replica_history view contains information about the timespan of each replica, including the times at which it was created and dropped (if applicable).

Field Type Meaning
replica_id text The ID of a cluster replica.
size text The size of the cluster replica. Corresponds to mz_cluster_replica_sizes.size.
cluster_name text The name of the cluster associated with the replica.
replica_name text The name of the replica.
created_at timestamp with time zone The time at which the replica was created.
dropped_at timestamp with time zone The time at which the replica was dropped, or NULL if it still exists.
credits_per_hour numeric The number of compute credits consumed per hour. Corresponds to mz_cluster_replica_sizes.credits_per_hour.

mz_internal_cluster_replicas

The mz_internal_cluster_replicas table lists the replicas that are created and maintained by Materialize support.

Field Type Meaning
id text The ID of a cluster replica. Corresponds to mz_cluster_replicas.id.

mz_comments

The mz_comments table stores optional comments (descriptions) for objects in the database.

Field Type Meaning
id text The ID of the object. Corresponds to mz_objects.id.
object_type text The type of object the comment is associated with.
object_sub_id uint8 For a comment on a column of a relation, this is the column number. For all other object types this column is NULL.
comment text The comment itself.

mz_compute_dependencies

The mz_compute_dependencies table describes the dependency structure between each compute object (index, materialized view, or subscription) and the sources of its data.

In contrast to mz_object_dependencies, this table only lists dependencies in the compute layer. SQL objects that don’t exist in the compute layer (such as views) are omitted.

Field Type Meaning
object_id text The ID of a compute object. Corresponds to mz_catalog.mz_indexes.id, mz_catalog.mz_materialized_views.id, or mz_internal.mz_subscriptions.
dependency_id text The ID of a compute dependency. Corresponds to mz_catalog.mz_indexes.id, mz_catalog.mz_materialized_views.id, mz_catalog.mz_sources.id, or mz_catalog.mz_tables.id.

mz_frontiers

The mz_frontiers table describes the frontiers of each source, sink, table, materialized view, index, and subscription in the system, as observed from the coordinator.

For objects that are installed on replicas (e.g., materialized views and indexes), the replica_id field is always non-NULL. If an object is installed on multiple replicas, it has multiple entries describing the frontier on each individual replica. For objects that are not installed on replicas (e.g., tables), the replica_id field is NULL.

mz_compute_frontiers is similar to mz_frontiers, but mz_compute_frontiers reports the frontiers known to the active compute replica, while mz_frontiers reports the frontiers of all replicas. Note also that mz_compute_frontiers is restricted to compute objects (indexes, materialized views, and subscriptions) while mz_frontiers contains storage objects (sources, sinks, and tables) as well.

At this time, we do not make any guarantees about the freshness of these numbers.

Field Type Meaning
object_id text The ID of the source, sink, table, index, materialized view, or subscription.
replica_id text The ID of a cluster replica, or NULL if the object is not installed on a replica.
time mz_timestamp The next timestamp at which the output may change.

mz_global_frontiers

The mz_global_frontiers view describes the global frontiers of each source, sink, table, materialized view, index, and subscription in the system, as observed from the coordinator.

For objects that are installed on replicas (e.g., materialized views and indexes), the global frontier is the maximum of the per-replica frontiers. Objects that are not installed on replicas only have a single, global frontier.

At this time, we do not make any guarantees about the freshness of these numbers.

Field Type Meaning
object_id text The ID of the source, sink, table, index, materialized view, or subscription.
time mz_timestamp The next timestamp at which the output may change.

mz_kafka_sources

The mz_kafka_sources table contains a row for each Kafka source in the system.

Field Type Meaning
id text The ID of the Kafka source. Corresponds to mz_catalog.mz_sources.id.
group_id_base text The prefix of the group ID that Materialize will use when consuming data for the Kafka source.

mz_object_dependencies

The mz_object_dependencies table describes the dependency structure between all database objects in the system.

Field Type Meaning
object_id text The ID of the dependent object. Corresponds to mz_objects.id.
referenced_object_id text The ID of the referenced object. Corresponds to mz_objects.id.

mz_object_fully_qualified_names

The mz_object_fully_qualified_names view enriches the mz_catalog.mz_objects view with namespace information.

Field Type Meaning
id text Materialize’s unique ID for the object.
name text The name of the object.
object_type text The type of the object: one of table, source, view, materialized view, sink, index, connection, secret, type, or function.
schema_name text The name of the schema to which the object belongs. Corresponds to mz_schemas.name.
database_name text The name of the database to which the object belongs. Corresponds to mz_databases.name.

mz_object_lifetimes

The mz_object_lifetimes view enriches the mz_catalog.mz_objects view with information about the last lifetime event that occurred for each object in the system.

Field Type Meaning
id text Materialize’s unique ID for the object.
object_type text The type of the object: one of table, source, view, materialized view, sink, index, connection, secret, type, or function.
event_type text The lifetime event, either create or drop.
occurred_at timestamp with time zone Wall-clock timestamp of when the event occurred.

mz_object_transitive_dependencies

The mz_object_transitive_dependencies view describes the transitive dependency structure between all database objects in the system. The view is defined as the transitive closure of mz_object_dependencies.

Field Type Meaning
object_id text The ID of the dependent object. Corresponds to mz_objects.id.
referenced_object_id text The ID of the (possibly transitively) referenced object. Corresponds to mz_objects.id.

mz_postgres_sources

The mz_postgres_sources table contains a row for each PostgreSQL source in the system.

Field Type Meaning
id text The ID of the source. Corresponds to mz_catalog.mz_sources.id.
replication_slot text The name of the replication slot in the PostgreSQL database that Materialize will create and stream data from.

mz_sessions

The mz_sessions table contains a row for each active session in the system.

Field Type Meaning
id uint4 The ID of the session.
role_id text The role ID of the role that the session is logged in as. Corresponds to mz_catalog.mz_roles.
connected_at timestamp with time zone The time at which the session connected to the system.

mz_show_all_privileges

The mz_show_all_privileges view contains a row for each privilege granted in the system on user objects to user roles.

Field Type Meaning
grantor text The role that granted the privilege.
grantee text The role that the privilege was granted to.
database text The name of the database containing the object.
schema text The name of the schema containing the object.
name text The name of the privilege target.
object_type text The type of object the privilege is granted on.
privilege_type text They type of privilege granted.

mz_show_cluster_privileges

The mz_show_cluster_privileges view contains a row for each cluster privilege granted in the system on user clusters to user roles.

Field Type Meaning
grantor text The role that granted the privilege.
grantee text The role that the privilege was granted to.
name text The name of the cluster.
privilege_type text They type of privilege granted.

mz_show_database_privileges

The mz_show_database_privileges view contains a row for each database privilege granted in the system on user databases to user roles.

Field Type Meaning
grantor text The role that granted the privilege.
grantee text The role that the privilege was granted to.
name text The name of the database.
privilege_type text They type of privilege granted.

mz_show_default_privileges

The mz_show_default_privileges view contains a row for each default privilege granted in the system in user databases and schemas to user roles.

Field Type Meaning
object_owner text Privileges described in this row will be granted on objects created by object_owner.
database text Privileges described in this row will be granted only on objects created in database if non-null.
schema text Privileges described in this row will be granted only on objects created in schema if non-null.
object_type text Privileges described in this row will be granted only on objects of type object_type.
grantee text Privileges described in this row will be granted to grantee.
privilege_type text They type of privilege to be granted.

mz_show_object_privileges

The mz_show_object_privileges view contains a row for each object privilege granted in the system on user objects to user roles.

Field Type Meaning
grantor text The role that granted the privilege.
grantee text The role that the privilege was granted to.
database text The name of the database containing the object.
schema text The name of the schema containing the object.
name text The name of the object.
object_type text The type of object the privilege is granted on.
privilege_type text They type of privilege granted.

mz_show_role_members

The mz_show_role_members view contains a row for each role membership in the system.

Field Type Meaning
role text The role that member is a member of.
member text The role that is a member of role.
grantor text The role that granted membership of member to role.

mz_show_schema_privileges

The mz_show_schema_privileges view contains a row for each schema privilege granted in the system on user schemas to user roles.

Field Type Meaning
grantor text The role that granted the privilege.
grantee text The role that the privilege was granted to.
database text The name of the database containing the schema.
name text The name of the schema.
privilege_type text They type of privilege granted.

mz_show_system_privileges

The mz_show_system_privileges view contains a row for each system privilege granted in the system on to user roles.

Field Type Meaning
grantor text The role that granted the privilege.
grantee text The role that the privilege was granted to.
privilege_type text They type of privilege granted.

mz_show_all_my_privileges

The mz_show_all_my_privileges view is the same as mz_show_all_privileges, but only includes rows where the current role is a direct or indirect member of grantee.

Field Type Meaning
grantor text The role that granted the privilege.
grantee text The role that the privilege was granted to.
database text The name of the database containing the object.
schema text The name of the schema containing the object.
name text The name of the privilege target.
object_type text The type of object the privilege is granted on.
privilege_type text They type of privilege granted.

mz_show_my_cluster_privileges

The mz_show_my_cluster_privileges view is the same as mz_show_cluster_privileges, but only includes rows where the current role is a direct or indirect member of grantee.

Field Type Meaning
grantor text The role that granted the privilege.
grantee text The role that the privilege was granted to.
name text The name of the cluster.
privilege_type text They type of privilege granted.

mz_show_my_database_privileges

The mz_show_my_database_privileges view is the same as mz_show_database_privileges, but only includes rows where the current role is a direct or indirect member of grantee.

Field Type Meaning
grantor text The role that granted the privilege.
grantee text The role that the privilege was granted to.
name text The name of the cluster.
privilege_type text They type of privilege granted.

mz_show_my_default_privileges

The mz_show_my_default_privileges view is the same as mz_show_default_privileges, but only includes rows where the current role is a direct or indirect member of grantee.

Field Type Meaning
object_owner text Privileges described in this row will be granted on objects created by object_owner.
database text Privileges described in this row will be granted only on objects created in database if non-null.
schema text Privileges described in this row will be granted only on objects created in schema if non-null.
object_type text Privileges described in this row will be granted only on objects of type object_type.
grantee text Privileges described in this row will be granted to grantee.
privilege_type text They type of privilege to be granted.

mz_show_my_object_privileges

The mz_show_my_object_privileges view is the same as mz_show_object_privileges, but only includes rows where the current role is a direct or indirect member of grantee.

Field Type Meaning
grantor text The role that granted the privilege.
grantee text The role that the privilege was granted to.
database text The name of the database containing the object.
schema text The name of the schema containing the object.
name text The name of the object.
object_type text The type of object the privilege is granted on.
privilege_type text They type of privilege granted.

mz_show_my_role_members

The mz_show_my_role_members view is the same as mz_show_role_members, but only includes rows where the current role is a direct or indirect member of member.

Field Type Meaning
role text The role that member is a member of.
member text The role that is a member of role.
grantor text The role that granted membership of member to role.

mz_show_my_schema_privileges

The mz_show_my_schema_privileges view is the same as mz_show_schema_privileges, but only includes rows where the current role is a direct or indirect member of grantee.

Field Type Meaning
grantor text The role that granted the privilege.
grantee text The role that the privilege was granted to.
database text The name of the database containing the schema.
name text The name of the schema.
privilege_type text They type of privilege granted.

mz_show_my_system_privileges

The mz_show_my_system_privileges view is the same as mz_show_system_privileges, but only includes rows where the current role is a direct or indirect member of grantee.

Field Type Meaning
grantor text The role that granted the privilege.
grantee text The role that the privilege was granted to.
privilege_type text They type of privilege granted.

mz_sink_statistics

The mz_sink_statistics table contains statistics for each worker thread of each sink in the system.

Materialize does not make any guarantees about the exactness or freshness of these statistics. They are occasionally reset to zero as internal components of the system are restarted.

Field Type Meaning
id text The ID of the source. Corresponds to mz_catalog.mz_sources.id.
worker_id uint8 The ID of the worker thread.
messages_staged uint8 The number of messages staged but possibly not committed to the sink.
messages_committed uint8 The number of messages committed to the sink.
bytes_staged uint8 The number of bytes staged but possibly not committed to the sink. This counts both keys and values, if applicable.
bytes_committed uint8 The number of bytes committed to the sink. This counts both keys and values, if applicable.

mz_sink_statuses

The mz_sink_statuses view provides the current state for each sink in the system, including potential error messages and additional metadata helpful for debugging.

Field Type Meaning
id text The ID of the sink. Corresponds to mz_catalog.mz_sinks.id.
name text The name of the sink.
type text The type of the sink.
last_status_change_at timestamp with time zone Wall-clock timestamp of the sink status change.
status text The status of the sink: one of created, starting, running, stalled, failed, or dropped.
error text If the sink is in an error state, the error message.
details jsonb Additional metadata provided by the sink. In case of error, may contain a hint field with helpful suggestions.

mz_sink_status_history

The mz_sink_status_history table contains rows describing the history of changes to the status of each sink in the system, including potential error messages and additional metadata helpful for debugging.

Field Type Meaning
occurred_at timestamp with time zone Wall-clock timestamp of the sink status change.
sink_id text The ID of the sink. Corresponds to mz_catalog.mz_sinks.id.
status text The status of the sink: one of created, starting, running, stalled, failed, or dropped.
error text If the sink is in an error state, the error message.
details jsonb Additional metadata provided by the sink. In case of error, may contain a hint field with helpful suggestions.

mz_source_statistics

The mz_source_statistics table contains statistics for each worker thread of each source in the system.

Materialize does not make any guarantees about the exactness or freshness of these statistics. They are occasionally reset to zero as internal components of the system are restarted.

Field Type Meaning
id text The ID of the source. Corresponds to mz_catalog.mz_sources.id.
worker_id uint8 The ID of the worker thread.
snapshot_committed boolean Whether the worker has committed the initial snapshot for a source.
messages_received uint8 The number of messages the worker has received from the external system. Messages are counted in a source type-specific manner. Messages do not correspond directly to updates: some messages produce multiple updates, while other messages may be coalesced into a single update.
updates_staged uint8 The number of updates (insertions plus deletions) the worker has written but not yet committed to the storage layer.
updates_committed uint8 The number of updates (insertions plus deletions) the worker has committed to the storage layer.
bytes_received uint8 The number of bytes the worker has read from the external system. Bytes are counted in a source type-specific manner and may or may not include protocol overhead.
envelope_state_bytes uint8 The number of bytes stored in the source envelope state.
envelope_state_count uint8 The number of individual records stored in the source envelope state.
rehydration_latency_ms uint8 The amount of time in milliseconds it took for the worker to rehydrate the source envelope state.

mz_source_statuses

The mz_source_statuses view provides the current state for each source in the system, including potential error messages and additional metadata helpful for debugging.

Field Type Meaning
id text The ID of the source. Corresponds to mz_catalog.mz_sources.id.
name text The name of the source.
type text The type of the source.
last_status_change_at timestamp with time zone Wall-clock timestamp of the source status change.
status text The status of the source: one of created, starting, running, stalled, failed, or dropped.
error text If the source is in an error state, the error message.
details jsonb Additional metadata provided by the source. In case of error, may contain a hint field with helpful suggestions.

mz_source_status_history

The mz_source_status_history table contains a row describing the status of the historical state for each source in the system, including potential error messages and additional metadata helpful for debugging.

Field Type Meaning
occurred_at timestamp with time zone Wall-clock timestamp of the source status change.
source_id text The ID of the source. Corresponds to mz_catalog.mz_sources.id.
status text The status of the source: one of created, starting, running, stalled, failed, or dropped.
error text If the source is in an error state, the error message.
details jsonb Additional metadata provided by the source. In case of error, may contain a hint field with helpful suggestions.

mz_subscriptions

The mz_subscriptions table describes all active SUBSCRIBE operations in the system.

Field Type Meaning
id text The ID of the subscription.
session_id uint4 The ID of the session that runs the subscription. Corresponds to mz_sessions.id.
cluster_id text The ID of the cluster on which the subscription is running. Corresponds to mz_clusters.id.
created_at timestamp with time zone The time at which the subscription was created.
referenced_object_ids text list The IDs of objects referenced by the subscription. Corresponds to mz_objects.id

mz_webhook_sources

The mz_webhook_sources table contains a row for each webhook source in the system.

Field Type Meaning
id text The ID of the webhook source. Corresponds to mz_sources.id.
name text The name of the webhook source.
url text The URL which can be used to send events to the source.

Replica Introspection Relations

This section lists the available replica introspection relations.

Introspection relations are maintained by independently collecting internal logging information within each of the replicas of a cluster. Thus, in a multi-replica cluster, queries to these relations need to be directed to a specific replica by issuing the command SET cluster_replica = <replica_name>. Note that once this command is issued, all subsequent SELECT queries, for introspection relations or not, will be directed to the targeted replica. Replica targeting can be cancelled by issuing the command RESET cluster_replica.

For each of the below introspection relations, there exists also a variant with a _per_worker name suffix. Per-worker relations expose the same data as their global counterparts, but have an extra worker_id column that splits the information by Timely Dataflow worker.

mz_active_peeks

The mz_active_peeks view describes all read queries (“peeks”) that are pending in the dataflow layer.

Field Type Meaning
id uuid The ID of the peek request.
index_id text The ID of the index the peek is targeting. Corresponds to mz_catalog.mz_indexes.id.
time mz_timestamp The timestamp the peek has requested.

mz_arrangement_sharing

The mz_arrangement_sharing view describes how many times each arrangement in the system is used.

Field Type Meaning
operator_id uint8 The ID of the operator that created the arrangement. Corresponds to mz_dataflow_operators.id.
count bigint The number of operators that share the arrangement.

mz_arrangement_sizes

The mz_arrangement_sizes view describes the size of each arrangement in the system.

The size, capacity, and allocations are an approximation, which may underestimate the actual size in memory. Specifically, reductions can use more memory than we show here.

Field Type Meaning
operator_id uint8 The ID of the operator that created the arrangement. Corresponds to mz_dataflow_operators.id.
records numeric The number of records in the arrangement.
batches numeric The number of batches in the arrangement.
size numeric The utilized size in bytes of the arrangement.
capacity numeric The capacity in bytes of the arrangement. Can be larger than the size.
allocations numeric The number of separate memory allocations backing the arrangement.

mz_compute_delays_histogram

The mz_compute_delays_histogram view describes a histogram of the wall-clock delay in nanoseconds between observations of import frontier advancements of a dataflow and the advancements of the corresponding export frontiers.

Field Type Meaning
export_id text The ID of the dataflow export. Corresponds to mz_compute_exports.export_id.
import_id text The ID of the dataflow import. Corresponds to either mz_catalog.mz_sources.id or mz_catalog.mz_tables.id or mz_catalog.mz_materialized_views.id.
delay_ns uint8 The upper bound of the bucket in nanoseconds.
count numeric The (noncumulative) count of delay measurements in this bucket.

mz_compute_exports

The mz_compute_exports view describes the objects exported by dataflows in the system.

Field Type Meaning
export_id text The ID of the index, materialized view, or subscription exported by the dataflow. Corresponds to mz_catalog.mz_indexes.id, mz_catalog.mz_materialized_views.id, or mz_internal.mz_subscriptions.
dataflow_id uint8 The ID of the dataflow. Corresponds to mz_dataflows.id.

mz_compute_frontiers

The mz_compute_frontiers view describes the frontier of each dataflow export in the system. The frontier describes the earliest timestamp at which the output of the dataflow may change; data prior to that timestamp is sealed.

Field Type Meaning
export_id text The ID of the dataflow export. Corresponds to mz_compute_exports.export_id.
time mz_timestamp The next timestamp at which the dataflow output may change.

mz_compute_import_frontiers

The mz_compute_import_frontiers view describes the frontiers of each dataflow import in the system. The frontier describes the earliest timestamp at which the input into the dataflow may change; data prior to that timestamp is sealed.

Field Type Meaning
export_id text The ID of the dataflow export. Corresponds to mz_compute_exports.export_id.
import_id text The ID of the dataflow import. Corresponds to mz_catalog.mz_sources.id or mz_catalog.mz_tables.id or mz_compute_exports.export_id.
time mz_timestamp The next timestamp at which the dataflow input may change.

mz_compute_operator_durations_histogram

The mz_compute_operator_durations_histogram view describes a histogram of the duration in nanoseconds of each invocation for each dataflow operator.

Field Type Meaning
id uint8 The ID of the operator. Corresponds to mz_dataflow_operators.id.
duration_ns uint8 The upper bound of the duration bucket in nanoseconds.
count numeric The (noncumulative) count of invocations in the bucket.

mz_dataflows

The mz_dataflows view describes the dataflows in the system.

Field Type Meaning
id uint8 The ID of the dataflow.
name text The internal name of the dataflow.

mz_dataflow_addresses

The mz_dataflow_addresses view describes how the dataflow channels and operators in the system are nested into scopes.

Field Type Meaning
id uint8 The ID of the channel or operator. Corresponds to mz_dataflow_channels.id or mz_dataflow_operators.id.
address bigint list A list of scope-local indexes indicating the path from the root to this channel or operator.

mz_dataflow_arrangement_sizes

The mz_dataflow_arrangement_sizes view describes the size of arrangements per operators under each dataflow.

Field Type Meaning
id uint8 The ID of the dataflow. Corresponds to mz_dataflows.id.
name text The name of the dataflow.
records numeric The number of records in all arrangements in the dataflow.
batches numeric The number of batches in all arrangements in the dataflow.
size numeric The utilized size in bytes of the arrangements.
capacity numeric The capacity in bytes of the arrangements. Can be larger than the size.
allocations numeric The number of separate memory allocations backing the arrangements.

mz_dataflow_channels

The mz_dataflow_channels view describes the communication channels between dataflow operators. A communication channel connects one of the outputs of a source operator to one of the inputs of a target operator.

Field Type Meaning
id uint8 The ID of the channel.
from_index uint8 The scope-local index of the source operator. Corresponds to mz_dataflow_addresses.address.
from_port uint8 The source operator’s output port.
to_index uint8 The scope-local index of the target operator. Corresponds to mz_dataflow_addresses.address.
to_port uint8 The target operator’s input port.

mz_dataflow_channel_operators

The mz_dataflow_channel_operators view associates dataflow channels with the operators that are their endpoints.

Field Type Meaning
id uint8 The ID of the channel. Corresponds to mz_dataflow_channels.id.
from_operator_id uint8 The ID of the source of the channel. Corresponds to mz_dataflow_operators.id.
from_operator_address [uint8 list] The address of the source of the channel. Corresponds to mz_dataflow_addresses.address.
to_operator_id uint8 The ID of the target of the channel. Corresponds to mz_dataflow_operators.id.
to_operator_address [uint8 list] The address of the target of the channel. Corresponds to mz_dataflow_addresses.address.

mz_dataflow_operators

The mz_dataflow_operators view describes the dataflow operators in the system.

Field Type Meaning
id uint8 The ID of the operator.
name text The internal name of the operator.

mz_dataflow_operator_dataflows

The mz_dataflow_operator_dataflows view describes the dataflow to which each operator belongs.

Field Type Meaning
id uint8 The ID of the operator. Corresponds to mz_dataflow_operators.id.
name text The internal name of the operator.
dataflow_id uint8 The ID of the dataflow hosting the operator. Corresponds to mz_dataflows.id.
dataflow_name text The internal name of the dataflow hosting the operator.

mz_dataflow_operator_parents

The mz_dataflow_operator_parents view describes how dataflow operators are nested into scopes, by relating operators to their parent operators.

Field Type Meaning
id uint8 The ID of the operator. Corresponds to mz_dataflow_operators.id.
parent_id uint8 The ID of the operator’s parent operator. Corresponds to mz_dataflow_operators.id.

mz_dataflow_shutdown_durations_histogram

The mz_dataflow_shutdown_durations_histogram view describes a histogram of the time in nanoseconds required to fully shut down dropped dataflows.

Field Type Meaning
duration_ns uint8 The upper bound of the bucket in nanoseconds.
count numeric The (noncumulative) count of dataflows in this bucket.

mz_expected_group_size_advice

The mz_expected_group_size_advice view provides advice on opportunities to set query hints. Query hints are applicable to dataflows maintaining MIN, MAX, or Top K query patterns. The maintainance of these query patterns is implemented inside an operator scope, called a region, through a hierarchical scheme for either aggregation or Top K computations.

Field Type Meaning
dataflow_id uint8 The ID of the dataflow. Corresponds to mz_dataflows.id.
dataflow_name text The internal name of the dataflow hosting the min/max aggregation or Top K.
region_id uint8 The ID of the root operator scope. Corresponds to mz_dataflow_operators.id.
region_name text The internal name of the root operator scope for the min/max aggregation or Top K.
levels bigint The number of levels in the hierarchical scheme implemented by the region.
to_cut bigint The number of levels that can be eliminated (cut) from the region’s hierarchy.
savings numeric A conservative estimate of the amount of memory in bytes to be saved by applying the hint.
hint double precision The hint value that will eliminate to_cut levels from the region’s hierarchy.

mz_message_counts

The mz_message_counts view describes the messages and message batches sent and received over the dataflow channels in the system. It distinguishes between individual records (sent, received) and batches of records (batch_sent, batch_sent).

Field Type Meaning
channel_id uint8 The ID of the channel. Corresponds to mz_dataflow_channels.id.
sent numeric The number of messages sent.
received numeric The number of messages received.
batch_sent numeric The number of batches sent.
batch_received numeric The number of batches received.

mz_peek_durations_histogram

The mz_peek_durations_histogram view describes a histogram of the duration in nanoseconds of read queries (“peeks”) in the dataflow layer.

Field Type Meaning
duration_ns uint8 The upper bound of the bucket in nanoseconds.
count numeric The (noncumulative) count of peeks in this bucket.

mz_records_per_dataflow

The mz_records_per_dataflow view describes the number of records in each dataflow.

Field Type Meaning
id uint8 The ID of the dataflow. Corresponds to mz_dataflows.id.
name text The internal name of the dataflow.
records numeric The number of records in the dataflow.
batches numeric The number of batches in the dataflow.
size numeric The utilized size in bytes of the arrangements.
capacity numeric The capacity in bytes of the arrangements. Can be larger than the size.
allocations numeric The number of separate memory allocations backing the arrangements.

mz_records_per_dataflow_operator

The mz_records_per_dataflow_operator view describes the number of records in each dataflow operator in the system.

Field Type Meaning
id uint8 The ID of the operator. Corresponds to mz_dataflow_operators.id.
name text The internal name of the operator.
dataflow_id uint8 The ID of the dataflow. Corresponds to mz_dataflows.id.
records numeric The number of records in the operator.
batches numeric The number of batches in the dataflow.
size numeric The utilized size in bytes of the arrangement.
capacity numeric The capacity in bytes of the arrangement. Can be larger than the size.
allocations numeric The number of separate memory allocations backing the arrangement.

mz_scheduling_elapsed

The mz_scheduling_elapsed view describes the total amount of time spent in each dataflow operator.

Field Type Meaning
id uint8 The ID of the operator. Corresponds to mz_dataflow_operators.id.
elapsed_ns numeric The total elapsed time spent in the operator in nanoseconds.

mz_scheduling_parks_histogram

The mz_scheduling_parks_histogram view describes a histogram of dataflow worker park events. A park event occurs when a worker has no outstanding work.

Field Type Meaning
slept_for_ns uint8 The actual length of the park event in nanoseconds.
requested_ns uint8 The requested length of the park event in nanoseconds.
count numeric The (noncumulative) count of park events in this bucket.
Back to top ↑