mz_internal
The following sections describe the available objects in the mz_internal
schema.
mz_internal
schema are not part of Materialize’s stable interface.
Backwards-incompatible changes to these tables may be made at any time.
SELECT
statements may reference these objects, but creating views that
reference these objects is not allowed.
System Relations
mz_cluster_replica_metrics
The mz_cluster_replica_metrics
table gives the last known CPU and RAM utilization statistics
for all processes of all extant cluster replicas.
At this time, we do not make any guarantees about the exactness or freshness of these numbers.
Field | Type | Meaning |
---|---|---|
replica_id |
text |
The ID of a cluster replica. |
process_id |
uint8 |
An identifier of a compute process within a replica. |
cpu_nano_cores |
uint8 |
Approximate CPU usage, in billionths of a vCPU core. |
memory_bytes |
uint8 |
Approximate RAM usage, in bytes. |
disk_bytes |
uint8 |
Approximate disk usage in bytes, if the replica has a disk attached. NULL otherwise. |
mz_cluster_replica_sizes
The mz_cluster_replica_sizes
table contains a mapping of logical sizes
(e.g. “xlarge”) to physical sizes (number of processes, and CPU and memory allocations per process).
Field | Type | Meaning |
---|---|---|
size |
text |
The human-readable replica size. |
processes |
uint8 |
The number of processes in the replica. |
workers |
uint8 |
The number of Timely Dataflow workers per process. |
cpu_nano_cores |
uint8 |
The CPU allocation per process, in billionths of a vCPU core. |
memory_bytes |
uint8 |
The RAM allocation per process, in billionths of a vCPU core. |
disk_bytes |
uint8 |
The disk allocation per process, if the replica has a disk attached. NULL otherwise. |
credits_per_hour |
numeric |
The number of compute credits consumed per hour. |
mz_cluster_links
The mz_cluster_links
table contains a row for each cluster that is linked to a
source or sink. When present, the lifetime of the specified cluster is tied to
the lifetime of the specified source or sink: the cluster cannot be dropped
without dropping the linked source or sink, and dropping the linked source or
sink will also drop the cluster. There is at most one row per cluster.
Field | Type | Meaning |
---|---|---|
cluster_id |
text |
The ID of the cluster. Corresponds to mz_clusters.id . |
object_id |
text |
The ID of the source or sink. Corresponds to mz_objects.id . |
mz_cluster_replica_statuses
The mz_cluster_replica_statuses
table contains a row describing the status
of each process in each cluster replica in the system.
Field | Type | Meaning |
---|---|---|
replica_id |
text |
Materialize’s unique ID for the cluster replica. |
process_id |
uint8 |
The ID of the process within the cluster replica. |
status |
text |
The status of the cluster replica: ready or not-ready . |
reason |
text |
If the cluster replica is in a not-ready state, the reason (if available). For example, oom-killed . |
updated_at |
timestamp with time zone |
The time at which the status was last updated. |
mz_cluster_replica_utilization
The mz_cluster_replica_utilization
view gives the last known CPU and RAM utilization statistics
for all processes of all extant cluster replicas, as a percentage of the total resource allocation.
At this time, we do not make any guarantees about the exactness or freshness of these numbers.
Field | Type | Meaning |
---|---|---|
replica_id |
text |
The ID of a cluster replica. |
process_id |
uint8 |
An identifier of a compute process within a replica. |
cpu_percent |
double precision |
Approximate CPU usage in percent of the total allocation. |
memory_percent |
double precision |
Approximate RAM usage in percent of the total allocation. |
disk_percent |
double precision |
Approximate disk usage in percent of the total allocation, if the replica has a disk attached. NULL otherwise. |
mz_cluster_replica_heartbeats
The mz_cluster_replica_heartbeats
table gives the last known heartbeat of all
extant cluster replicas.
Field | Type | Meaning |
---|---|---|
replica_id |
text |
The ID of a cluster replica. |
last_heartbeat |
timestamp with time zone |
The time of the replica’s last heartbeat. |
mz_cluster_replica_history
The mz_cluster_replica_history
view contains information about the timespan of
each replica, including the times at which it was created and dropped
(if applicable).
Field | Type | Meaning |
---|---|---|
replica_id |
text |
The ID of a cluster replica. |
size |
text |
The size of the cluster replica. Corresponds to mz_cluster_replica_sizes.size . |
cluster_name |
text |
The name of the cluster associated with the replica. |
replica_name |
text |
The name of the replica. |
created_at |
timestamp with time zone |
The time at which the replica was created. |
dropped_at |
timestamp with time zone |
The time at which the replica was dropped, or NULL if it still exists. |
credits_per_hour |
numeric |
The number of compute credits consumed per hour. Corresponds to mz_cluster_replica_sizes.credits_per_hour . |
mz_internal_cluster_replicas
The mz_internal_cluster_replicas
table lists the replicas that are created and maintained by Materialize support.
Field | Type | Meaning |
---|---|---|
id | text |
The ID of a cluster replica. Corresponds to mz_cluster_replicas.id . |
mz_comments
The mz_comments
table stores optional comments (descriptions) for objects in the database.
Field | Type | Meaning |
---|---|---|
id |
text |
The ID of the object. Corresponds to mz_objects.id . |
object_type |
text |
The type of object the comment is associated with. |
object_sub_id |
uint8 |
For a comment on a column of a relation, this is the column number. For all other object types this column is NULL . |
comment |
text |
The comment itself. |
mz_compute_dependencies
The mz_compute_dependencies
table describes the dependency structure between each compute object (index, materialized view, or subscription) and the sources of its data.
In contrast to mz_object_dependencies
, this table only lists dependencies in the compute layer.
SQL objects that don’t exist in the compute layer (such as views) are omitted.
Field | Type | Meaning |
---|---|---|
object_id |
text |
The ID of a compute object. Corresponds to mz_catalog.mz_indexes.id , mz_catalog.mz_materialized_views.id , or mz_internal.mz_subscriptions . |
dependency_id |
text |
The ID of a compute dependency. Corresponds to mz_catalog.mz_indexes.id , mz_catalog.mz_materialized_views.id , mz_catalog.mz_sources.id , or mz_catalog.mz_tables.id . |
mz_frontiers
The mz_frontiers
table describes the frontiers of each source, sink, table,
materialized view, index, and subscription in the system, as observed from the
coordinator.
For objects that are installed on replicas (e.g., materialized views and
indexes), the replica_id
field is always non-NULL
. If an object is installed
on multiple replicas, it has multiple entries describing the frontier on each
individual replica. For objects that are not installed on replicas (e.g.,
tables), the replica_id
field is NULL
.
mz_compute_frontiers
is similar to mz_frontiers
,
but mz_compute_frontiers
reports the frontiers known to the active compute
replica, while mz_frontiers
reports the frontiers of all replicas. Note also
that mz_compute_frontiers
is restricted to compute objects (indexes,
materialized views, and subscriptions) while mz_frontiers
contains storage
objects (sources, sinks, and tables) as well.
At this time, we do not make any guarantees about the freshness of these numbers.
Field | Type | Meaning |
---|---|---|
object_id |
text |
The ID of the source, sink, table, index, materialized view, or subscription. |
replica_id |
text |
The ID of a cluster replica, or NULL if the object is not installed on a replica. |
time |
mz_timestamp |
The next timestamp at which the output may change. |
mz_global_frontiers
The mz_global_frontiers
view describes the global frontiers of each source,
sink, table, materialized view, index, and subscription in the system, as
observed from the coordinator.
For objects that are installed on replicas (e.g., materialized views and indexes), the global frontier is the maximum of the per-replica frontiers. Objects that are not installed on replicas only have a single, global frontier.
At this time, we do not make any guarantees about the freshness of these numbers.
Field | Type | Meaning |
---|---|---|
object_id |
text |
The ID of the source, sink, table, index, materialized view, or subscription. |
time |
mz_timestamp |
The next timestamp at which the output may change. |
mz_kafka_sources
The mz_kafka_sources
table contains a row for each Kafka source in the system.
Field | Type | Meaning |
---|---|---|
id |
text |
The ID of the Kafka source. Corresponds to mz_catalog.mz_sources.id . |
group_id_base |
text |
The prefix of the group ID that Materialize will use when consuming data for the Kafka source. |
mz_object_dependencies
The mz_object_dependencies
table describes the dependency structure between
all database objects in the system.
Field | Type | Meaning |
---|---|---|
object_id |
text |
The ID of the dependent object. Corresponds to mz_objects.id . |
referenced_object_id |
text |
The ID of the referenced object. Corresponds to mz_objects.id . |
mz_object_fully_qualified_names
The mz_object_fully_qualified_names
view enriches the mz_catalog.mz_objects
view with namespace information.
Field | Type | Meaning |
---|---|---|
id |
text |
Materialize’s unique ID for the object. |
name |
text |
The name of the object. |
object_type |
text |
The type of the object: one of table , source , view , materialized view , sink , index , connection , secret , type , or function . |
schema_name |
text |
The name of the schema to which the object belongs. Corresponds to mz_schemas.name . |
database_name |
text |
The name of the database to which the object belongs. Corresponds to mz_databases.name . |
mz_object_lifetimes
The mz_object_lifetimes
view enriches the mz_catalog.mz_objects
view with information about the last lifetime event that occurred for each object in the system.
Field | Type | Meaning |
---|---|---|
id |
text |
Materialize’s unique ID for the object. |
object_type |
text |
The type of the object: one of table , source , view , materialized view , sink , index , connection , secret , type , or function . |
event_type |
text |
The lifetime event, either create or drop . |
occurred_at |
timestamp with time zone |
Wall-clock timestamp of when the event occurred. |
mz_object_transitive_dependencies
The mz_object_transitive_dependencies
view describes the transitive dependency structure between
all database objects in the system.
The view is defined as the transitive closure of mz_object_dependencies
.
Field | Type | Meaning |
---|---|---|
object_id |
text |
The ID of the dependent object. Corresponds to mz_objects.id . |
referenced_object_id |
text |
The ID of the (possibly transitively) referenced object. Corresponds to mz_objects.id . |
mz_postgres_sources
The mz_postgres_sources
table contains a row for each PostgreSQL source in the
system.
Field | Type | Meaning |
---|---|---|
id |
text |
The ID of the source. Corresponds to mz_catalog.mz_sources.id . |
replication_slot |
text |
The name of the replication slot in the PostgreSQL database that Materialize will create and stream data from. |
mz_sessions
The mz_sessions
table contains a row for each active session in the system.
Field | Type | Meaning |
---|---|---|
id |
uint4 |
The ID of the session. |
role_id |
text |
The role ID of the role that the session is logged in as. Corresponds to mz_catalog.mz_roles . |
connected_at |
timestamp with time zone |
The time at which the session connected to the system. |
mz_show_all_privileges
The mz_show_all_privileges
view contains a row for each privilege granted
in the system on user objects to user roles.
Field | Type | Meaning |
---|---|---|
grantor |
text |
The role that granted the privilege. |
grantee |
text |
The role that the privilege was granted to. |
database |
text |
The name of the database containing the object. |
schema |
text |
The name of the schema containing the object. |
name |
text |
The name of the privilege target. |
object_type |
text |
The type of object the privilege is granted on. |
privilege_type |
text |
They type of privilege granted. |
mz_show_cluster_privileges
The mz_show_cluster_privileges
view contains a row for each cluster privilege granted
in the system on user clusters to user roles.
Field | Type | Meaning |
---|---|---|
grantor |
text |
The role that granted the privilege. |
grantee |
text |
The role that the privilege was granted to. |
name |
text |
The name of the cluster. |
privilege_type |
text |
They type of privilege granted. |
mz_show_database_privileges
The mz_show_database_privileges
view contains a row for each database privilege granted
in the system on user databases to user roles.
Field | Type | Meaning |
---|---|---|
grantor |
text |
The role that granted the privilege. |
grantee |
text |
The role that the privilege was granted to. |
name |
text |
The name of the database. |
privilege_type |
text |
They type of privilege granted. |
mz_show_default_privileges
The mz_show_default_privileges
view contains a row for each default privilege granted
in the system in user databases and schemas to user roles.
Field | Type | Meaning |
---|---|---|
object_owner |
text |
Privileges described in this row will be granted on objects created by object_owner . |
database |
text |
Privileges described in this row will be granted only on objects created in database if non-null. |
schema |
text |
Privileges described in this row will be granted only on objects created in schema if non-null. |
object_type |
text |
Privileges described in this row will be granted only on objects of type object_type . |
grantee |
text |
Privileges described in this row will be granted to grantee . |
privilege_type |
text |
They type of privilege to be granted. |
mz_show_object_privileges
The mz_show_object_privileges
view contains a row for each object privilege granted
in the system on user objects to user roles.
Field | Type | Meaning |
---|---|---|
grantor |
text |
The role that granted the privilege. |
grantee |
text |
The role that the privilege was granted to. |
database |
text |
The name of the database containing the object. |
schema |
text |
The name of the schema containing the object. |
name |
text |
The name of the object. |
object_type |
text |
The type of object the privilege is granted on. |
privilege_type |
text |
They type of privilege granted. |
mz_show_role_members
The mz_show_role_members
view contains a row for each role membership in the system.
Field | Type | Meaning |
---|---|---|
role |
text |
The role that member is a member of. |
member |
text |
The role that is a member of role . |
grantor |
text |
The role that granted membership of member to role . |
mz_show_schema_privileges
The mz_show_schema_privileges
view contains a row for each schema privilege granted
in the system on user schemas to user roles.
Field | Type | Meaning |
---|---|---|
grantor |
text |
The role that granted the privilege. |
grantee |
text |
The role that the privilege was granted to. |
database |
text |
The name of the database containing the schema. |
name |
text |
The name of the schema. |
privilege_type |
text |
They type of privilege granted. |
mz_show_system_privileges
The mz_show_system_privileges
view contains a row for each system privilege granted
in the system on to user roles.
Field | Type | Meaning |
---|---|---|
grantor |
text |
The role that granted the privilege. |
grantee |
text |
The role that the privilege was granted to. |
privilege_type |
text |
They type of privilege granted. |
mz_show_all_my_privileges
The mz_show_all_my_privileges
view is the same as
mz_show_all_privileges
, but
only includes rows where the current role is a direct or indirect member of grantee
.
Field | Type | Meaning |
---|---|---|
grantor |
text |
The role that granted the privilege. |
grantee |
text |
The role that the privilege was granted to. |
database |
text |
The name of the database containing the object. |
schema |
text |
The name of the schema containing the object. |
name |
text |
The name of the privilege target. |
object_type |
text |
The type of object the privilege is granted on. |
privilege_type |
text |
They type of privilege granted. |
mz_show_my_cluster_privileges
The mz_show_my_cluster_privileges
view is the same as
mz_show_cluster_privileges
, but
only includes rows where the current role is a direct or indirect member of grantee
.
Field | Type | Meaning |
---|---|---|
grantor |
text |
The role that granted the privilege. |
grantee |
text |
The role that the privilege was granted to. |
name |
text |
The name of the cluster. |
privilege_type |
text |
They type of privilege granted. |
mz_show_my_database_privileges
The mz_show_my_database_privileges
view is the same as
mz_show_database_privileges
, but
only includes rows where the current role is a direct or indirect member of grantee
.
Field | Type | Meaning |
---|---|---|
grantor |
text |
The role that granted the privilege. |
grantee |
text |
The role that the privilege was granted to. |
name |
text |
The name of the cluster. |
privilege_type |
text |
They type of privilege granted. |
mz_show_my_default_privileges
The mz_show_my_default_privileges
view is the same as
mz_show_default_privileges
, but
only includes rows where the current role is a direct or indirect member of grantee
.
Field | Type | Meaning |
---|---|---|
object_owner |
text |
Privileges described in this row will be granted on objects created by object_owner . |
database |
text |
Privileges described in this row will be granted only on objects created in database if non-null. |
schema |
text |
Privileges described in this row will be granted only on objects created in schema if non-null. |
object_type |
text |
Privileges described in this row will be granted only on objects of type object_type . |
grantee |
text |
Privileges described in this row will be granted to grantee . |
privilege_type |
text |
They type of privilege to be granted. |
mz_show_my_object_privileges
The mz_show_my_object_privileges
view is the same as
mz_show_object_privileges
, but
only includes rows where the current role is a direct or indirect member of grantee
.
Field | Type | Meaning |
---|---|---|
grantor |
text |
The role that granted the privilege. |
grantee |
text |
The role that the privilege was granted to. |
database |
text |
The name of the database containing the object. |
schema |
text |
The name of the schema containing the object. |
name |
text |
The name of the object. |
object_type |
text |
The type of object the privilege is granted on. |
privilege_type |
text |
They type of privilege granted. |
mz_show_my_role_members
The mz_show_my_role_members
view is the same as
mz_show_role_members
, but
only includes rows where the current role is a direct or indirect member of member
.
Field | Type | Meaning |
---|---|---|
role |
text |
The role that member is a member of. |
member |
text |
The role that is a member of role . |
grantor |
text |
The role that granted membership of member to role . |
mz_show_my_schema_privileges
The mz_show_my_schema_privileges
view is the same as
mz_show_schema_privileges
, but
only includes rows where the current role is a direct or indirect member of grantee
.
Field | Type | Meaning |
---|---|---|
grantor |
text |
The role that granted the privilege. |
grantee |
text |
The role that the privilege was granted to. |
database |
text |
The name of the database containing the schema. |
name |
text |
The name of the schema. |
privilege_type |
text |
They type of privilege granted. |
mz_show_my_system_privileges
The mz_show_my_system_privileges
view is the same as
mz_show_system_privileges
, but
only includes rows where the current role is a direct or indirect member of grantee
.
Field | Type | Meaning |
---|---|---|
grantor |
text |
The role that granted the privilege. |
grantee |
text |
The role that the privilege was granted to. |
privilege_type |
text |
They type of privilege granted. |
mz_sink_statistics
The mz_sink_statistics
table contains statistics for each worker thread of
each sink in the system.
Materialize does not make any guarantees about the exactness or freshness of these statistics. They are occasionally reset to zero as internal components of the system are restarted.
Field | Type | Meaning |
---|---|---|
id |
text |
The ID of the source. Corresponds to mz_catalog.mz_sources.id . |
worker_id |
uint8 |
The ID of the worker thread. |
messages_staged |
uint8 |
The number of messages staged but possibly not committed to the sink. |
messages_committed |
uint8 |
The number of messages committed to the sink. |
bytes_staged |
uint8 |
The number of bytes staged but possibly not committed to the sink. This counts both keys and values, if applicable. |
bytes_committed |
uint8 |
The number of bytes committed to the sink. This counts both keys and values, if applicable. |
mz_sink_statuses
The mz_sink_statuses
view provides the current state for each sink in the
system, including potential error messages and additional metadata helpful for
debugging.
Field | Type | Meaning |
---|---|---|
id |
text |
The ID of the sink. Corresponds to mz_catalog.mz_sinks.id . |
name |
text |
The name of the sink. |
type |
text |
The type of the sink. |
last_status_change_at |
timestamp with time zone |
Wall-clock timestamp of the sink status change. |
status |
text |
The status of the sink: one of created , starting , running , stalled , failed , or dropped . |
error |
text |
If the sink is in an error state, the error message. |
details |
jsonb |
Additional metadata provided by the sink. In case of error, may contain a hint field with helpful suggestions. |
mz_sink_status_history
The mz_sink_status_history
table contains rows describing the
history of changes to the status of each sink in the system, including potential error
messages and additional metadata helpful for debugging.
Field | Type | Meaning |
---|---|---|
occurred_at |
timestamp with time zone |
Wall-clock timestamp of the sink status change. |
sink_id |
text |
The ID of the sink. Corresponds to mz_catalog.mz_sinks.id . |
status |
text |
The status of the sink: one of created , starting , running , stalled , failed , or dropped . |
error |
text |
If the sink is in an error state, the error message. |
details |
jsonb |
Additional metadata provided by the sink. In case of error, may contain a hint field with helpful suggestions. |
mz_source_statistics
The mz_source_statistics
table contains statistics for each worker thread of
each source in the system.
Materialize does not make any guarantees about the exactness or freshness of these statistics. They are occasionally reset to zero as internal components of the system are restarted.
Field | Type | Meaning |
---|---|---|
id |
text |
The ID of the source. Corresponds to mz_catalog.mz_sources.id . |
worker_id |
uint8 |
The ID of the worker thread. |
snapshot_committed |
boolean |
Whether the worker has committed the initial snapshot for a source. |
messages_received |
uint8 |
The number of messages the worker has received from the external system. Messages are counted in a source type-specific manner. Messages do not correspond directly to updates: some messages produce multiple updates, while other messages may be coalesced into a single update. |
updates_staged |
uint8 |
The number of updates (insertions plus deletions) the worker has written but not yet committed to the storage layer. |
updates_committed |
uint8 |
The number of updates (insertions plus deletions) the worker has committed to the storage layer. |
bytes_received |
uint8 |
The number of bytes the worker has read from the external system. Bytes are counted in a source type-specific manner and may or may not include protocol overhead. |
envelope_state_bytes |
uint8 |
The number of bytes stored in the source envelope state. |
envelope_state_count |
uint8 |
The number of individual records stored in the source envelope state. |
rehydration_latency_ms |
uint8 |
The amount of time in milliseconds it took for the worker to rehydrate the source envelope state. |
mz_source_statuses
The mz_source_statuses
view provides the current state for each source in the
system, including potential error messages and additional metadata helpful for
debugging.
Field | Type | Meaning |
---|---|---|
id |
text |
The ID of the source. Corresponds to mz_catalog.mz_sources.id . |
name |
text |
The name of the source. |
type |
text |
The type of the source. |
last_status_change_at |
timestamp with time zone |
Wall-clock timestamp of the source status change. |
status |
text |
The status of the source: one of created , starting , running , stalled , failed , or dropped . |
error |
text |
If the source is in an error state, the error message. |
details |
jsonb |
Additional metadata provided by the source. In case of error, may contain a hint field with helpful suggestions. |
mz_source_status_history
The mz_source_status_history
table contains a row describing the status of the
historical state for each source in the system, including potential error
messages and additional metadata helpful for debugging.
Field | Type | Meaning |
---|---|---|
occurred_at |
timestamp with time zone |
Wall-clock timestamp of the source status change. |
source_id |
text |
The ID of the source. Corresponds to mz_catalog.mz_sources.id . |
status |
text |
The status of the source: one of created , starting , running , stalled , failed , or dropped . |
error |
text |
If the source is in an error state, the error message. |
details |
jsonb |
Additional metadata provided by the source. In case of error, may contain a hint field with helpful suggestions. |
mz_subscriptions
The mz_subscriptions
table describes all active SUBSCRIBE
operations in the system.
Field | Type | Meaning |
---|---|---|
id |
text |
The ID of the subscription. |
session_id |
uint4 |
The ID of the session that runs the subscription. Corresponds to mz_sessions.id . |
cluster_id |
text |
The ID of the cluster on which the subscription is running. Corresponds to mz_clusters.id . |
created_at |
timestamp with time zone |
The time at which the subscription was created. |
referenced_object_ids |
text list |
The IDs of objects referenced by the subscription. Corresponds to mz_objects.id |
mz_webhook_sources
The mz_webhook_sources
table contains a row for each webhook source in the system.
Field | Type | Meaning |
---|---|---|
id |
text |
The ID of the webhook source. Corresponds to mz_sources.id . |
name |
text |
The name of the webhook source. |
url |
text |
The URL which can be used to send events to the source. |
Replica Introspection Relations
This section lists the available replica introspection relations.
Introspection relations are maintained by independently collecting internal logging information within each of the replicas of a cluster.
Thus, in a multi-replica cluster, queries to these relations need to be directed to a specific replica by issuing the command SET cluster_replica = <replica_name>
.
Note that once this command is issued, all subsequent SELECT
queries, for introspection relations or not, will be directed to the targeted replica.
Replica targeting can be cancelled by issuing the command RESET cluster_replica
.
For each of the below introspection relations, there exists also a variant with a _per_worker
name suffix.
Per-worker relations expose the same data as their global counterparts, but have an extra worker_id
column that splits the information by Timely Dataflow worker.
mz_active_peeks
The mz_active_peeks
view describes all read queries (“peeks”) that are pending in the dataflow layer.
Field | Type | Meaning |
---|---|---|
id |
uuid |
The ID of the peek request. |
index_id |
text |
The ID of the index the peek is targeting. Corresponds to mz_catalog.mz_indexes.id . |
time |
mz_timestamp |
The timestamp the peek has requested. |
mz_arrangement_sharing
The mz_arrangement_sharing
view describes how many times each arrangement in the system is used.
Field | Type | Meaning |
---|---|---|
operator_id |
uint8 |
The ID of the operator that created the arrangement. Corresponds to mz_dataflow_operators.id . |
count |
bigint |
The number of operators that share the arrangement. |
mz_arrangement_sizes
The mz_arrangement_sizes
view describes the size of each arrangement in the system.
The size, capacity, and allocations are an approximation, which may underestimate the actual size in memory. Specifically, reductions can use more memory than we show here.
Field | Type | Meaning |
---|---|---|
operator_id |
uint8 |
The ID of the operator that created the arrangement. Corresponds to mz_dataflow_operators.id . |
records |
numeric |
The number of records in the arrangement. |
batches |
numeric |
The number of batches in the arrangement. |
size |
numeric |
The utilized size in bytes of the arrangement. |
capacity |
numeric |
The capacity in bytes of the arrangement. Can be larger than the size. |
allocations |
numeric |
The number of separate memory allocations backing the arrangement. |
mz_compute_delays_histogram
The mz_compute_delays_histogram
view describes a histogram of the wall-clock delay in nanoseconds between observations of import frontier advancements of a dataflow and the advancements of the corresponding export frontiers.
Field | Type | Meaning |
---|---|---|
export_id |
text |
The ID of the dataflow export. Corresponds to mz_compute_exports.export_id . |
import_id |
text |
The ID of the dataflow import. Corresponds to either mz_catalog.mz_sources.id or mz_catalog.mz_tables.id or mz_catalog.mz_materialized_views.id . |
delay_ns |
uint8 |
The upper bound of the bucket in nanoseconds. |
count |
numeric |
The (noncumulative) count of delay measurements in this bucket. |
mz_compute_exports
The mz_compute_exports
view describes the objects exported by dataflows in the system.
Field | Type | Meaning |
---|---|---|
export_id |
text |
The ID of the index, materialized view, or subscription exported by the dataflow. Corresponds to mz_catalog.mz_indexes.id , mz_catalog.mz_materialized_views.id , or mz_internal.mz_subscriptions . |
dataflow_id |
uint8 |
The ID of the dataflow. Corresponds to mz_dataflows.id . |
mz_compute_frontiers
The mz_compute_frontiers
view describes the frontier of each dataflow export in the system.
The frontier describes the earliest timestamp at which the output of the dataflow may change; data prior to that timestamp is sealed.
Field | Type | Meaning |
---|---|---|
export_id |
text |
The ID of the dataflow export. Corresponds to mz_compute_exports.export_id . |
time |
mz_timestamp |
The next timestamp at which the dataflow output may change. |
mz_compute_import_frontiers
The mz_compute_import_frontiers
view describes the frontiers of each dataflow import in the system.
The frontier describes the earliest timestamp at which the input into the dataflow may change; data prior to that timestamp is sealed.
Field | Type | Meaning |
---|---|---|
export_id |
text |
The ID of the dataflow export. Corresponds to mz_compute_exports.export_id . |
import_id |
text |
The ID of the dataflow import. Corresponds to mz_catalog.mz_sources.id or mz_catalog.mz_tables.id or mz_compute_exports.export_id . |
time |
mz_timestamp |
The next timestamp at which the dataflow input may change. |
mz_compute_operator_durations_histogram
The mz_compute_operator_durations_histogram
view describes a histogram of the duration in nanoseconds of each invocation for each dataflow operator.
Field | Type | Meaning |
---|---|---|
id |
uint8 |
The ID of the operator. Corresponds to mz_dataflow_operators.id . |
duration_ns |
uint8 |
The upper bound of the duration bucket in nanoseconds. |
count |
numeric |
The (noncumulative) count of invocations in the bucket. |
mz_dataflows
The mz_dataflows
view describes the dataflows in the system.
Field | Type | Meaning |
---|---|---|
id |
uint8 |
The ID of the dataflow. |
name |
text |
The internal name of the dataflow. |
mz_dataflow_addresses
The mz_dataflow_addresses
view describes how the dataflow channels and operators in the system are nested into scopes.
Field | Type | Meaning |
---|---|---|
id |
uint8 |
The ID of the channel or operator. Corresponds to mz_dataflow_channels.id or mz_dataflow_operators.id . |
address |
bigint list |
A list of scope-local indexes indicating the path from the root to this channel or operator. |
mz_dataflow_arrangement_sizes
The mz_dataflow_arrangement_sizes
view describes the size of arrangements per
operators under each dataflow.
Field | Type | Meaning |
---|---|---|
id |
uint8 |
The ID of the dataflow. Corresponds to mz_dataflows.id . |
name |
text |
The name of the dataflow. |
records |
numeric |
The number of records in all arrangements in the dataflow. |
batches |
numeric |
The number of batches in all arrangements in the dataflow. |
size |
numeric |
The utilized size in bytes of the arrangements. |
capacity |
numeric |
The capacity in bytes of the arrangements. Can be larger than the size. |
allocations |
numeric |
The number of separate memory allocations backing the arrangements. |
mz_dataflow_channels
The mz_dataflow_channels
view describes the communication channels between dataflow operators.
A communication channel connects one of the outputs of a source operator to one of the inputs of a target operator.
Field | Type | Meaning |
---|---|---|
id |
uint8 |
The ID of the channel. |
from_index |
uint8 |
The scope-local index of the source operator. Corresponds to mz_dataflow_addresses.address . |
from_port |
uint8 |
The source operator’s output port. |
to_index |
uint8 |
The scope-local index of the target operator. Corresponds to mz_dataflow_addresses.address . |
to_port |
uint8 |
The target operator’s input port. |
mz_dataflow_channel_operators
The mz_dataflow_channel_operators
view associates dataflow channels with the operators that are their endpoints.
Field | Type | Meaning |
---|---|---|
id |
uint8 |
The ID of the channel. Corresponds to mz_dataflow_channels.id . |
from_operator_id |
uint8 |
The ID of the source of the channel. Corresponds to mz_dataflow_operators.id . |
from_operator_address |
[uint8 list ] |
The address of the source of the channel. Corresponds to mz_dataflow_addresses.address . |
to_operator_id |
uint8 |
The ID of the target of the channel. Corresponds to mz_dataflow_operators.id . |
to_operator_address |
[uint8 list ] |
The address of the target of the channel. Corresponds to mz_dataflow_addresses.address . |
mz_dataflow_operators
The mz_dataflow_operators
view describes the dataflow operators in the system.
Field | Type | Meaning |
---|---|---|
id |
uint8 |
The ID of the operator. |
name |
text |
The internal name of the operator. |
mz_dataflow_operator_dataflows
The mz_dataflow_operator_dataflows
view describes the dataflow to which each operator belongs.
Field | Type | Meaning |
---|---|---|
id |
uint8 |
The ID of the operator. Corresponds to mz_dataflow_operators.id . |
name |
text |
The internal name of the operator. |
dataflow_id |
uint8 |
The ID of the dataflow hosting the operator. Corresponds to mz_dataflows.id . |
dataflow_name |
text |
The internal name of the dataflow hosting the operator. |
mz_dataflow_operator_parents
The mz_dataflow_operator_parents
view describes how dataflow operators are nested into scopes, by relating operators to their parent operators.
Field | Type | Meaning |
---|---|---|
id |
uint8 |
The ID of the operator. Corresponds to mz_dataflow_operators.id . |
parent_id |
uint8 |
The ID of the operator’s parent operator. Corresponds to mz_dataflow_operators.id . |
mz_dataflow_shutdown_durations_histogram
The mz_dataflow_shutdown_durations_histogram
view describes a histogram of the time in nanoseconds required to fully shut down dropped dataflows.
Field | Type | Meaning |
---|---|---|
duration_ns |
uint8 |
The upper bound of the bucket in nanoseconds. |
count |
numeric |
The (noncumulative) count of dataflows in this bucket. |
mz_expected_group_size_advice
The mz_expected_group_size_advice
view provides advice on opportunities to set query hints.
Query hints are applicable to dataflows maintaining MIN
, MAX
, or Top K query patterns.
The maintainance of these query patterns is implemented inside an operator scope, called a region,
through a hierarchical scheme for either aggregation or Top K computations.
Field | Type | Meaning |
---|---|---|
dataflow_id |
uint8 |
The ID of the dataflow. Corresponds to mz_dataflows.id . |
dataflow_name |
text |
The internal name of the dataflow hosting the min/max aggregation or Top K. |
region_id |
uint8 |
The ID of the root operator scope. Corresponds to mz_dataflow_operators.id . |
region_name |
text |
The internal name of the root operator scope for the min/max aggregation or Top K. |
levels |
bigint |
The number of levels in the hierarchical scheme implemented by the region. |
to_cut |
bigint |
The number of levels that can be eliminated (cut) from the region’s hierarchy. |
savings |
numeric |
A conservative estimate of the amount of memory in bytes to be saved by applying the hint. |
hint |
double precision |
The hint value that will eliminate to_cut levels from the region’s hierarchy. |
mz_message_counts
The mz_message_counts
view describes the messages and message batches sent and received over the dataflow channels in the system.
It distinguishes between individual records (sent
, received
) and batches of records (batch_sent
, batch_sent
).
Field | Type | Meaning |
---|---|---|
channel_id |
uint8 |
The ID of the channel. Corresponds to mz_dataflow_channels.id . |
sent |
numeric |
The number of messages sent. |
received |
numeric |
The number of messages received. |
batch_sent |
numeric |
The number of batches sent. |
batch_received |
numeric |
The number of batches received. |
mz_peek_durations_histogram
The mz_peek_durations_histogram
view describes a histogram of the duration in nanoseconds of read queries (“peeks”) in the dataflow layer.
Field | Type | Meaning |
---|---|---|
duration_ns |
uint8 |
The upper bound of the bucket in nanoseconds. |
count |
numeric |
The (noncumulative) count of peeks in this bucket. |
mz_records_per_dataflow
The mz_records_per_dataflow
view describes the number of records in each dataflow.
Field | Type | Meaning |
---|---|---|
id |
uint8 |
The ID of the dataflow. Corresponds to mz_dataflows.id . |
name |
text |
The internal name of the dataflow. |
records |
numeric |
The number of records in the dataflow. |
batches |
numeric |
The number of batches in the dataflow. |
size |
numeric |
The utilized size in bytes of the arrangements. |
capacity |
numeric |
The capacity in bytes of the arrangements. Can be larger than the size. |
allocations |
numeric |
The number of separate memory allocations backing the arrangements. |
mz_records_per_dataflow_operator
The mz_records_per_dataflow_operator
view describes the number of records in each dataflow operator in the system.
Field | Type | Meaning |
---|---|---|
id |
uint8 |
The ID of the operator. Corresponds to mz_dataflow_operators.id . |
name |
text |
The internal name of the operator. |
dataflow_id |
uint8 |
The ID of the dataflow. Corresponds to mz_dataflows.id . |
records |
numeric |
The number of records in the operator. |
batches |
numeric |
The number of batches in the dataflow. |
size |
numeric |
The utilized size in bytes of the arrangement. |
capacity |
numeric |
The capacity in bytes of the arrangement. Can be larger than the size. |
allocations |
numeric |
The number of separate memory allocations backing the arrangement. |
mz_scheduling_elapsed
The mz_scheduling_elapsed
view describes the total amount of time spent in each dataflow operator.
Field | Type | Meaning |
---|---|---|
id |
uint8 |
The ID of the operator. Corresponds to mz_dataflow_operators.id . |
elapsed_ns |
numeric |
The total elapsed time spent in the operator in nanoseconds. |
mz_scheduling_parks_histogram
The mz_scheduling_parks_histogram
view describes a histogram of dataflow worker park events. A park event occurs when a worker has no outstanding work.
Field | Type | Meaning |
---|---|---|
slept_for_ns |
uint8 |
The actual length of the park event in nanoseconds. |
requested_ns |
uint8 |
The requested length of the park event in nanoseconds. |
count |
numeric |
The (noncumulative) count of park events in this bucket. |