Materialize v0.87

v0.87

Sources and sinks

  • Add support for handling batched events formatted as NDJSON in the webhook source.

    CREATE SOURCE webhook_json IN CLUSTER quickstart FROM WEBHOOK
    BODY FORMAT JSON;
    
    -- Send multiple events delimited by newlines to the webhook source.
    HTTP POST to 'webhook_json'
      { 'event_type': 'foo' }
      { 'event_type': 'bar' }
    
    SELECT COUNT(*) FROM webhook_json;
    2
    
  • Allow specifying a default AWS PrivateLink connection when creating a Kafka connection over PrivateLink using the AWS PRIVATELINK top-level option. The default connection will be used to connect to all brokers, and is exclusive with the BROKER and BROKERS options.

    CREATE CONNECTION privatelink_svc TO AWS PRIVATELINK (
        SERVICE NAME 'com.amazonaws.vpce.us-east-1.vpce-svc-0e123abc123198abc',
        AVAILABILITY ZONES ('use1-az1')
    );
    
    CREATE CONNECTION kafka_connection TO KAFKA (
        AWS PRIVATELINK (PORT 30292)
        SECURITY PROTOCOL = 'SASL_PLAINTEXT',
        SASL MECHANISMS = 'SCRAM-SHA-256',
        SASL USERNAME = 'foo',
        SASL PASSWORD = SECRET red_panda_password
    );
    
  • Add topic to the mz_internal.mz_kafka_sources system catalog table. This column contains the name of the Kafka topic the source is reading from.

SQL

  • Support user-configured data retention for tables via the RETAIN HISTORY syntax.

Bug fixes and other improvements

  • Add a node_ids output modifier for EXPLAIN PHYSICAL PLAN statements, to show the unique ID of each subplan in the plan (#24944).
Back to top ↑