Materialize v0.86

v0.86

Sources and sinks

  • Add support for handling batched events in the webhook source via the new JSON ARRAY format.

    CREATE SOURCE webhook_source_json_batch IN CLUSTER my_cluster FROM WEBHOOK
    BODY FORMAT JSON ARRAY
    INCLUDE HEADERS;
    
    POST webhook_source_json_batch
    [
      { "event_type": "a" },
      { "event_type": "b" },
      { "event_type": "c" }
    ]
    
    SELECT COUNT(body) FROM webhook_source_json_batch;
    ----
    3
    
  • Decrease memory utilization for unpacking Kafka headers. Use the new map_build function to turn all headers exposed via INCLUDE HEADERS into a map, which makes it easier to extract header values.

    SELECT
        id,
        seller,
        item,
        convert_from(map_build(headers)->'client_id', 'utf-8') AS client_id,
        map_build(headers)->'encryption_key' AS encryption_key,
    FROM kafka_metadata;
    
     id | seller |        item        | client_id |    encryption_key
    ----+--------+--------------------+-----------+----------------------
      2 |   1592 | Custom Art         |        23 | \x796f75207769736821
      3 |   1411 | City Bar Crawl     |        42 | \x796f75207769736821
    

SQL

  • Add support for new SQL functions:
    Function Description
    map_build Builds a map from a list of records whose fields are two elements, the first of which is text.
    map_agg Aggregate keys and values (including nulls) as a map.

Bug fixes and other improvements

  • Mitigate queue saturation is Kafka sinks (#24871).

  • Fix a correctness issue with subqueries that referred to ungrouped columns when columns of the same name existed in an outer scope (#24354).

  • Fix casts from interval to time for large negative intervals (#24795).

  • Prevent INSERTs with table references in VALUES in transactions (#24697).

Back to top ↑