Private Preview: AWS IAM authentication for Amazon MSK

If you're streaming data into Materialize from an Amazon MSK cluster, you now have the option to authenticate using an AWS Identity and Access Management (IAM) role! Here's how:

1. Create an AWS connection to allow Materialize to securely authenticate with your Amazon MSK cluster.

-- Replace <account-id> with the 12-digit number that identifies your AWS

-- account, and <role> with the name of the IAM role you want Materialize to

-- assume.

CREATE CONNECTION aws_msk TO AWS (

  ASSUME ROLE ARN = 'arn:aws:iam::<account-id>:role/<role>'

);

1. Validate the connection, to double-check that you didn’t miss any IAM configuration steps.

VALIDATE CONNECTION aws_msk;

1. Create a Kafka connection with the access and authentication details for your Amazon MSK cluster, and use the AWS connection you just created.

CREATE CONNECTION kafka_msk TO KAFKA (

    BROKER 'msk.mycorp.com:9092',

    -- To use IAM authentication, the security protocol must be

    -- SASL_PLAINTEXT or SASL_SSL.

    SECURITY PROTOCOL = 'SASL_SSL',

    AWS CONNECTION = aws_msk

);

Depending on your security architecture, you can also use AWS IAM authentication in combination with PrivateLink connections. And yes — this feature is already supported in the Materialize Terraform provider (v0.8.7+)! 👾