We get it: schemas change.
We’ve landed some quality-of-life improvements that make it easier to
handle schema evolution and replication errors in PostgreSQL sources.
By easier, we mean that you no longer have to drop and recreate the whole
source, but can instead use the new ALTER SOURCE...{ADD|DROP} TABLE syntax to
patch a specific table (or set of tables).
As an example, if you add a important_col column to the important_tbl table
in your upstream PostgreSQL database, and want to make sure this new column
starts being ingested in the pg_source PostgreSQL source in Materialize:
-- List all subsources in pg_source
SHOW SUBSOURCES ON pg_source;
-- Get rid of the outdated subsource
ALTER SOURCE pg_source DROP TABLE important_tbl;
-- Start ingesting the table with the updated schema
ALTER SOURCE pg_source ADD TABLE important_tbl;The same approach works for incompatible schema changes that might cause one or more subsources to stall. For more details on how to handle each scenario, check the updated documentation.