Skip to main content

​
Required settings

  • Host name
  • Port (default is 5432)
  • Service account
  • Database with logical replication enabled

​
Provider specific instructions

Amazon Aurora or RDS

Google CloudSQL

Supabase

​
Additional features

Artie will automatically detect partitioned tables and will automatically fan data into a single table in the destination.
  • If you have specified publish_via_partition_root to be true in your publication, no additional work is needed.
  • Else, you will want to open up the pipeline advanced settings and make sure the regular expression matches your partitioned table names. Further, if your partitions are stored in a separate schema, you can specify the child schema name under the table advanced settings.
Publication settings in Artie
Logical replication on standby replicas is not available for Amazon Aurora.
PostgreSQL 16 introduces the ability to use logical replication on a standby server.Steps to enable this:
  1. Create dbz_publication publication on the primary database
CREATE PUBLICATION dbz_publication FOR ALL TABLES;
  1. Enable hot_standby_feedback on the standby server
postgres> SHOW hot_standby_feedback;
 hot_standby_feedback
----------------------
 on
  1. WAL_LEVEL is set to logical on the standby server
postgres> SHOW WAL_LEVEL;
 wal_level
----------------------
 logical
Once you selected Enable heartbeats under Advanced settings, you will then need to run the following command:
CREATE TABLE test_heartbeat_table (id text PRIMARY KEY, ts timestamp);
-- Grant access to the heartbeat table
GRANT SELECT, UPDATE ON TABLE test_heartbeat_table TO artie;
-- Then insert one row into this table.
-- Artie's periodic pings will be this:
-- UPDATE test_heartbeat_table set ts = now() WHERE id = '1';
-- Such that we never end up adding additional rows.
INSERT INTO test_heartbeat_table (id, ts) VALUES (1, NOW());
Don’t want to enable publications to capture all the tables? You can change this behavior by selecting Filtered under the Pipeline advanced settings.If you change this to be filtered, you are responsible for adding and removing any captured tables from the publication. Our validation will check that the publications is configured correctly.

Changing the behavior of Postgres publications

TOAST (The Oversized-Attribute Storage Technique) is a mechanism used by Postgres to store large column values to alleviate the page size limit. TOAST columns may be different data types such as text, jsonb, bytea, array, etc.One thing that is special about TOAST columns is that if the column value has not changed, the column value will not be included in WAL if the table’s replica identity is set to default. Artie has automatic detection built-in to detect an unchanged TOAST column and will conditionally replicate this into the target database.Our built-in process will detect if an incoming CDC event has an unchanged TOAST column and will not update the column value in the target database. We are able to do all of this without having to increase your table replica identity.
We have regular monitors that provide additional guardrails around your database replication, and will do the following:
  • Regularly check and monitor your replication slot size in 15-minute intervals and notify if the slot exceeds a certain threshold.
  • Heartbeats verification. For folks that are leveraging heartbeats, we will alert you if a heartbeat query fails.
  • Terminate any idle queries that are lingering for more than a day. You can avoid having this problem by setting idle_in_transaction_session_timeout.
⌘I