Commit b82ea98
Add configurable conflict log table for Logical Replication
This patch adds a feature to provide a structured, queryable record of all
logical replication conflicts. The current approach of logging conflicts as
plain text in the server logs makes it difficult to query, analyze, and
use for external monitoring and automation.
This patch addresses these limitations by introducing a configurable
conflict_log_table option in the CREATE SUBSCRIPTION command. Key design
decisions include:
User-Managed Table: The conflict log is stored in a user-managed table
rather than a system catalog.
Structured Data: Conflict details, including the original and remote tuples,
are stored in JSON columns, providing a flexible format to accommodate different
table schemas.
Comprehensive Information: The log table captures essential attributes such as
local and remote transaction IDs, LSNs, commit timestamps, and conflict type,
providing a complete record for post-mortem analysis.
This feature will make logical replication conflicts easier to monitor and manage,
significantly improving the overall resilience and operability of replication setups.
The conflict log tables will not be included in a publication, even if the publication
is configured to include ALL TABLES or ALL TABLES IN SCHEMA.
Note: A single remote tuple may conflict with multiple local tuples when conflict type
is CT_MULTIPLE_UNIQUE_CONFLICTS, so for handling this case we create a single row in
conflict log table with respect to each remote conflict tuple even if it conflicts with
multiple local tuples and we store the multiple conflict tuples as a single JSON array
element in format as
[ { "xid": "1001", "commit_ts": "...", "origin": "...", "tuple": {...} }, ... ]
We can extract the elements of the local tuples from the conflict log table row
as given in below example.
SELECT remote_xid, relname, remote_origin, local_conflicts[1] ->> 'xid' AS local_xid,
local_conflicts[1] ->> 'tuple' AS local_tuple
FROM myschema.conflict_log_history2;
remote_xid | relname | remote_origin | local_xid | local_tuple
------------+----------+---------------+-----------+---------------------
760 | test | pg_16406 | 771 | {"a":1,"b":10}
765 | conf_tab | pg_16406 | 775 | {"a":2,"b":2,"c":2}1 parent 795e94c commit b82ea98
File tree
15 files changed
+1381
-128
lines changed- src
- backend
- catalog
- commands
- replication/logical
- utils/cache
- bin/psql
- include
- catalog
- commands
- replication
- utils
- test/regress
- expected
- sql
15 files changed
+1381
-128
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| 34 | + | |
34 | 35 | | |
35 | 36 | | |
36 | 37 | | |
| |||
85 | 86 | | |
86 | 87 | | |
87 | 88 | | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
88 | 98 | | |
89 | 99 | | |
90 | 100 | | |
| |||
145 | 155 | | |
146 | 156 | | |
147 | 157 | | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
148 | 165 | | |
149 | 166 | | |
150 | 167 | | |
| |||
169 | 186 | | |
170 | 187 | | |
171 | 188 | | |
172 | | - | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
173 | 193 | | |
174 | 194 | | |
175 | 195 | | |
| |||
890 | 910 | | |
891 | 911 | | |
892 | 912 | | |
| 913 | + | |
893 | 914 | | |
| 915 | + | |
894 | 916 | | |
895 | 917 | | |
896 | 918 | | |
| |||
1018 | 1040 | | |
1019 | 1041 | | |
1020 | 1042 | | |
1021 | | - | |
| 1043 | + | |
1022 | 1044 | | |
1023 | 1045 | | |
1024 | 1046 | | |
| |||
0 commit comments