Comparing changes

…fest.postgresql.org/patch/5438/ merged in single commit. it is required for stability of stress tests.

Introduce stress tests for concurrent index operations: - test concurrent inserts/updates during CREATE/REINDEX INDEX CONCURRENTLY - cover various index types (btree, gin, gist, brin, hash, spgist) - test unique and non-unique indexes - test with expressions and predicates - test both parallel and non-parallel operations These tests verify the behavior of the following commits.

This patch provides infrastructure for following enhancements to concurrent index builds by: - ii_Auxiliary in IndexInfo: indicates that an index is an auxiliary index used during concurrent index build - validate_index in IndexVacuumInfo: set if index_bulk_delete called during the validation phase of concurrent index build - STIR(Short-Term Index Replacement) access method is introduced, intended solely for short-lived, auxiliary usage STIR functions designed as an ephemeral helper during concurrent index builds, temporarily storing TIDs without providing the full features of a typical access method. As such, it raises warnings or errors when accessed outside its specialized usage path. Planned to be used in following commits.

Extend tuplestore to store individual Datum values: - fixed-length datatypes: store raw bytes without a length header - variable-length datatypes: include a length header and padding - by-value types: store inline This support enables usages tuplestore for non-tuple data (TIDs) in the next commit.

Replace the second table full scan in concurrent index builds with an auxiliary index approach: - create a STIR auxiliary index with the same predicate (if exists) as in main index - use it to track tuples inserted during the first phase - merge auxiliary index with main index during validation to catch up new index with any tuples missed during the first phase - automatically drop auxiliary when main index is ready To merge main and auxiliary indexes: - index_bulk_delete called for both, TIDs put into tuplesort - both tuplesort are being sorted - both tuplesort scanned with two pointers looking for the TIDs present in auxiliary index, but absent in main one - all such TIDs are put into tuplestore - all TIDs in tuplestore are fetched using the stream, tuplestore used in heapam_index_validate_scan_read_stream_next to provide the next page to prefetch - if fetched tuple is alive - it is inserted into the main index This eliminates the need for a second full table scan during validation, improving performance especially for large tables. Affects both CREATE INDEX CONCURRENTLY and REINDEX INDEX CONCURRENTLY operations.

During concurrent index operations, auxiliary indexes may be left as orphaned objects when errors occur (junk auxiliary indexes). This patch improves the handling of such auxiliary indexes: - add auxiliaryForIndexId parameter to index_create() to track dependencies between main and auxiliary indexes - automatically drop auxiliary indexes when the main index is dropped - delete junk auxiliary indexes properly during REINDEX operations

Skip unnecessary computations for auxiliary indices by: - in the index‐insert path, detect auxiliary indexes and bypass Datum value computation - set indexUnchanged=false for auxiliary indices to avoid redundant checks These optimizations reduce overhead during concurrent index operations.

Enhances validation phase of concurrently built indexes by periodically refreshing snapshots rather than using a single reference snapshot. This addresses issues with xmin propagation during long-running validations. The validation now takes a fresh snapshot every few pages, allowing the xmin horizon to advance. This restores feature of commit d9d0762, which was reverted in commit e28bb88. New STIR-based approach is not depends on single reference snapshot anymore.

This branch was automatically generated by a robot using patches from an email thread registered at: https://commitfest.postgresql.org/patch/4971 The branch will be overwritten each time a new patch version is posted to the thread, and also periodically to check for bitrot caused by changes on the master branch. Patch(es): https://www.postgresql.org/message-id/CADzfLwWrGYGE8=cg+6C57Nypv1Y-1mBv8BVzzPWVJy5EfR6YfQ@mail.gmail.com Author(s): Michail Nikolaev, Mihail Nikalayeu

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Comparing changes

Open a pull request

Commits on Nov 24, 2025

This comparison is taking too long to generate.

Uh oh!