Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: postgresql-cfbot/postgresql
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: cf/5326~1
Choose a base ref
...
head repository: postgresql-cfbot/postgresql
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: cf/5326
Choose a head ref
  • 6 commits
  • 11 files changed
  • 2 contributors

Commits on Sep 24, 2025

  1. Introduces table AM APIs for parallel table vacuuming. 

    This commit introduces the following new table AM APIs for parallel
table vacuuming:

- parallel_vacuum_compute_workers
- parallel_vacuum_estimate
- parallel_vacuum_initialize
- parallel_vacuum_initialize_worker
- parallel_vacuum_collect_dead_items

All callbacks are optional. parallel_vacuum_compute_workers needs to
return 0 to disable parallel table vacuuming.

There is no code using these new APIs for now. Upcoming parallel
vacuum patches utilize these APIs.

Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com>
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/CAD21AoAEfCNv-GgaDheDJ+s-p_Lv1H24AiJeNoPGCmZNSwL1YA@mail.gmail.com
    @MasahikoSawada
    MasahikoSawada authored and Commitfest Bot committed Sep 24, 2025
    Configuration menu
    Copy the full SHA
    2a01909 View commit details
    Browse the repository at this point in the history

  2. vacuumparallel.c: Support parallel vacuuming for tables to collect de… 

    …ad items.

Previously, parallel vacuum was available only for index vacuuming and
index cleanup, ParallelVacuumState was initialized only when the table
has at least two indexes that are eligible for parallel index
vacuuming and cleanup.

This commit extends vacuumparallel.c to support parallel table
vacuuming. parallel_vacuum_init() now initializes ParallelVacuumState
to perform parallel heap scan to collect dead items, or paralel index
vacuuming/cleanup, or both. During the initialization, it asks the
table AM for the number of parallel workers required for parallel
table vacuuming. If >0, it enables parallel table vacuuming and calls
further table AM APIs such as parallel_vacuum_estimate.

For parallel table vacuuming, this commit introduces
parallel_vacuum_collect_dead_items_begin() function, which can be used
to collect dead items in the table (for example, the first pass over
heap table in lazy vacuum for heap tables).

Heap table AM disables the parallel heap vacuuming for now, but an
upcoming patch uses it.

Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com>
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/CAD21AoAEfCNv-GgaDheDJ+s-p_Lv1H24AiJeNoPGCmZNSwL1YA@mail.gmail.com
    @MasahikoSawada
    MasahikoSawada authored and Commitfest Bot committed Sep 24, 2025
    Configuration menu
    Copy the full SHA
    4ca0fb7 View commit details
    Browse the repository at this point in the history

  3. Move lazy heap scan related variables to new struct LVScanData. 

    This is a pure refactoring for upcoming parallel heap scan, which
requires storing relation statistics and relation data such as extant
oldest XID/MXID collected during lazy heap scan to a shared memory
area.

Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com>
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Discussion: https://postgr.es/m/CAD21AoAEfCNv-GgaDheDJ+s-p_Lv1H24AiJeNoPGCmZNSwL1YA@mail.gmail.com
    @MasahikoSawada
    MasahikoSawada authored and Commitfest Bot committed Sep 24, 2025
    Configuration menu
    Copy the full SHA
    b4ec567 View commit details
    Browse the repository at this point in the history

  4. Support parallelism for collecting dead items during lazy vacuum. 

    This feature allows the vacuum to leverage multiple CPUs in order to
collect dead items (i.e. the first pass over heap table) with parallel
workers. The parallel degree for parallel heap vacuuming is determined
based on the number of blocks to vacuum unless PARALLEL option of
VACUUM command is specified, and further limited by
max_parallel_maintenance_workers.

For the parallel heap scan to collect dead items, we utilize a
parallel block table scan, controlled by ParallelBlockTableScanDesc,
in conjunction with the read stream. The workers' parallel scan
descriptions are stored in the DSM space, enabling different parallel
workers to resume the heap scan (phase 1) after a cycle of heap
vacuuming and index vacuuming (phase 2 and 3) from their previous
state. However, due to the potential presence of pinned buffers loaded
by the read stream's look-ahead mechanism, we cannot abruptly stop
phase 1 even when the space of dead_items TIDs exceeds the
limit. Therefore, once the space of dead_items TIDs exceeds the limit,
we begin processing pages without attempting to retrieve additional
blocks by look-ahead mechanism until the read stream is exhausted,
even if the the memory limit is surpassed. While this approach may
increase the memory usage, it typically doesn't pose a significant
problem, as processing a few 10s-100s buffers doesn't substantially
increase the size of dead_items TIDs.

Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Reviewed-by: Tomas Vondra <tomas@vondra.me>
Reviewed-by: Dilip Kumar <dilipbalaut@gmail.com>
Reviewed-by: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CAD21AoAEfCNv-GgaDheDJ+s-p_Lv1H24AiJeNoPGCmZNSwL1YA@mail.gmail.com
    @MasahikoSawada
    MasahikoSawada authored and Commitfest Bot committed Sep 24, 2025
    Configuration menu
    Copy the full SHA
    964ce65 View commit details
    Browse the repository at this point in the history

  5. Add more parallel vacuum tests.

    @MasahikoSawada
    MasahikoSawada authored and Commitfest Bot committed Sep 24, 2025
    Configuration menu
    Copy the full SHA
    c6be54e View commit details
    Browse the repository at this point in the history

  6. [CF 5326] v19 - Parallel heap vacuum 

    This branch was automatically generated by a robot using patches from an
email thread registered at:

https://commitfest.postgresql.org/patch/5326

The branch will be overwritten each time a new patch version is posted to
the thread, and also periodically to check for bitrot caused by changes
on the master branch.

Patch(es): https://www.postgresql.org/message-id/df858858-72da-4649-aaf9-ab97e4fb2d6f@vondra.me
Author(s): Masahiko Sawada
    Commitfest Bot committed Sep 24, 2025
    Configuration menu
    Copy the full SHA
    63ec3f8 View commit details
    Browse the repository at this point in the history
Loading