Comparing changes

This patch implements in-memory B+tree structure. It will be used as index for special type of grouping using index. Size of each node is set using macro. For convenience equals 2^n - 1, so for internal nodes we effectively calculate size of each page and find split node (exactly in the middle), and for leaf nodes we can distribute tuples for each node uniformely (according to the newly inserted tuple). It supports different memory contexts for tracking memory allocations. And just like in TupleHashTable during Lookup it uses 'isnew' pointer to prevent new tuple creation (i.e. when memory limit is reached). Also it has key abbreviation optimization support like in tuplesort. But some code was copied and looks exactly the same way, so it is worth separating such logic into a separate function.

AGG_INDEX is a new grouping strategy that builds in-memory index and use it for grouping. The main advantage of this approach is that output is ordered by grouping columns and if there are any ORDER BY specified, then it will use this to build grouping/sorting columns. For index it uses B+tree which was implemented in previous commit. And overall it's implementation is very close to AGG_HASHED: - maintain in-memory grouping structure - track memory consuption - if memory limit reached spill data to disk in batches (using hash of key columns) - hash batches are processed one after another and for each batch fill new in-memory structure For this reason many code logic is generalized to support both index and hash implementations: function generalization using boolean arguments (i.e. 'ishash'), rename spill logic members in AggState with prefix 'spill_' instead of 'hash_', etc. Most differences are in spill logic: to preserve sort order in case of disk spill we must dump all indexes to disk to create sorted runs and perform final external merge. One problem is external merge. It's adapted from tuplesort.c - introduce new operational mode - tuplemerge (with it's own prefix). Internally we just setup state accordingly and process as earlier without any significant code changes. Another problem is what tuples to save into sorted runs. We decided to store tuples after projection (when it's aggregates are finalized), because internal transition info is represented by value/isnull/novalue tripple (in AggStatePerGroupData) which is quiet hard to serialize and handle, but actually, after projection all group by attributes are saved, so we can access them during merge. Also, projection applies filter, so it can discard some tuples.

This commit adds usage of IndexAggregate in planner and explain (analyze). We calculate cost of IndexAggregate and add AGG_INDEX node to the pathlist. Cost of this node is cost of building B+tree (in memory), disk spill and final external merge. For EXPLAIN there is only little change - show sort information in "Group Key".

Now IndexAggregate support partial aggregates. The main problem was with partial aggregates which creates SortGroupClause for same expression as in target list, but different sortgroupclause, so make_pathkeys_for_sortclases failed to find required target list entry and throws ERROR. To fix this now we explicitly pass pathkeys to create_agg_path (but only for AGG_INDEX for now), so caller is responsible for searching and building pathkeys list.

After adding IndexAggregate node some test output changed and tests broke. This patch updates expected output. Also it adds some IndexAggregate specific tests into aggregates.sql and partition_aggregate.sql.

This branch was automatically generated by a robot using patches from an email thread registered at: https://commitfest.postgresql.org/patch/6299 The branch will be overwritten each time a new patch version is posted to the thread, and also periodically to check for bitrot caused by changes on the master branch. Patch(es): https://www.postgresql.org/message-id/345c715f-edb2-4903-9ae1-00d6ca7e92f1@tantorlabs.ru Author(s): Sergey Solovev

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Comparing changes

Open a pull request

Commits on Dec 12, 2025

This comparison is taking too long to generate.

Uh oh!