# pg_plan_advice — help the planner get the right plan

The `pg_plan_advice` module allows key planner decisions to be described, reproduced, and altered using a special-purpose "plan advice" mini-language. It is intended to allow stabilization of plan choices that the user believes to be good, as well as experimentation with plans that the planner believes to be non-optimal.

Note that, since the planner often makes good decisions, overriding its judgment can easily backfire. For example, if the distribution of the underlying data changes, the planner normally has the option to adjust the plan in an attempt to preserve good performance. If the plan advice prevents this, a very poor plan may be chosen. It is important to use plan advice only when the risks of constraining the planner's choices are outweighed by the benefits.

## Getting Started

First, you must arrange to load the `pg_plan_advice` module. You can do this on a system-wide basis by adding `pg_plan_advice` to guc-shared-preload-libraries and restarting the server, or by adding it to guc-session-preload-libraries and starting a new session, or by loading it into an individual session using the `LOAD` command.

Once the `pg_plan_advice` module is loaded, `EXPLAIN` will support a `PLAN_ADVICE` option. You can use this option to see a plan advice string for the chosen plan. For example:

    EXPLAIN (COSTS OFF, PLAN_ADVICE)
            SELECT * FROM join_fact f JOIN join_dim d ON f.dim_id = d.id;
                 QUERY PLAN
    ------------------------------------
     Hash Join
       Hash Cond: (f.dim_id = d.id)
       ->  Seq Scan on join_fact f
       ->  Hash
             ->  Seq Scan on join_dim d
     Generated Plan Advice:
       JOIN_ORDER(f d)
       HASH_JOIN(d)
       SEQ_SCAN(f d)
       NO_GATHER(f d)

In this example, the user has not specified any advice; instead, the planner has been permitted to make whatever decisions it thinks best, and those decisions are memorialized in the form of an advice string. `JOIN_ORDER(f d)` means that `f` should be the driving table, and the first table to which it should be joined is `d`. `HASH_JOIN(d)` means that `d` should appear on the inner side of a hash join. `SEQ_SCAN(f d)` means that both `f` and `d` should be accessed via a sequential scan. `NO_GATHER(f d)` means that neither `f` nor `d` should appear beneath a `Gather` or `Gather Merge` node. For more details on the plan advice mini-language, see the information on advice targets and advice tags, below.

Once you have an advice string for a query, you can use it to control how that query is planned. You can do this by setting `pg_plan_advice.advice` to the advice string you've chosen. This can be an advice string that was generated by the system, or one you've written yourself. One good way of creating your own advice string is to take the string generated by the system and pick out just those elements that you wish to enforce. In the example above, `pg_plan_advice` emits advice for the join order, the join method, the scan method, and the use of parallelism, but you might only want to control the join order:

    SET pg_plan_advice.advice = 'JOIN_ORDER(f d)';
    EXPLAIN (COSTS OFF)
            SELECT * FROM join_fact f JOIN join_dim d ON f.dim_id = d.id;
                 QUERY PLAN
    ------------------------------------
     Hash Join
       Hash Cond: (f.dim_id = d.id)
       ->  Seq Scan on join_fact f
       ->  Hash
             ->  Seq Scan on join_dim d
     Supplied Plan Advice:
       JOIN_ORDER(f d) /* matched */

Since the `PLAN_ADVICE` option to `EXPLAIN` was not specified, no advice string is generated for the plan. However, the supplied plan advice is still shown so that anyone looking at the `EXPLAIN` output knows that the chosen plan was influenced by plan advice. If information about supplied plan advice is not desired, it can be suppressed by configuring `pg_plan_advice.always_explain_supplied_advice = false`. For each piece of supplied advice, the output shows advice feedback indicating whether or not the advice was successfully applied to the query. In this case, the feedback says `/* matched */`, which means that `f` and `d` were found in the query and that the resulting query plan conforms to the specified advice.

## How It Works

Plan advice is written imperatively; that is, it specifies what should be done. However, at an implementation level, `pg_plan_advice` works by telling the core planner what should not be done. In other words, it operates by constraining the planner's choices, not by replacing it. Therefore, no matter what advice you provide, you will only ever get a plan that the core planner would have considered for the query in question. If you attempt to force what you believe to be the correct plan by supplying an advice string, and the planner still fails to produce the desired plan, this means that either there is a bug in your advice string, or the plan in question was not considered viable by the core planner. This commonly happens for one of two reasons. First, it might be that the planner believes that the plan you're trying to force would be semantically incorrect - that is, it would produce the wrong results - and for that reason it wasn't considered. Second, it might be that the planner rejected the plan you were hoping to generate on some grounds other than cost. For example, given a very simple query such as `SELECT * FROM some_table`, the query planner will decide that the use of an index is worthless here before it performs any costing calculations. You cannot force it to use an index for this query even if you set `enable_seqscan = false`, and you can't force it to use an index using plan advice, either.

Specifying plan advice should never cause planner failure. However, if you specify plan advice that asks for something impossible, you may get a plan where some plan nodes are flagged as `Disabled: true` in the `EXPLAIN` output. In some cases, such plans will be basically the same plan you would have gotten with no supplied advice at all, but in other cases, they may be much worse. For example:

    SET pg_plan_advice.advice = 'JOIN_ORDER(x f d)';
    EXPLAIN (COSTS OFF)
            SELECT * FROM join_fact f JOIN join_dim d ON f.dim_id = d.id;
                         QUERY PLAN
    ----------------------------------------------------
     Nested Loop
       Disabled: true
       ->  Seq Scan on join_fact f
       ->  Index Scan using join_dim_pkey on join_dim d
             Index Cond: (id = f.dim_id)
     Supplied Plan Advice:
       JOIN_ORDER(x f d) /* partially matched */

Because neither `f` nor `d` is the first table in the `JOIN_ORDER()` specification, the planner disables all direct joins between the two of them, thinking that the join to `x` should happen first. Since planning isn't allowed to fail, a disabled plan between the two relations is eventually selected anyway, but here it's a `Nested Loop` rather than the `Hash Join` that was chosen in the above example where no advice was specified. There are several different ways that this kind of thing can happen; when it does, the resulting plan is generally worse than if no advice had been specified at all. Therefore, it is a good idea to validate that the advice you specify applies to the query to which it is applied and that the results are as expected.

## Advice Targets

An advice target uniquely identifies a particular instance of a particular relation involved in a particular query. In simple cases, such as the examples shown above, the advice target is simply the relation alias. However, a more complex syntax is required when subqueries are used, when tables are partitioned, or when the same relation alias is mentioned more than once in the same subquery (e.g., `(foo JOIN bar ON foo.a = bar.a) x JOIN foo ON x.b = foo.b`). Any combination of these three things can occur simultaneously: a relation could be mentioned more than once, be partitioned, and be used inside of a subquery.

Because of this, the general syntax for a relation identifier is:

    alias_name#occurrence_number/partition_schema.partition_name@plan_name

All components except for the `alias_name` are optional and are included only when required. When a component is omitted, the preceding punctuation must also be omitted. For the first occurrence of a relation within a given subquery, generated advice will omit the occurrence number, but it is legal to write `#1`, if desired. The partition schema and partition name are included only for children of partitioned tables. In generated advice, `pg_plan_advice` always includes both, but it is legal to omit the schema. The plan name is omitted for the top-level plan, and must be included for any subplan.

It is not always easy to determine the correct advice target by examining the query. For instance, if the planner pulls up a subquery into the parent query level, everything inside of it becomes part of the parent query level, and uses the parent query's subplan name (or no subplan name, if pulled up to the top level). Furthermore, the correct subquery name is sometimes not obvious. For example, when two queries are joined using an operation such as `UNION` or `INTERSECT`, no name for the subqueries is present in the SQL syntax; instead, a system-generated name is assigned to each branch. The easiest way to discover the proper advice targets is to use `EXPLAIN (PLAN_ADVICE)` and examine the generated advice.

## Advice Tags

An advice tag specifies a particular behavior that should be enforced for some portion of the query, such as a particular join order or join method. All advice tags take advice targets as arguments, and many allow lists of advice targets, which in some cases can be nested multiple levels deep. Several different classes of advice tags exist, each controlling a different aspect of query planning.

### Scan Method Advice

    SEQ_SCAN(target [ ... ])
    TID_SCAN(target [ ... ])
    INDEX_SCAN(target index_name [ ... ])
    INDEX_ONLY_SCAN(target index_name [ ... ])
    FOREIGN_JOIN((target [ ... ]) [ ... ])
    BITMAP_HEAP_SCAN(target [ ... ])
    DO_NOT_SCAN(target [ ... ])

`SEQ_SCAN` specifies that each target should be scanned using a `Seq Scan`. `TID_SCAN` specifies that each target should be scanned using a `TID Scan` or `TID Range Scan`. `BITMAP_HEAP_SCAN` specifies that each target should be scanned using a `Bitmap Heap Scan`.

`INDEX_SCAN` specifies that each target should be scanned using an `Index Scan` on the given index name. `INDEX_ONLY_SCAN` is similar, but specifies the use of an `Index Only Scan`. In either case, the index name can be, but does not have to be, schema-qualified.

`FOREIGN_JOIN` specifies that a join between two or more foreign tables should be pushed down to a remote server so that it can be implemented as a single `Foreign Scan`. Specifying `FOREIGN_JOIN` for a single foreign table is neither necessary nor permissible: a `Foreign Scan` will need to be used regardless. If you want to prevent a join from being pushed down, consider using the `JOIN_ORDER` tag for that purpose.

`DO_NOT_SCAN` specifies that a particular target should not appear in the final plan at all. In most cases, this is impossible, and will simply cause the scan of the target relation to be marked disabled. However, in certain cases, the planner considers optimizations where a portion of the plan tree is copied and mutated, and then considered as an alternative to the original. In those cases, `DO_NOT_SCAN` can be used to exclude the non-preferred alternative.

The planner supports many types of scans other than those listed here; however, in most of those cases, there is no meaningful decision to be made, and hence no need for advice. For example, the output of a set-returning function that appears in the `FROM` clause can only ever be scanned using a `Function Scan`, so there is no opportunity for advice to change anything.

### Join Order Advice

    JOIN_ORDER(join_order_item [ ... ])

    where join_order_item is:

    advice_target |
    ( join_order_item [ ... ] ) |
    { join_order_item [ ... ] }

When `JOIN_ORDER` is used without any sublists, it specifies an outer-deep join with the first advice target as the driving table, joined to each subsequent advice target in turn in the order specified. For instance, `JOIN_ORDER(a b c)` means that `a` should be the driving table, and that it should be joined first to `b` and then to `c`. If there are more relations in the query than `a`, `b`, and `c`, the rest can be joined afterwards in any manner.

If a `JOIN_ORDER` list contains a parenthesized sublist, it specifies a non-outer-deep join. The relations in the sublist must first be joined to each other much as if the sublist were a top-level `JOIN_ORDER` list, and the resulting join product must then appear on the inner side of a join at the appropriate point in the join order. For example, `JOIN_ORDER(a (b c) d)` requires a plan of this form:

    Join
      ->  Join
            -> Scan on a
            -> Join
                 -> Scan on b
                 -> Scan on c
      ->  Scan on d

If a `JOIN_ORDER` list contains a sublist surrounded by curly braces, this also specifies a non-outer-deep join. However, the join order within the sublist is not constrained. For example, specifying `JOIN_ORDER(a {b c} d)` would allow the scans of `b` and `c` to be swapped in the previous example, which is not allowed when parentheses are used.

Parenthesized sublists can be arbitrarily nested, but sublists surrounded by curly braces cannot themselves contain sublists.

Multiple instances of `JOIN_ORDER()` can sometimes be needed in order to fully constrain the join order. This occurs when there are multiple join problems that are optimized separately by the planner. This can happen due to the presence of subqueries, or because there is a partitionwise join. In the latter case, each branch of the partitionwise join can have its own join order, independent of every other branch.

### Join Method Advice

    join_method_name(join_method_item [ ... ])

    where join_method_name is:

    { MERGE_JOIN_MATERIALIZE | MERGE_JOIN_PLAIN | NESTED_LOOP_MATERIALIZE | NESTED_LOOP_MEMOIZE | NESTED_LOOP_PLAIN | HASH_JOIN }

    and join_method_item is:

    { advice_target |
    ( advice_target [ ... ] ) }

Join method advice specifies the relation, or set of relations, that should appear on the inner side of a join using the named join method. For example, `HASH_JOIN(a b)` means that each of `a` and `b` should appear on the inner side of a hash join; a conforming plan must contain at least two hash joins, one of which has `a` and nothing else on the inner side, and the other of which has `b` and nothing else on the inner side. On the other hand, `HASH_JOIN((a b))` means that the join product of `a` and `b` should appear together on the inner side of a single hash join.

Note that join method advice implies a negative join order constraint. Since the named relation or relations must be on the inner side of a join using the specified method, none of them can be the driving table for the entire join problem. Moreover, no relation inside the set should be joined to any relation outside the set until all relations within the set have been joined to each other. For example, if the advice specifies `HASH_JOIN((a b))` and the system begins by joining either of those to some third relation `c`, the resulting plan could never be compliant with the request to put exactly those two relations on the inner side of a hash join. When using both join order advice and join method advice for the same query, it is a good idea to make sure that they do not mandate incompatible join orders.

### Partitionwise Advice

    PARTITIONWISE(partitionwise_item [ ... ])

    where partitionwise_item is:

    { advice_target |
    ( advice_target [ ... ] ) }

When applied to a single target, `PARTITIONWISE` specifies that the specified table should not be part of any partitionwise join. When applied to a list of targets, `PARTITIONWISE` specifies that exactly that set of relations should be joined in partitionwise fashion. Note that, regardless of what advice is specified, no partitionwise joins will be possible if `enable_partitionwise_join = off`.

### Semijoin Uniqueness Advice

    SEMIJOIN_UNIQUE(sj_unique_item [ ... ])
    SEMIJOIN_NON_UNIQUE(sj_unique_item [ ... ])

    where sj_unique_item is:

    { advice_target |
    ( advice_target [ ... ] ) }

The planner sometimes has a choice between implementing a semijoin directly and implementing a semijoin by making the nullable side unique and then performing an inner join. `SEMIJOIN_UNIQUE` specifies the latter strategy, while `SEMIJOIN_NON_UNIQUE` specifies the former strategy. In either case, the argument is the single relation or list of relations that appear beneath the nullable side of the join.

### Parallel Query Advice

    GATHER(gather_item [ ... ])
    GATHER_MERGE(gather_item [ ... ])
    NO_GATHER(advice_target [ ... ])

    where gather_item is:

    { advice_target |
    ( advice_target [ ... ] ) }

`GATHER` or `GATHER_MERGE` specifies that `Gather` or `Gather Merge`, respectively, should be placed on top of the single relation specified as a target, or on top of the join between the list of relations specified as a target. This means that `GATHER(a b c)` is a request for three different `Gather` nodes, while `GATHER((a b c))` is a request for a single `Gather` node on top of a 3-way join.

`NO_GATHER` specifies that no `Gather` or `Gather Merge` node should appear above any of the targets, but it only constrains the planning of an individual subquery, and outer subquery levels can still use parallel query. For example, `NO_GATHER(inner_example@any_1)` precludes using a `Parallel Seq Scan` to access the `inner_example` table within the `any_1` subquery, but it does not prevent the planner from placing `SubPlan any_1` beneath a `Gather` or `Gather Merge` node. The following plan is compatible with `NO_GATHER(inner_example@any_1)`, but not with `NO_GATHER(outer_example)`:

    Finalize Aggregate
      ->  Gather
            ->  Partial Aggregate
                  ->  Parallel Seq Scan on outer_example
                        Filter: (something = (hashed SubPlan any_1).col1)
                        SubPlan any_1
                          ->  Seq Scan on inner_example
                                Filter: (something_else > 100)

Here is the reverse case, that is, a plan compatible with `NO_GATHER(outer_example)` but not with `NO_GATHER(inner_example@any_1)`:

    Aggregate
      ->  Seq Scan on outer_example
            Filter: (something = (hashed SubPlan any_1).col1)
            SubPlan any_1
              ->  Gather
                    -> Parallel Seq Scan on inner_example
                         Filter: (something_else > 100)

## Advice Feedback

`EXPLAIN` provides feedback on whether supplied advice was successfully applied to the query in the form of a comment on each piece of supplied advice. For example:

    SET pg_plan_advice.advice = 'hash_join(f g) join_order(f g) index_scan(f no_such_index)';
    EXPLAIN (COSTS OFF)
        SELECT * FROM jo_fact f
        LEFT JOIN jo_dim1 d1 ON f.dim1_id = d1.id
        LEFT JOIN jo_dim2 d2 ON f.dim2_id = d2.id
        WHERE val1 = 1 AND val2 = 1;
                                QUERY PLAN
    -------------------------------------------------------------------
     Hash Join
       Hash Cond: ((d1.id = f.dim1_id) AND (d2.id = f.dim2_id))
       ->  Nested Loop
             ->  Seq Scan on jo_dim2 d2
                   Filter: (val2 = 1)
             ->  Materialize
                   ->  Seq Scan on jo_dim1 d1
                         Filter: (val1 = 1)
       ->  Hash
             ->  Seq Scan on jo_fact f
     Supplied Plan Advice:
       INDEX_SCAN(f no_such_index) /* matched, inapplicable, failed */
       HASH_JOIN(f) /* matched */
       HASH_JOIN(g) /* not matched */
       JOIN_ORDER(f g) /* partially matched */

For this query, `f` is a valid advice target, but `g` is not. Therefore, the request to place `f` on the inner side of a hash join is listed as `matched`, but the request to place `g` on the inner side of a hash join is listed as `not matched`. The `JOIN_ORDER` advice tag involves one valid target and one invalid target, and so is listed as `partially matched`. Note that `HASH_JOIN(f g)` is actually a request for two logically separate behaviors, so in the feedback it is decomposed into `HASH_JOIN(f)` and `HASH_JOIN(g)`. By contrast, `JOIN_ORDER(f g)` is a single request and appears as-is.

Advice feedback can include any of the following:

- `matched` means that all of the specified advice targets were observed together during query planning, at a time at which the advice could be enforced.

- `partially matched` means that some but not all of the specified advice targets were observed during query planning, or all of the advice targets were observed but not together. For example, this may happen if all the targets of `JOIN_ORDER` advice individually match the query, but the proposed join order is not legal.

- `not matched` means that none of the specified advice targets were observed during query planning. This may happen if the advice simply doesn't match the query, or it may occur if the relevant portion of the query was not planned, perhaps because it was gated by a condition that was simplified to constant false.

- `inapplicable` means that the advice tag could not be applied to the advice targets for some reason. For example, this will happen if the use of a nonexistent index is requested, or if an attempt is made to control semijoin uniqueness for a non-semijoin.

- `conflicting` means that two or more pieces of advice request incompatible behaviors. For example, if you advise a sequential scan and an index scan for the same table, both requests will be flagged as conflicting. This also commonly happens if join method advice or semijoin uniqueness advice implies a join order incompatible with the one explicitly specified; see pgplanadvice-join-method.

- `failed` means that the query plan does not comply with the advice. This only occurs for entries that are also shown as `matched`. It frequently occurs for entries that are also marked as `conflicting` or `inapplicable`. However, it can also occur when the advice is valid insofar as `pg_plan_advice` is able to determine, but the planner is not able to construct a legal plan that can comply with the advice. It is important to note that the sanity checks performed by `pg_plan_advice` are fairly superficial and focused mostly on looking for logical inconsistencies in the advice string; only the planner knows what will actually work.

All advice should be marked as exactly one of `matched`, `partially matched`, or `not matched`.

## Configuration Parameters

`pg_plan_advice.advice` (`string`)

- `pg_plan_advice.advice` is an advice string to be used during query planning.

`pg_plan_advice.always_explain_supplied_advice` (`boolean`)

- `pg_plan_advice.always_explain_supplied_advice` causes `EXPLAIN` to always show any supplied advice and the associated advice feedback. The default value is `true`. If set to `false`, this information will be displayed only when `EXPLAIN (PLAN_ADVICE)` is used.

`pg_plan_advice.always_store_advice_details` (`boolean`)

- `pg_plan_advice.always_store_advice_details` allows `EXPLAIN` to show details related to plan advice even when prepared queries are used. The default value is `false`. When planning a prepared query, it is not possible to know whether `EXPLAIN` will later be used, so by default, to reduce overhead, `pg_plan_advice` will not generate plan advice or feedback on supplied advice. This means that if `EXPLAIN EXECUTE` is used on the prepared query, it will not be able to show this information. Changing this setting to `true` avoids this problem, but adds additional overhead. It is probably a good idea to enable this option only in sessions where it is needed, rather than on a system-wide basis.

`pg_plan_advice.feedback_warnings` (`boolean`)

- When set to true, `pg_plan_advice.feedback_warnings` emits a warning whenever supplied plan advice is not successfully enforced. The default value is `false`.

`pg_plan_advice.trace_mask` (`boolean`)

- When `pg_plan_advice.trace_mask` is `true`, `pg_plan_advice` will print messages during query planning each time that `pg_plan_advice` alters the mask of allowable query plan types in response to supplied plan advice. The default value is `false`. The messages printed by this setting are not expected to be useful except for purposes of debugging this module.

## Limitations

It is currently not possible to control any aspect of the planner's behavior with respect to aggregation. This includes both whether aggregates are computed by sorting or hashing, and also whether strategies such as eager aggregation or partitionwise aggregation are used.

It also is currently not possible to control any aspect of the planner's behavior with respect to set operations such as `UNION` or `INTERSECT`.

As discussed above under How It Works, the use of plan advice can only affect which plan the planner chooses from among those it believes to be viable. It can never force the choice of a plan which the planner refused to consider in the first place.

## Author

Robert Haas rhaas@postgresql.org