List of extensions that are verified to be working with Citus, and some
special cases that needs attention. Thanks to the efforts of @emelsimsek
, @m3hm3t , @alperkocatas , @eaydingol
Enhance security by addressing a code scanning alert and refactoring the
background worker setup code for better maintainability and clarity.
---------
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
fixes#8110
This patch updates the `normalize.sed` script used in pg18 psql
regression tests:
- Replaces the headings “List of tables”, “List of indexes”, and “List
of sequences” with a single, uniform heading: “List of relations”.
fixes#8105
This change lets `FindReferencedTableColumn()` correctly resolve columns
through a CTE even when the expression comes from an outer query level
(`varlevelsup > 0`, `skipOuterVars = false`). Before, we hit an
`Assert(skipOuterVars)` in this path.
**Problem**
* Hitting a CTE after walking outer Vars triggered
`Assert(skipOuterVars)`.
* Cause: we modified `parentQueryList` in place and didn’t rebuild the
correct parent chain before recursing into the CTE, so the path was
considered unsafe.
**Fix**
* Remove the `Assert(skipOuterVars)` in the `RTE_CTE` branch.
* Find the CTE’s owning level via `ctelevelsup` and compute
`cteParentListIndex`.
* Rebuild a private parent list for recursion: `list_copy` →
`list_truncate` → `lappend(current query)`.
* Add a bounds check before indexing the CTE’s `targetList`.
**Why it works**
```diff
-parentQueryList = lappend(parentQueryList, query);
-FindReferencedTableColumn(targetEntry->expr, parentQueryList,
- cteQuery, column, rteContainingReferencedColumn,
- skipOuterVars);
+ /* hand a private, bounded parent list to the recursion */
+ List *newParent = list_copy(parentQueryList);
+ newParent = list_truncate(newParent, cteParentListIndex + 1);
+ newParent = lappend(newParent, query);
+
+ FindReferencedTableColumn(targetEntry->expr,
+ newParent,
+ cteQuery,
+ column,
+ rteContainingReferencedColumn,
+ skipOuterVars);
+}
```
**Before:** We changed `parentQueryList` in place (`parentQueryList =
lappend(...)`) and didn’t trim it to the CTE’s owner level.
**After:** We copy the list, trim it to the CTE’s owner level, then
append the current query. This keeps the parent list accurate for the
current recursion and safe when following outer Vars.
**Example: Nested subquery referencing the CTE (two levels down)**
```
WITH c AS MATERIALIZED (SELECT user_id FROM raw_events_first)
SELECT 1
FROM raw_events_first t
WHERE EXISTS (
SELECT 1
FROM (SELECT user_id FROM c) c2
WHERE c2.user_id = t.user_id
);
```
Levels:
Q0 = top SELECT
Q1 = EXISTS subquery
Q2 = inner (SELECT user_id FROM c)
When resolving c2.user_id inside Q2:
- parentQueryList is [Q0, Q1, Q2].
- `ctelevelsup`: 2
`cteParentListIndex = length(parentQueryList) - ctelevelsup - 1`
- Recurse into the CTE’s query with [Q0, Q2].
**Tests (added in `multi_insert_select`)**
* **T1:** Correlated subquery that references a CTE (one level down)
Verifies that resolving through `RTE_CTE` after following an outer `Var`
succeeds, row count matches source table.
* **T2:** Nested subquery that references a CTE (two levels down)
Exercises deeper recursion and confirms identical to T1.
* **T3:** Scalar subquery in a target list that reads from the outer CTE
Checks expected row count and that no NULLs are inserted.
These tests cover the cases that previously hit `Assert(skipOuterVars)`
and confirm CTE references while following outer Vars.
DESCRIPTION: Fixes potential memory corruptions that could happen when
accessing pg_dist_background_task after a Citus downgrade is followed by
a Citus upgrade.
In case of Citus downgrade and further upgrade an undefined behavior may
be encountered. The reason is that Citus hardcoded the number of columns
in the extension's tables, but in case of downgrade and following update
some of these tables can have more columns, and some of them can be
marked as dropped.
This PR fixes all such tables using the approach introduced in #7950,
which solved the problem for the pg_dist_partition table.
See #7515 for a more thorough explanation.
---------
Co-authored-by: Karina Litskevich <litskevichkarina@gmail.com>
Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
DESCRIPTION: Introduce a new check to push down a query including union
and outer join to fix#8091 .
In "SafeToPushdownUnionSubquery", we check if the distribution column of
the outer relation is in the target list.
DESCRIPTION: Fixes potential memory corruptions that could happen when a
Citus downgrade is followed by a Citus upgrade.
In case of citus downgrade and further upgrade citus crash with core
dump.
The reason is that citus hardcoded number of columns in
pg_dist_partition table,
but in case of downgrade and following update table can have more
columns, and
some of then can be marked as dropped.
Patch suggest decision for this problem with using
tupleDescriptor->nattrs(postgres internal approach).
Fixes#7933.
---------
Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
We never update an older version of a SQL object for consistency across
release tags, so this commit moves "DROP FUNCTION .." for the older
version of "pg_catalog.worker_last_saved_explain_analyze();" to the
appropriate migration script.
See https://github.com/citusdata/citus/pull/8017.
DESCRIPTION: Fixed a bug in EXPLAIN ANALYZE to prevent unintended (duplicate) execution of the (sub)plans during the explain phase.
Fixes#4212
### 🐞 Bug #4212 : Redundant (Subplan) Execution in `EXPLAIN ANALYZE`
codepath
#### 🔍 Background
In the standard PostgreSQL execution path, `ExplainOnePlan()` is
responsible for two distinct operations depending on whether `EXPLAIN
ANALYZE` is requested:
1. **Execute the plan**
```c
if (es->analyze)
ExecutorRun(queryDesc, direction, 0L, true);
```
2. **Print the plan tree**
```c
ExplainPrintPlan(es, queryDesc);
```
When printing the plan, the executor should **not run the plan again**.
Execution is only expected to happen once—at the top level when
`es->analyze = true`.
---
#### ⚠️ Issue in Citus
In the Citus implementation of `CustomScanMethods.ExplainCustomScan =
CitusExplainScan`, which is a custom scan explain callback function used
to print explain information of a Citus plan incorrectly performs
**redundant execution** inside the explain path of `ExplainPrintPlan()`
```c
ExplainOnePlan()
ExplainPrintPlan()
ExplainNode()
CitusExplainScan()
if (distributedPlan->subPlanList != NIL)
{
ExplainSubPlans(distributedPlan, es);
{
PlannedStmt *plan = subPlan->plan;
ExplainOnePlan(plan, ...); // ⚠️ May re-execute subplan if es->analyze is true
}
}
```
This causes the subplans to be **executed again**, even though they have
already been executed during the top-level plan execution. This behavior
violates the expectation in PostgreSQL where `EXPLAIN ANALYZE` should
**execute each node exactly once** for analysis.
---
#### ✅ Fix (proposed)
Save the output of Subplans during `ExecuteSubPlans()`, and later use it
in `ExplainSubPlans()`
fixes#8072fixes#8055706054b11b
before fix
when try to create cluster with assert on
`citus_dev make test1 --destroy`
```
TRAP: failed Assert("HaveRegisteredOrActiveSnapshot()"), File: "heapam.c", Line: 232, PID: 75572
postgres: citus citus [local] SELECT(ExceptionalCondition+0x6e)[0x5585e16123e6]
postgres: citus citus [local] SELECT(heap_insert+0x220)[0x5585e10709af]
postgres: citus citus [local] SELECT(simple_heap_insert+0x33)[0x5585e1071a20]
postgres: citus citus [local] SELECT(CatalogTupleInsert+0x32)[0x5585e1135843]
/home/citus/.pgenv/pgsql-18beta2/lib/citus.so(+0x11e0aa)[0x7fa26f1ca0aa]
/home/citus/.pgenv/pgsql-18beta2/lib/citus.so(+0x11b607)[0x7fa26f1c7607]
/home/citus/.pgenv/pgsql-18beta2/lib/citus.so(+0x11bf25)[0x7fa26f1c7f25]
/home/citus/.pgenv/pgsql-18beta2/lib/citus.so(+0x11d4e2)[0x7fa26f1c94e2]
postgres: citus citus [local] SELECT(+0x1c267d)[0x5585e10e967d]
postgres: citus citus [local] SELECT(+0x1c6ba0)[0x5585e10edba0]
postgres: citus citus [local] SELECT(+0x1c7b80)[0x5585e10eeb80]
postgres: citus citus [local] SELECT(CommitTransactionCommand+0xd)[0x5585e10eef0a]
postgres: citus citus [local] SELECT(+0x575b3d)[0x5585e149cb3d]
postgres: citus citus [local] SELECT(+0x5788ce)[0x5585e149f8ce]
postgres: citus citus [local] SELECT(PostgresMain+0xae7)[0x5585e14a2088]
postgres: citus citus [local] SELECT(BackendMain+0x51)[0x5585e149ab36]
postgres: citus citus [local] SELECT(postmaster_child_launch+0x101)[0x5585e13d6b32]
postgres: citus citus [local] SELECT(+0x4b273f)[0x5585e13d973f]
postgres: citus citus [local] SELECT(+0x4b49f3)[0x5585e13db9f3]
postgres: citus citus [local] SELECT(PostmasterMain+0x1089)[0x5585e13dcee2]
postgres: citus citus [local] SELECT(main+0x1d7)[0x5585e12e3428]
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7fa271421d90]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7fa271421e40]
```
Bumps [black](https://github.com/psf/black) from 23.11.0 to 24.3.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/psf/black/releases">black's
releases</a>.</em></p>
<blockquote>
<h2>24.3.0</h2>
<h3>Highlights</h3>
<p>This release is a milestone: it fixes Black's first CVE security
vulnerability. If you
run Black on untrusted input, or if you habitually put thousands of
leading tab
characters in your docstrings, you are strongly encouraged to upgrade
immediately to fix
<a
href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-21503">CVE-2024-21503</a>.</p>
<p>This release also fixes a bug in Black's AST safety check that
allowed Black to make
incorrect changes to certain f-strings that are valid in Python 3.12 and
higher.</p>
<h3>Stable style</h3>
<ul>
<li>Don't move comments along with delimiters, which could cause crashes
(<a
href="https://redirect.github.com/psf/black/issues/4248">#4248</a>)</li>
<li>Strengthen AST safety check to catch more unsafe changes to strings.
Previous versions
of Black would incorrectly format the contents of certain unusual
f-strings containing
nested strings with the same quote type. Now, Black will crash on such
strings until
support for the new f-string syntax is implemented. (<a
href="https://redirect.github.com/psf/black/issues/4270">#4270</a>)</li>
<li>Fix a bug where line-ranges exceeding the last code line would not
work as expected
(<a
href="https://redirect.github.com/psf/black/issues/4273">#4273</a>)</li>
</ul>
<h3>Performance</h3>
<ul>
<li>Fix catastrophic performance on docstrings that contain large
numbers of leading tab
characters. This fixes
<a
href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-21503">CVE-2024-21503</a>.
(<a
href="https://redirect.github.com/psf/black/issues/4278">#4278</a>)</li>
</ul>
<h3>Documentation</h3>
<ul>
<li>Note what happens when <code>--check</code> is used with
<code>--quiet</code> (<a
href="https://redirect.github.com/psf/black/issues/4236">#4236</a>)</li>
</ul>
<h2>24.2.0</h2>
<h3>Stable style</h3>
<ul>
<li>Fixed a bug where comments where mistakenly removed along with
redundant parentheses
(<a
href="https://redirect.github.com/psf/black/issues/4218">#4218</a>)</li>
</ul>
<h3>Preview style</h3>
<ul>
<li>Move the <code>hug_parens_with_braces_and_square_brackets</code>
feature to the unstable style
due to an outstanding crash and proposed formatting tweaks (<a
href="https://redirect.github.com/psf/black/issues/4198">#4198</a>)</li>
<li>Fixed a bug where base expressions caused inconsistent formatting of
** in tenary
expression (<a
href="https://redirect.github.com/psf/black/issues/4154">#4154</a>)</li>
<li>Checking for newline before adding one on docstring that is almost
at the line limit
(<a
href="https://redirect.github.com/psf/black/issues/4185">#4185</a>)</li>
<li>Remove redundant parentheses in <code>case</code> statement
<code>if</code> guards (<a
href="https://redirect.github.com/psf/black/issues/4214">#4214</a>).</li>
</ul>
<h3>Configuration</h3>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/psf/black/blob/main/CHANGES.md">black's
changelog</a>.</em></p>
<blockquote>
<h2>24.3.0</h2>
<h3>Highlights</h3>
<p>This release is a milestone: it fixes Black's first CVE security
vulnerability. If you
run Black on untrusted input, or if you habitually put thousands of
leading tab
characters in your docstrings, you are strongly encouraged to upgrade
immediately to fix
<a
href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-21503">CVE-2024-21503</a>.</p>
<p>This release also fixes a bug in Black's AST safety check that
allowed Black to make
incorrect changes to certain f-strings that are valid in Python 3.12 and
higher.</p>
<h3>Stable style</h3>
<ul>
<li>Don't move comments along with delimiters, which could cause crashes
(<a
href="https://redirect.github.com/psf/black/issues/4248">#4248</a>)</li>
<li>Strengthen AST safety check to catch more unsafe changes to strings.
Previous versions
of Black would incorrectly format the contents of certain unusual
f-strings containing
nested strings with the same quote type. Now, Black will crash on such
strings until
support for the new f-string syntax is implemented. (<a
href="https://redirect.github.com/psf/black/issues/4270">#4270</a>)</li>
<li>Fix a bug where line-ranges exceeding the last code line would not
work as expected
(<a
href="https://redirect.github.com/psf/black/issues/4273">#4273</a>)</li>
</ul>
<h3>Performance</h3>
<ul>
<li>Fix catastrophic performance on docstrings that contain large
numbers of leading tab
characters. This fixes
<a
href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-21503">CVE-2024-21503</a>.
(<a
href="https://redirect.github.com/psf/black/issues/4278">#4278</a>)</li>
</ul>
<h3>Documentation</h3>
<ul>
<li>Note what happens when <code>--check</code> is used with
<code>--quiet</code> (<a
href="https://redirect.github.com/psf/black/issues/4236">#4236</a>)</li>
</ul>
<h2>24.2.0</h2>
<h3>Stable style</h3>
<ul>
<li>Fixed a bug where comments where mistakenly removed along with
redundant parentheses
(<a
href="https://redirect.github.com/psf/black/issues/4218">#4218</a>)</li>
</ul>
<h3>Preview style</h3>
<ul>
<li>Move the <code>hug_parens_with_braces_and_square_brackets</code>
feature to the unstable style
due to an outstanding crash and proposed formatting tweaks (<a
href="https://redirect.github.com/psf/black/issues/4198">#4198</a>)</li>
<li>Fixed a bug where base expressions caused inconsistent formatting of
** in tenary
expression (<a
href="https://redirect.github.com/psf/black/issues/4154">#4154</a>)</li>
<li>Checking for newline before adding one on docstring that is almost
at the line limit
(<a
href="https://redirect.github.com/psf/black/issues/4185">#4185</a>)</li>
<li>Remove redundant parentheses in <code>case</code> statement
<code>if</code> guards (<a
href="https://redirect.github.com/psf/black/issues/4214">#4214</a>).</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="552baf8229"><code>552baf8</code></a>
Prepare release 24.3.0 (<a
href="https://redirect.github.com/psf/black/issues/4279">#4279</a>)</li>
<li><a
href="f000936726"><code>f000936</code></a>
Fix catastrophic performance in lines_with_leading_tabs_expanded() (<a
href="https://redirect.github.com/psf/black/issues/4278">#4278</a>)</li>
<li><a
href="7b5a657285"><code>7b5a657</code></a>
Fix --line-ranges behavior when ranges are at EOF (<a
href="https://redirect.github.com/psf/black/issues/4273">#4273</a>)</li>
<li><a
href="1abcffc818"><code>1abcffc</code></a>
Use regex where we ignore case on windows (<a
href="https://redirect.github.com/psf/black/issues/4252">#4252</a>)</li>
<li><a
href="719e67462c"><code>719e674</code></a>
Fix 4227: Improve documentation for --quiet --check (<a
href="https://redirect.github.com/psf/black/issues/4236">#4236</a>)</li>
<li><a
href="e5510afc06"><code>e5510af</code></a>
update plugin url for Thonny (<a
href="https://redirect.github.com/psf/black/issues/4259">#4259</a>)</li>
<li><a
href="6af7d11096"><code>6af7d11</code></a>
Fix AST safety check false negative (<a
href="https://redirect.github.com/psf/black/issues/4270">#4270</a>)</li>
<li><a
href="f03ee113c9"><code>f03ee11</code></a>
Ensure <code>blib2to3.pygram</code> is initialized before use (<a
href="https://redirect.github.com/psf/black/issues/4224">#4224</a>)</li>
<li><a
href="e4bfedbec2"><code>e4bfedb</code></a>
fix: Don't move comments while splitting delimiters (<a
href="https://redirect.github.com/psf/black/issues/4248">#4248</a>)</li>
<li><a
href="d0287e1f75"><code>d0287e1</code></a>
Make trailing comma logic more concise (<a
href="https://redirect.github.com/psf/black/issues/4202">#4202</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/psf/black/compare/23.11.0...24.3.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/citusdata/citus/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
The assert on the number of shards incorrectly used the value of
citus.shard_replication_factor; it should check the table's metadata
to determine the replication factor of its data, and not assume it is
the current GUC value.
Commit 245a62df3e included an assertion on a struct field that is
in PG16+, without PG_VERSION_NUM check. This commit removes the
offending line of code. The same assertion is present later in the
function with the PG_VERSION_NUM check, so the offending line of code is
redundant.
DESCRIPTION: Fixes problematic UPDATE statements with indirection and array/jsonb subscripting with more than one field.
Fixes#4092, #7674 and #5621. Issues #7674 and #4092 involve an UPDATE with out of order columns and a sublink (SELECT) in the source, e.g. `UPDATE T SET (col3, col1, col4) = (SELECT 3, 1, 4)` where an incorrect value could get written to a column because query deparsing generated an incorrect SQL statement. To address this the fix adds an additional
check to `ruleutils` to ensure that the target list of an UPDATE statement is in an order so that deparsing can be done safely. It is needed when the source of the UPDATE has a sublink, because Postgres `rewrite` will have put the target list in attribute order, but for deparsing to produce a correct SQL text the target list needs to be in order of the references (or `paramids`) to the target list of the sublink(s). Issue #5621 involves an UPDATE with array/jsonb subscripting that can behave incorrectly with more than one field, again because Citus query deparsing is receiving a post-`rewrite` query tree. The fix also adds a
check to `ruleutils` to enable correct query deparsing of the UPDATE.
---------
Co-authored-by: Ibrahim Halatci <ihalatci@gmail.com>
Co-authored-by: Colm McHugh <colm.mchugh@gmail.com>
DESCRIPTION: Avoid query deparse and planning of shard query in local execution. Adds citus.enable_local_execution_local_plan GUC to allow avoiding unnecessary query deparsing to improve performance of fast-path queries targeting local shards.
If a fast path query resolves to a shard that is local to the node planning the query, a shortcut can be taken so that the OID of the shard is plugged into the parse tree, which is then planned by Postgres. In `local_executor.c` the task uses that plan instead of parsing and planning a shard query. How this is done: The fast path planner identifies if the shortcut is possible, and then the distributed planner checks, using `CheckAndBuildDelayedFastPathPlan()`, if a local plan can be generated or if the shard query should be generated.
This optimization is controlled by a GUC `citus.enable_local_execution_local_plan` which is on by default. A new
regress test `local_execution_local_plan` tests both row-sharding and schema sharding. Negative tests are added to
`local_shard_execution_dropped_column` to verify that the optimization is not taken when the shard is local but there is a difference between the shard and distributed table because of a dropped column.
Bumps [black](https://github.com/psf/black) from 24.2.0 to 24.3.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/psf/black/releases">black's
releases</a>.</em></p>
<blockquote>
<h2>24.3.0</h2>
<h3>Highlights</h3>
<p>This release is a milestone: it fixes Black's first CVE security
vulnerability. If you
run Black on untrusted input, or if you habitually put thousands of
leading tab
characters in your docstrings, you are strongly encouraged to upgrade
immediately to fix
<a
href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-21503">CVE-2024-21503</a>.</p>
<p>This release also fixes a bug in Black's AST safety check that
allowed Black to make
incorrect changes to certain f-strings that are valid in Python 3.12 and
higher.</p>
<h3>Stable style</h3>
<ul>
<li>Don't move comments along with delimiters, which could cause crashes
(<a
href="https://redirect.github.com/psf/black/issues/4248">#4248</a>)</li>
<li>Strengthen AST safety check to catch more unsafe changes to strings.
Previous versions
of Black would incorrectly format the contents of certain unusual
f-strings containing
nested strings with the same quote type. Now, Black will crash on such
strings until
support for the new f-string syntax is implemented. (<a
href="https://redirect.github.com/psf/black/issues/4270">#4270</a>)</li>
<li>Fix a bug where line-ranges exceeding the last code line would not
work as expected
(<a
href="https://redirect.github.com/psf/black/issues/4273">#4273</a>)</li>
</ul>
<h3>Performance</h3>
<ul>
<li>Fix catastrophic performance on docstrings that contain large
numbers of leading tab
characters. This fixes
<a
href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-21503">CVE-2024-21503</a>.
(<a
href="https://redirect.github.com/psf/black/issues/4278">#4278</a>)</li>
</ul>
<h3>Documentation</h3>
<ul>
<li>Note what happens when <code>--check</code> is used with
<code>--quiet</code> (<a
href="https://redirect.github.com/psf/black/issues/4236">#4236</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/psf/black/blob/main/CHANGES.md">black's
changelog</a>.</em></p>
<blockquote>
<h2>24.3.0</h2>
<h3>Highlights</h3>
<p>This release is a milestone: it fixes Black's first CVE security
vulnerability. If you
run Black on untrusted input, or if you habitually put thousands of
leading tab
characters in your docstrings, you are strongly encouraged to upgrade
immediately to fix
<a
href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-21503">CVE-2024-21503</a>.</p>
<p>This release also fixes a bug in Black's AST safety check that
allowed Black to make
incorrect changes to certain f-strings that are valid in Python 3.12 and
higher.</p>
<h3>Stable style</h3>
<ul>
<li>Don't move comments along with delimiters, which could cause crashes
(<a
href="https://redirect.github.com/psf/black/issues/4248">#4248</a>)</li>
<li>Strengthen AST safety check to catch more unsafe changes to strings.
Previous versions
of Black would incorrectly format the contents of certain unusual
f-strings containing
nested strings with the same quote type. Now, Black will crash on such
strings until
support for the new f-string syntax is implemented. (<a
href="https://redirect.github.com/psf/black/issues/4270">#4270</a>)</li>
<li>Fix a bug where line-ranges exceeding the last code line would not
work as expected
(<a
href="https://redirect.github.com/psf/black/issues/4273">#4273</a>)</li>
</ul>
<h3>Performance</h3>
<ul>
<li>Fix catastrophic performance on docstrings that contain large
numbers of leading tab
characters. This fixes
<a
href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-21503">CVE-2024-21503</a>.
(<a
href="https://redirect.github.com/psf/black/issues/4278">#4278</a>)</li>
</ul>
<h3>Documentation</h3>
<ul>
<li>Note what happens when <code>--check</code> is used with
<code>--quiet</code> (<a
href="https://redirect.github.com/psf/black/issues/4236">#4236</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="552baf8229"><code>552baf8</code></a>
Prepare release 24.3.0 (<a
href="https://redirect.github.com/psf/black/issues/4279">#4279</a>)</li>
<li><a
href="f000936726"><code>f000936</code></a>
Fix catastrophic performance in lines_with_leading_tabs_expanded() (<a
href="https://redirect.github.com/psf/black/issues/4278">#4278</a>)</li>
<li><a
href="7b5a657285"><code>7b5a657</code></a>
Fix --line-ranges behavior when ranges are at EOF (<a
href="https://redirect.github.com/psf/black/issues/4273">#4273</a>)</li>
<li><a
href="1abcffc818"><code>1abcffc</code></a>
Use regex where we ignore case on windows (<a
href="https://redirect.github.com/psf/black/issues/4252">#4252</a>)</li>
<li><a
href="719e67462c"><code>719e674</code></a>
Fix 4227: Improve documentation for --quiet --check (<a
href="https://redirect.github.com/psf/black/issues/4236">#4236</a>)</li>
<li><a
href="e5510afc06"><code>e5510af</code></a>
update plugin url for Thonny (<a
href="https://redirect.github.com/psf/black/issues/4259">#4259</a>)</li>
<li><a
href="6af7d11096"><code>6af7d11</code></a>
Fix AST safety check false negative (<a
href="https://redirect.github.com/psf/black/issues/4270">#4270</a>)</li>
<li><a
href="f03ee113c9"><code>f03ee11</code></a>
Ensure <code>blib2to3.pygram</code> is initialized before use (<a
href="https://redirect.github.com/psf/black/issues/4224">#4224</a>)</li>
<li><a
href="e4bfedbec2"><code>e4bfedb</code></a>
fix: Don't move comments while splitting delimiters (<a
href="https://redirect.github.com/psf/black/issues/4248">#4248</a>)</li>
<li><a
href="d0287e1f75"><code>d0287e1</code></a>
Make trailing comma logic more concise (<a
href="https://redirect.github.com/psf/black/issues/4202">#4202</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/psf/black/compare/24.2.0...24.3.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/citusdata/citus/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [black](https://github.com/psf/black) from 24.2.0 to 24.3.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/psf/black/releases">black's
releases</a>.</em></p>
<blockquote>
<h2>24.3.0</h2>
<h3>Highlights</h3>
<p>This release is a milestone: it fixes Black's first CVE security
vulnerability. If you
run Black on untrusted input, or if you habitually put thousands of
leading tab
characters in your docstrings, you are strongly encouraged to upgrade
immediately to fix
<a
href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-21503">CVE-2024-21503</a>.</p>
<p>This release also fixes a bug in Black's AST safety check that
allowed Black to make
incorrect changes to certain f-strings that are valid in Python 3.12 and
higher.</p>
<h3>Stable style</h3>
<ul>
<li>Don't move comments along with delimiters, which could cause crashes
(<a
href="https://redirect.github.com/psf/black/issues/4248">#4248</a>)</li>
<li>Strengthen AST safety check to catch more unsafe changes to strings.
Previous versions
of Black would incorrectly format the contents of certain unusual
f-strings containing
nested strings with the same quote type. Now, Black will crash on such
strings until
support for the new f-string syntax is implemented. (<a
href="https://redirect.github.com/psf/black/issues/4270">#4270</a>)</li>
<li>Fix a bug where line-ranges exceeding the last code line would not
work as expected
(<a
href="https://redirect.github.com/psf/black/issues/4273">#4273</a>)</li>
</ul>
<h3>Performance</h3>
<ul>
<li>Fix catastrophic performance on docstrings that contain large
numbers of leading tab
characters. This fixes
<a
href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-21503">CVE-2024-21503</a>.
(<a
href="https://redirect.github.com/psf/black/issues/4278">#4278</a>)</li>
</ul>
<h3>Documentation</h3>
<ul>
<li>Note what happens when <code>--check</code> is used with
<code>--quiet</code> (<a
href="https://redirect.github.com/psf/black/issues/4236">#4236</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/psf/black/blob/main/CHANGES.md">black's
changelog</a>.</em></p>
<blockquote>
<h2>24.3.0</h2>
<h3>Highlights</h3>
<p>This release is a milestone: it fixes Black's first CVE security
vulnerability. If you
run Black on untrusted input, or if you habitually put thousands of
leading tab
characters in your docstrings, you are strongly encouraged to upgrade
immediately to fix
<a
href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-21503">CVE-2024-21503</a>.</p>
<p>This release also fixes a bug in Black's AST safety check that
allowed Black to make
incorrect changes to certain f-strings that are valid in Python 3.12 and
higher.</p>
<h3>Stable style</h3>
<ul>
<li>Don't move comments along with delimiters, which could cause crashes
(<a
href="https://redirect.github.com/psf/black/issues/4248">#4248</a>)</li>
<li>Strengthen AST safety check to catch more unsafe changes to strings.
Previous versions
of Black would incorrectly format the contents of certain unusual
f-strings containing
nested strings with the same quote type. Now, Black will crash on such
strings until
support for the new f-string syntax is implemented. (<a
href="https://redirect.github.com/psf/black/issues/4270">#4270</a>)</li>
<li>Fix a bug where line-ranges exceeding the last code line would not
work as expected
(<a
href="https://redirect.github.com/psf/black/issues/4273">#4273</a>)</li>
</ul>
<h3>Performance</h3>
<ul>
<li>Fix catastrophic performance on docstrings that contain large
numbers of leading tab
characters. This fixes
<a
href="https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2024-21503">CVE-2024-21503</a>.
(<a
href="https://redirect.github.com/psf/black/issues/4278">#4278</a>)</li>
</ul>
<h3>Documentation</h3>
<ul>
<li>Note what happens when <code>--check</code> is used with
<code>--quiet</code> (<a
href="https://redirect.github.com/psf/black/issues/4236">#4236</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="552baf8229"><code>552baf8</code></a>
Prepare release 24.3.0 (<a
href="https://redirect.github.com/psf/black/issues/4279">#4279</a>)</li>
<li><a
href="f000936726"><code>f000936</code></a>
Fix catastrophic performance in lines_with_leading_tabs_expanded() (<a
href="https://redirect.github.com/psf/black/issues/4278">#4278</a>)</li>
<li><a
href="7b5a657285"><code>7b5a657</code></a>
Fix --line-ranges behavior when ranges are at EOF (<a
href="https://redirect.github.com/psf/black/issues/4273">#4273</a>)</li>
<li><a
href="1abcffc818"><code>1abcffc</code></a>
Use regex where we ignore case on windows (<a
href="https://redirect.github.com/psf/black/issues/4252">#4252</a>)</li>
<li><a
href="719e67462c"><code>719e674</code></a>
Fix 4227: Improve documentation for --quiet --check (<a
href="https://redirect.github.com/psf/black/issues/4236">#4236</a>)</li>
<li><a
href="e5510afc06"><code>e5510af</code></a>
update plugin url for Thonny (<a
href="https://redirect.github.com/psf/black/issues/4259">#4259</a>)</li>
<li><a
href="6af7d11096"><code>6af7d11</code></a>
Fix AST safety check false negative (<a
href="https://redirect.github.com/psf/black/issues/4270">#4270</a>)</li>
<li><a
href="f03ee113c9"><code>f03ee11</code></a>
Ensure <code>blib2to3.pygram</code> is initialized before use (<a
href="https://redirect.github.com/psf/black/issues/4224">#4224</a>)</li>
<li><a
href="e4bfedbec2"><code>e4bfedb</code></a>
fix: Don't move comments while splitting delimiters (<a
href="https://redirect.github.com/psf/black/issues/4248">#4248</a>)</li>
<li><a
href="d0287e1f75"><code>d0287e1</code></a>
Make trailing comma logic more concise (<a
href="https://redirect.github.com/psf/black/issues/4202">#4202</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/psf/black/compare/24.2.0...24.3.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/citusdata/citus/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [werkzeug](https://github.com/pallets/werkzeug) from 2.3.7 to
3.0.6.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/releases">werkzeug's
releases</a>.</em></p>
<blockquote>
<h2>3.0.6</h2>
<p>This is the Werkzeug 3.0.6 security fix release, which fixes security
issues but does not otherwise change behavior and should not result in
breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.6/">https://pypi.org/project/Werkzeug/3.0.6/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/stable/changes/#version-3-0-6">https://werkzeug.palletsprojects.com/en/stable/changes/#version-3-0-6</a></p>
<ul>
<li>Fix how <code>max_form_memory_size</code> is applied when parsing
large non-file fields. <a
href="https://github.com/advisories/GHSA-q34m-jh98-gwm2">GHSA-q34m-jh98-gwm2</a></li>
<li><code>safe_join</code> catches certain paths on Windows that were
not caught by <code>ntpath.isabs</code> on Python < 3.11. <a
href="https://github.com/advisories/GHSA-f9vj-2wh5-fj8j">GHSA-f9vj-2wh5-fj8j</a></li>
</ul>
<h2>3.0.5</h2>
<p>This is the Werkzeug 3.0.5 fix release, which fixes bugs but does not
otherwise change behavior and should not result in breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.5/">https://pypi.org/project/Werkzeug/3.0.5/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/stable/changes/#version-3-0-5">https://werkzeug.palletsprojects.com/en/stable/changes/#version-3-0-5</a>
Milestone: <a
href="https://github.com/pallets/werkzeug/milestone/37?closed=1">https://github.com/pallets/werkzeug/milestone/37?closed=1</a></p>
<ul>
<li>The Watchdog reloader ignores file closed no write events. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2945">#2945</a></li>
<li>Logging works with client addresses containing an IPv6 scope. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2952">#2952</a></li>
<li>Ignore invalid authorization parameters. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2955">#2955</a></li>
<li>Improve type annotation fore <code>SharedDataMiddleware</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2958">#2958</a></li>
<li>Compatibility with Python 3.13 when generating debugger pin and the
current UID does not have an associated name. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2957">#2957</a></li>
</ul>
<h2>3.0.4</h2>
<p>This is the Werkzeug 3.0.4 fix release, which fixes bugs but does not
otherwise change behavior and should not result in breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.4/">https://pypi.org/project/Werkzeug/3.0.4/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-4">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-4</a>
Milestone: <a
href="https://github.com/pallets/werkzeug/milestone/36?closed=1">https://github.com/pallets/werkzeug/milestone/36?closed=1</a></p>
<ul>
<li>Restore behavior where parsing
<code>multipart/x-www-form-urlencoded</code> data with
invalid UTF-8 bytes in the body results in no form data parsed rather
than a
413 error. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2930">#2930</a></li>
<li>Improve <code>parse_options_header</code> performance when parsing
unterminated
quoted string values. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2904">#2904</a></li>
<li>Debugger pin auth is synchronized across threads/processes when
tracking
failed entries. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2916">#2916</a></li>
<li>Dev server handles unexpected <code>SSLEOFError</code> due to issue
in Python < 3.13.
<a
href="https://redirect.github.com/pallets/werkzeug/issues/2926">#2926</a></li>
<li>Debugger pin auth works when the URL already contains a query
string.
<a
href="https://redirect.github.com/pallets/werkzeug/issues/2918">#2918</a></li>
</ul>
<h2>3.0.3</h2>
<p>This is the Werkzeug 3.0.3 security release, which fixes security
issues and bugs but does not otherwise change behavior and should not
result in breaking changes.</p>
<p>PyPI: <a
href="https://pypi.org/project/Werkzeug/3.0.3/">https://pypi.org/project/Werkzeug/3.0.3/</a>
Changes: <a
href="https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3">https://werkzeug.palletsprojects.com/en/3.0.x/changes/#version-3-0-3</a>
Milestone: <a
href="https://github.com/pallets/werkzeug/milestone/35?closed=1">https://github.com/pallets/werkzeug/milestone/35?closed=1</a></p>
<ul>
<li>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified hostname when running the dev
server, to make debugger requests. Additional hosts can be added by
using the debugger middleware directly. The debugger UI makes requests
using the full URL rather than only the path. GHSA-2g68-c3qc-8985</li>
<li>Make reloader more robust when <code>""</code> is in
<code>sys.path</code>. <a
href="https://redirect.github.com/pallets/werkzeug/issues/2823">#2823</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pallets/werkzeug/blob/main/CHANGES.rst">werkzeug's
changelog</a>.</em></p>
<blockquote>
<h2>Version 3.0.6</h2>
<p>Released 2024-10-25</p>
<ul>
<li>Fix how <code>max_form_memory_size</code> is applied when parsing
large non-file
fields. :ghsa:<code>q34m-jh98-gwm2</code></li>
<li><code>safe_join</code> catches certain paths on Windows that were
not caught by
<code>ntpath.isabs</code> on Python < 3.11.
:ghsa:<code>f9vj-2wh5-fj8j</code></li>
</ul>
<h2>Version 3.0.5</h2>
<p>Released 2024-10-24</p>
<ul>
<li>The Watchdog reloader ignores file closed no write events.
:issue:<code>2945</code></li>
<li>Logging works with client addresses containing an IPv6 scope
:issue:<code>2952</code></li>
<li>Ignore invalid authorization parameters.
:issue:<code>2955</code></li>
<li>Improve type annotation fore <code>SharedDataMiddleware</code>.
:issue:<code>2958</code></li>
<li>Compatibility with Python 3.13 when generating debugger pin and the
current
UID does not have an associated name. :issue:<code>2957</code></li>
</ul>
<h2>Version 3.0.4</h2>
<p>Released 2024-08-21</p>
<ul>
<li>Restore behavior where parsing
<code>multipart/x-www-form-urlencoded</code> data with
invalid UTF-8 bytes in the body results in no form data parsed rather
than a
413 error. :issue:<code>2930</code></li>
<li>Improve <code>parse_options_header</code> performance when parsing
unterminated
quoted string values. :issue:<code>2904</code></li>
<li>Debugger pin auth is synchronized across threads/processes when
tracking
failed entries. :issue:<code>2916</code></li>
<li>Dev server handles unexpected <code>SSLEOFError</code> due to issue
in Python < 3.13.
:issue:<code>2926</code></li>
<li>Debugger pin auth works when the URL already contains a query
string.
:issue:<code>2918</code></li>
</ul>
<h2>Version 3.0.3</h2>
<p>Released 2024-05-05</p>
<ul>
<li>Only allow <code>localhost</code>, <code>.localhost</code>,
<code>127.0.0.1</code>, or the specified
hostname when running the dev server, to make debugger requests.
Additional
hosts can be added by using the debugger middleware directly. The
debugger</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="5eaefc3996"><code>5eaefc3</code></a>
release version 3.0.6</li>
<li><a
href="2767bcb10a"><code>2767bcb</code></a>
Merge commit from fork</li>
<li><a
href="87cc78a25f"><code>87cc78a</code></a>
catch special absolute path on Windows Python < 3.11</li>
<li><a
href="50cfeebcb0"><code>50cfeeb</code></a>
Merge commit from fork</li>
<li><a
href="8760275afb"><code>8760275</code></a>
apply max_form_memory_size another level up in the parser</li>
<li><a
href="8d6a12e2af"><code>8d6a12e</code></a>
start version 3.0.6</li>
<li><a
href="a7b121abc7"><code>a7b121a</code></a>
release version 3.0.5 (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2961">#2961</a>)</li>
<li><a
href="9caf72ac06"><code>9caf72a</code></a>
release version 3.0.5</li>
<li><a
href="e28a2451e9"><code>e28a245</code></a>
catch OSError from getpass.getuser (<a
href="https://redirect.github.com/pallets/werkzeug/issues/2960">#2960</a>)</li>
<li><a
href="e6b4cce97e"><code>e6b4cce</code></a>
catch OSError from getpass.getuser</li>
<li>Additional commits viewable in <a
href="https://github.com/pallets/werkzeug/compare/2.3.7...3.0.6">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/citusdata/citus/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
DESCRIPTION: Automatically updates dynamic_library_path when CDC is
enabled
fix : #7715
According to the documentation and `pg_settings`, the context of the
`citus.enable_change_data_capture` parameter is user.
However, changing this parameter — even as a superuser — doesn't work as
expected: while the initial copy phase works correctly, subsequent
change events are not propagated.
This appears to be due to the fact that `dynamic_library_path` is only
updated to `$libdir/citus_decoders:$libdir` when the server is restarted
and the `_PG_init` function is invoked.
To address this, I added an `EnableChangeDataCaptureAssignHook` that
automatically updates `dynamic_library_path` at runtime when
`citus.enable_change_data_capture` is enabled, ensuring that the CDC
decoder libraries are properly loaded.
Note that `dynamic_library_path` is already a `superuser`-context
parameter in base PostgreSQL, so updating it from within the assign hook
should be safe and consistent with PostgreSQL’s configuration model.
If there’s any reason this approach might be problematic or if there’s a
preferred alternative, I’d appreciate any feedback.
cc. @jy-min
---------
Co-authored-by: Hanefi Onaldi <Hanefi.Onaldi@microsoft.com>
Co-authored-by: ibrahim halatci <ihalatci@gmail.com>
Fixes#8040
```
- Custom Scan (Citus Adaptive) (actual rows=0 loops=1)
+ Custom Scan (Citus Adaptive) (actual rows=0.00 loops=1)
```
Add a normalization rule to the pg_regress `normalize.sed` script that
strips any trailing decimal fraction from actual rows= counts (e.g.
turning `actual rows=0.00` into `actual rows=0`). This silences noise
diffs introduced by the new PostgreSQL 18 beta’s planner output.
commit b06bde5771
Fixes#8019
**Background / Problem**
- PostgreSQL 18 (commit
[a07e03f…](a07e03fd8f))
removed `heap_inplace_update()` and related helpers.
- Citus’ columnar writer relied on that API in
`UpdateStripeMetadataRow()` to patch the `columnar_stripe` catalog row
with the stripe file-offset, size, and row-count.
- Building the extension against PG 18 therefore failed at link-time
and, if stubbed out, left `file_offset = 0`, causing every insert to
abort with
`ERROR: attempted columnar write … to invalid logical offset: 0`
**Scope of This PR**
- Keep the fast-path on PG 12–17 (`heap_inplace_update()` unchanged).
- Switch to `CatalogTupleUpdate()` on PG 18+, matching core’s new
catalog-update API.
- Bump the lock level from `AccessShareLock` → `RowExclusiveLock` when
the normal heap-update path is taken.
- No behavioral changes for users on PG ≤ 17
Fixes#8020
PostgreSQL 18 introduces two new, *pseudo* rangetable‐entry kinds that
Citus’ downstream deparser must recognize:
1. **Pulled-up shard RTE clones** (`CITUS_RTE_SHARD` with `relid ==
InvalidOid`)
2. **Grouping-step RTE** (`RTE_GROUP`, alias `*GROUP*`, not actually in
the FROM clause)
Without special handling, Citus crashes or emits invalid SQL when
running against PG 18beta1:
* **`ERROR: could not open relation with OID 0`**
Citus was unconditionally calling `relation_open(rte->relid,…)` on
entries whose `relid` is 0.
* **`ERROR: missing FROM-clause entry for table "*GROUP*"`**
Citus’ `set_rtable_names()` assigned the synthetic `*GROUP*` alias but
never printed a matching FROM item.
This PR teaches Citus’ `ruleutils_18.c` to skip catalog lookups for RTEs
without valid OIDs and to suppress the grouping-RTE alias, restoring
compatibility with both PG 17 and PG 18.
---
## Background
* **Upstream commit
[[247dea8](247dea89f7)**
Introduced `RTE_GROUP` for the grouping step so that multiple subqueries
in `GROUP BY`/`HAVING` can be deduplicated and planned correctly.
* **Citus PR
[[#6428](https://github.com/citusdata/citus/pull/6428)](https://github.com/citusdata/citus/pull/6428)**
Added initial support for treating shard RTEs like real
relations—calling `relation_open()` to pick up renamed-column fixes.
Worked fine on PG 11–17, but PG 18’s pull-up logic clones those shard
RTEs with `relid=0`, leading to OID 0 crashes.
---
## Changes
1. **Guard `relation_open()`**
In `set_relation_column_names()`, only call `relation_open(rte->relid,
…)` when
```c
OidIsValid(rte->relid)
```
Prevents the “could not open relation with OID 0” crash on both
pulled-up shards and synthetic RTEs.
2. **Handle pulled-up shards** (`CITUS_RTE_SHARD` with `relid=0`)
Copy column names directly from `rte->eref->colnames` instead of hitting
the catalog.
3. **Handle grouping RTE** (`RTE_GROUP`)
* **In `set_relation_column_names()`**: fallback to
`rte->eref->colnames` for `RTE_GROUP`.
* **In `set_rtable_names()`**: explicitly assign
```c
refname = NULL; /* never show *GROUP* in FROM */
```
so that no `*GROUP*` alias is ever printed.
**Why this is required:**
PostgreSQL 18’s parser now represents the grouping step with a synthetic
RTE whose alias is always `*GROUP*`—and that RTE is **never** actually
listed in the `FROM` clause. If Citus’ deparser assigns and emits
`*GROUP*` as a table reference, the pushed-down SQL becomes:
```sql
SELECT *GROUP*.mygroupcol … -- but there is no “*GROUP*” in the FROM
list
```
Workers then fail:
```
ERROR: missing FROM-clause entry for table "*GROUP*"
```
By setting `refname = NULL` for `RTE_GROUP` in `set_rtable_names()`, the
deparser prints just the column name unqualified, exactly matching
upstream PG 18’s behavior and yielding valid SQL on the workers.
4. **Maintain existing behavior on PG 15–17**
* Shard RTEs *with* valid `relid` still open the catalog to pick up
renamed-column fixes.
* No impact on other RTE kinds or versions prior to PG 18.
---
eaydingol
ibrahim halatci
Mehmet YILMAZ
Karina
eaydingol
manaldush
Onur Tirtir
Teja Mupparti
dependabot[bot]
Colm
Cédric Villemain
Naisila Puka
SongYoungUk