The main issue lies in the following entries of PlannedStmt:
{
rtable
permInfos
}
Each rtable has an int perminfoindex, and its actual permission info
is obtained through the following:
permInfos[perminfoindex]
We had a crash because perminfoindexes were not updated in the
finalized planned statement after distributed planner hook.
So, basically, everywhere we set a query's or planned statement's
rtable entry, we need to set the rteperminfos/permInfos accordingly.
Relevant PG commits:
a61b1f7482
a61b1f74823c9c4f79c95226a461f1e7a367764b
b803b7d132
b803b7d132e3505ab77c29acf91f3d1caa298f95
If we allow to adjust partitioning, we get a crash when accessing
amcostestimate of partitioned indexes, because amcostestimate is NULL
for them. The following PG commit is the culprit
3c569049b7
3c569049b7b502bb4952483d19ce622ff0af5fd6
Previously, partitioned indexes would just be ignored.
Now, they are added in the list. However get_relation_info expects
the tables which have partitioned indexes to have the inh flag set
properly. AdjustPartitioningForDistributedPlanning plays with that
flag, hence we don't get the desired behaviour.
The hook is simply removing all partitioned indexes from the list.
Relevant PG commits:
Introduce SYSTEM_USER
0823d061b0
0823d061b0b7f1e20fbfd48bef3c2e093493dbd4
Rename shadowed local variables
2d0bbedda7
2d0bbedda7c4fbf7092c39b0c06b56cb238e15d7
Fix EXPLAIN of SEARCH BREADTH FIRST with a constant initial value
d57534740b
d57534740b7682c2816ce6c54318521787fcfacf
Add repalloc0 and repalloc0_array
b4b7ce8061
b4b7ce8061d34cea2b4915c41403b2a74d5fde0e
Improve ruleutils' printout of LATERAL references within subplans
adaf34241a
adaf34241acd83afaa45a8b614b6484a285da847
Switch SQLValueFunction on "name" to use COERCE_SQL_SYNTAX
fb32748e32
fb32748e32e2b6b2fcb32220980b93d5436f855e
Replace SQLValueFunction by COERCE_SQL_SYNTAX
f193883fc9
f193883fc9cebe8fa20359b0797832837a788112
Fix under-parenthesized display of AT TIME ZONE constructs
1dd6700f44
1dd6700f44d2e14616d464a5f490552b7036ad3f
Update copyright for 2023
c8e1ba736b
c8e1ba736b2b9e8c98d37a5b77c4ed31baf94147
Print the correct aliases for DML target tables in ruleutils
393430f575
393430f57544dfd550135e0191cc91139926b682
SQL/JSON: add standard JSON constructor functions
7081ac46ac
7081ac46ace8c459966174400b53418683c9fe5c
Fix inconsistencies and style issues in new SQL/JSON code
60966f56c3
60966f56c3e4db9f9bdff485371ef3a50d53431f
SQL/JSON: support the IS JSON predicate
6ee30209a6
6ee30209a6f161d0a267a33f090c70c579c87c00
Add ruleutils support for decompiling MERGE commands.
41e2c52fd6
41e2c52fd6ebed6eff4184f68048813cc8886ec1
Add back SQLValueFunction for SQL keywords
d8c3106bb6
d8c3106bb60e4f87be595f241e173ba3c2b7aa2c
Pre-beta mechanical code beautification
0245f8db36
0245f8db36f375326c2bae0c3420d3c77714e72d
Tradionally our planner works in the following order:
router - > pushdown -> repartition -> pull to coordinator
However, for INSERT .. SELECT commands, we did not support "router".
In practice, that is not a big issue, because pushdown planning can
handle router case as well.
However, with PG 16, certain outer joins are converted to JOIN without
any conditions (e.g., JOIN .. ON (true)) and the filters are pushed down
to the tables.
When the filters are pushed down to the tables, router planner can
detect. However, pushdown planner relies on JOIN conditions.
An example query:
```
INSERT INTO agg_events (user_id)
SELECT raw_events_first.user_id
FROM raw_events_first LEFT JOIN raw_events_second
ON raw_events_first.user_id = raw_events_second.user_id
WHERE raw_events_first.user_id = 10;
```
As a side effect of this change, now we can also relax certain
limitation that "pushdown" planner emposes, but not "router". So, with
this PR, we also allow those.
Closes https://github.com/citusdata/citus/pull/6772
DESCRIPTION: Prevents unnecessarily pulling the data into coordinator
for some INSERT .. SELECT queries that target a single-shard group
and the expression originating from the source. If the types are different, Citus uses
different hash functions for the two column types, which might lead to incorrect repartitioning
of the result data
Previously, we only checked whether the relations are colocated, but we
ignore the shard indexes. That causes certain queries still to be
accidentally router. We should enforce colocation checks for both shard
index and table colocation id to make the check restrictive enough.
For example, the following query should not be router, and after this
patch, it won't:
```SQL
SELECT
user_id
FROM
((SELECT user_id FROM raw_events_first WHERE user_id = 15) EXCEPT
(SELECT user_id FROM raw_events_second where user_id = 17)) as foo;
```
DESCRIPTION: Enforce shard level colocation with
citus.enable_non_colocated_router_query_pushdown
DESCRIPTION: PR description that will go into the change log, up to 78
characters
There are 4 errors arised recently and I fixed them in this PR. Problems
and fixes are as below:
1. When executing make step in packaging pipeline, if it gets error, we
can not detect it since there are additional operations after make in
one line.
With this fix, now if an error occured after make execution, we can
detect and see the step red and failed here,
2. Recently we started to get the error ` fatal: detected dubious
ownership in repository at '/__w/citus/citus' ` as below
https://github.com/citusdata/citus/actions/runs/5542692968/jobs/10117706723#step:7:9
There is a fix for that one as well.
3. fixed the requirements issue arised related to urllib3 library
version
4. Getting errors with centos-8 docker image with the new postgres-dev
packages. Now, changed centos-8 image with almalinux-8 and now it works
This PR provides successful compilation against PG16Beta2. It does some
necessary refactoring to prepare for full support of version 16, in
https://github.com/citusdata/citus/pull/6952 .
Change RelFileNode to RelFileNumber or RelFileLocator
Relevant PG commit
b0a55e43299c4ea2a9a8c757f9c26352407d0ccc
new header for varatt.h
Relevant PG commit:
d952373a987bad331c0e499463159dd142ced1ef
drop support for Abs, use fabs
Relevant PG commit
357cfefb09115292cfb98d504199e6df8201c957
tuplesort PGcommit: d37aa3d35832afde94e100c4d2a9618b3eb76472
Relevant PG commit:
d37aa3d35832afde94e100c4d2a9618b3eb76472
Fix vacuum in columnar
Relevant PG commit:
4ce3afb82ecfbf64d4f6247e725004e1da30f47c
older one:
b6074846cebc33d752f1d9a66e5a9932f21ad177
Add alloc_flags to pg_clean_ascii
Relevant PG commit:
45b1a67a0fcb3f1588df596431871de4c93cb76f
Merge GetNumConfigOptions() into get_guc_variables()
Relevant PG commit:
3057465acfbea2f3dd7a914a1478064022c6eecd
Minor PG refactor PG_FUNCNAME_MACRO __func__
Relevant PG commit
320f92b744b44f961e5d56f5f21de003e8027a7f
Pass NULL context to stringToQualifiedNameList, typeStringToTypeName
The pre-PG16 error behaviour for the following
stringToQualifiedNameList & typeStringToTypeName
was ereport(ERROR, ...)
Now with PG16 we have this context input. We preserve the same behaviour
by passing a NULL context, because of the following:
(copy paste comment from PG16)
If "context" isn't an ErrorSaveContext node, this behaves as
errstart(ERROR, domain), and the errsave() macro ends up acting
exactly like ereport(ERROR, ...).
Relevant PG commit
858e776c84f48841e7e16fba7b690b76e54f3675
Use RangeVarCallbackMaintainsTable instead of RangeVarCallbackOwnsTable
Relevant PG commit:
60684dd834a222fefedd49b19d1f0a6189c1632e
FIX THIS: Not implemented grant-level control of role inheritance
see PG commit
e3ce2de09d814f8770b2e3b3c152b7671bcdb83f
Make Scan node abstract
PG commit:
8c73c11a0d39049de2c1f400d8765a0eb21f5228
Change in Var representations, get_relids_in_jointree
PG commit
2489d76c4906f4461a364ca8ad7e0751ead8aa0d
Deadlock detection changes because SHM_QUEUE is removed
Relevant PG Commit:
d137cb52cb7fd44a3f24f3c750fbf7924a4e9532
TU_UpdateIndexes
Relevant PG commit
19d8e2308bc51ec4ab993ce90077342c915dd116
Use object_ownercheck and object_aclcheck functions
Relevant PG commits:
afbfc02983f86c4d71825efa6befd547fe81a926
c727f511bd7bf3c58063737bcf7a8f331346f253
Rework Permission Info for successful compilation
Relevant PG commits:
postgres/postgres@a61b1f7postgres/postgres@b803b7d
---------
Co-authored-by: onderkalaci <onderkalaci@gmail.com>
Index scans in PG16 return empty sets because of extra compatibility
enforcement for `ScanKeyInit` arguments.
Could be one of the relevant PG commits:
c8b2ef05f4
This PR fixes all incompatible `RegProcedure` and `Datum` arguments in
all `ScanKeyInit` functions used throughout the codebase.
Helpful for https://github.com/citusdata/citus/pull/6952
The link in our readme directly goes to our channel, meaning people
finding the link here for the first time are unable to join slack this
way.
Given that the target audience using this link is most likely not part
of the slack channel yet it would be better to link to our self serve
signup flow at slack.citusdata.com, which is the same we use on
citusdata.com.
From simple testing you should still get redirected to the channel if
you are already joined and signed in.
This PR fixes the following:
- in oraclelinux-7 `Make` step
```
/usr/bin/ld: utils/replication_origin_session_utils.o: relocation R_X86_64_PC32 against undefined symbol
`IsLocalReplicationOriginSessionActive' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
```
`IsLocalReplicationOriginSessionActive` function has improper inline
declaration, fixed that
- in centos-7 `Make` step
```
utils/background_jobs.c: In function 'StartCitusBackgroundTaskExecutor':
utils/background_jobs.c:1746:6: warning: function might be possible candidate for 'gnu_printf' format attribute
[-Wsuggest-attribute=format]
database, user, jobId, taskId);
^
```
should use `pg_attribute_printf(3,4)` instead of
`pg_attribute_printf(3,0)` since the number of arguments varies for
`SafeSnprintf(char *str, rsize_t count, const char *fmt, ...)`
---------
Co-authored-by: naisila <nicypp@gmail.com>
Some clients send ALTER TABLE .. ADD COLUMN .. commands together
with some other DDLs and this makes it impossible to directly send
the original DDL command to the workers.
For this reason, this commit adds support for deparsing such ALTER
TABLE commands so that we can avoid from directly sending the original
one to the workers.
Partially fixes https://github.com/citusdata/citus/issues/690.
Fixes#3678
We allow materialized view to exist in distrbuted schema but they should
not be tried to be converted to a tenant table since they cannot be
distributed.
Fixes https://github.com/citusdata/citus/issues/7041
Inserting into `pg_dist_schema` causes unexpected duplicate key errors,
for distributed schemas that already exist. With this commit we skip the
insertion if the schema already exists in `pg_dist_schema`.
The error:
```sql
SET citus.enable_schema_based_sharding TO ON;
CREATE SCHEMA sc2;
CREATE SCHEMA IF NOT EXISTS sc2;
NOTICE: schema "sc2" already exists, skipping
ERROR: duplicate key value violates unique constraint "pg_dist_schema_pkey"
DETAIL: Key (schemaid)=(17294) already exists.
```
fixes: #7042
Jelte Fennema
naisila
onderkalaci
Teja Mupparti
Nils Dijk
Carol Smith
zhjwpku
ahmet gedemenli
Gürkan İndibay
Naisila Puka
mulander
aykut-bozkurt
Halil Ozan Akgül
Onur Tirtir