Commit Graph

2656 Commits (8e4ff34a2ef0511a10ae025ff09393252d954640)

Author SHA1 Message Date
Halil Ozan Akgul c0eb67b24f Skip forceCloseAtTransactionEnd connections only if BEGIN was not sent on them 2021-11-01 17:43:04 +03:00
Jelte Fennema 57a0228c52
Fix string-concatenation warning on Clang 13 (#5425)
Clang 13 complains about a suspicious string concatenation. It thinks we
might have missed a comma. This adds parentheses to make it clear that
concatenation is indeed what we meant.
2021-11-01 13:55:43 +03:00
naisila 796d56a7b1 Rename ddlJob->commandString to ddlJob->metadataSyncCommand 2021-10-29 23:45:43 +03:00
Ahmet Gedemenli 67dca4363d
Dont auto-undistribute user-added citus local tables (#5314)
* Disable auto-undistribute for user-added citus local tables
2021-10-28 12:10:26 +03:00
Philip Dubé cc50682158 Fix typos. Spurred spotting "connectios" in logs 2021-10-25 13:54:09 +00:00
Jelte Fennema 3bdbfc3edf
Fix duplicate typedef which can cause compile failures (#5406)
ColumnarScanDesc is already defined in columnar_tableam.h. Redifining it
again causes a compiler error on some C compilers.

Useful reference: https://bugzilla.redhat.com/show_bug.cgi?id=767538

Fixes #5404
2021-10-25 12:20:13 +00:00
Onder Kalaci ce4c4540c5 Simplify 2PC decision in the executor
It seems like the decision for 2PC is more complicated than
it should be.

With this change, we do one behavioral change. In essense,
before this commit, when a SELECT task with replication factor > 1
is executed, the executor was triggering 2PC. And, in fact,
the transaction manager (`ConnectionModifiedPlacement()`) was
able to understand not to trigger 2PC when no modification happens.

However, for transaction blocks like:
BEGIN;
-- a command that triggers 2PC
-- A SELECT command on replication > 1
..
COMMIT;

The SELECT was used to be qualified as required 2PC. And, as a side-effect
the executor was setting `xactProperties.errorOnAnyFailure = true;`

So, the commands was failing at the time of execution. Now, they fail at
the end of the transaction.
2021-10-23 09:06:28 +02:00
Onder Kalaci 575bb6dde9 Drop support for Inactive Shard placements
Given that we do all operations via 2PC, there is no way
for any placement to be marked as INACTIVE.
2021-10-22 18:03:35 +02:00
Önder Kalacı b3299de81c
Drop support for citus.multi_shard_commit_protocol (#5380)
In the past, we allowed users to manually switch to 1PC
(e.g., one phase commit). However, with this commit, we
don't. All multi-shard modifications are done via 2PC.
2021-10-21 14:01:28 +02:00
Marco Slot dafba6c242 Deprecate master_get_table_metadata UDF 2021-10-21 12:08:05 +02:00
Marco Slot defb97b7f5 Support operator class parameters in indexes 2021-10-20 17:03:59 +02:00
Önder Kalacı 3f726c72e0
When replication factor > 1, all modifications are done via 2PC (#5379)
With Citus 9.0, we introduced `citus.single_shard_commit_protocol` which
defaults to 2PC.

With this commit, we prevent any user to set it to 1PC and drop support
for `citus.single_shard_commit_protocol`.

Although this might add some overhead for users, it is already the default
behaviour (so less likely) and marking placements as INVALID is much
worse.
2021-10-20 01:39:03 -07:00
Marco Slot 096660d61d Remove master_apply_delete_command 2021-10-18 22:29:37 +02:00
Marco Slot 93e79b9262 Never allow co-located joins of append-distributed tables 2021-10-18 21:11:16 +02:00
Marco Slot b97e5081c7 Disable co-located joins for append-distributed tables 2021-10-18 21:11:16 +02:00
Marco Slot dfad73d918 Disable implicit single re-partition joins for append tables 2021-10-18 21:11:16 +02:00
Marco Slot 2206e64e42 Disable single-repartition joins for append tables 2021-10-18 21:11:16 +02:00
Önder Kalacı 31c8f279ac
Add helper UDFs to inspect object dependencies (#5293)
- citus_get_all_dependencies_for_object: emulate what Citus
                                         would qualify as
					 dependency when adding
					 a new node
- citus_get_dependencies_for_object: emulate what Citus would qualify
				     as dependency when creating an
				     object

Example use:
```SQL
-- find all the depedencies of table test
SELECT
	pg_identify_object(t.classid, t.objid, t.objsubid)
FROM
	(SELECT * FROM pg_get_object_address('table', '{test}', '{}')) as addr
JOIN LATERAL
	citus_get_all_dependencies_for_object(addr.classid, addr.objid, addr.objsubid) as t(classid oid, objid oid, objsubid int)
ON TRUE
	ORDER BY 1;
```
2021-10-18 14:46:49 +03:00
Halil Ozan Akgul e3446692f3 Fix the bug by adding comma before the values 2021-10-15 18:42:23 +03:00
Ahmet Gedemenli 35f6fe5f9f
Refactor/Improve PreprocessAlterTableStmtAttachPartition (#5366)
* Refactor/Improve PreprocessAlterTableStmtAttachPartition
2021-10-14 11:39:39 +03:00
Hanefi Onaldi 3e64dc44c8
Fix some typos in comments (#5369) 2021-10-13 13:00:39 +03:00
Teja Mupparti a8348047c5
Pushdown procedures with OUT parameters (#5348) 2021-10-11 23:14:36 -07:00
Onur Tirtir f7f4a93073 Remove get_relation_trigger_oid_compat 2021-10-11 11:53:00 +03:00
Onur Tirtir a1e0511583 Remove get_relation_constraint_oid_compat 2021-10-11 11:53:00 +03:00
Ahmet Gedemenli d19793c174 Add partitioning support for citus local tables
Add/fix tests

Fix creating partitions

Add test for mx - partition creating case

Enable cascading to partitioned tables

Fix mx partition adding test

Fix cascading through fkeys

Style

Disable converting with non-inherited fkeys

Fix detach bug

Early return in case of cascade & Add tests

Style

Fix undistribute_table bug & Fix test outputs

Remove RemovePartitionRelationIds

Test with undistribute_table

Add test for mx+convert+undistribute

Remove redundant usage of CreatePartitionedCitusLocalTable

Add some comments

Introduce bulk functions for generating attach/detach partition commands

Fix: Convert partitioned tables after adding fkey

Change the error message for partitions

Introduce function ErrorIfPartitionTableAddedToMetadata

Polish attach/detach command generation functions

Use time_partitions for testing

Move mx tests to citus_local_tables_mx

Add new partitioned table to cascade test

Add test with time series management UDFs

Fix test output

Fix: Assertion fail on relation access tracking

Style

Refactor creating partitioned citus local tables

Remove CreatePartitionedCitusLocalTable

Style

Error out if converting multi-level table

Revert some old tests

Error out adding partitioned partition

Polish

Polish/address

Fix create table partition of case

Use CascadeOperationForRelationIdList if no cascade needed

Fix create partition bug

Revert / Add new tests to mx

Style

Fix dropping fkey bug

Add test with IF NOT EXISTS

Convert to CLT when doing ATTACH PARTITION

Add comments

Add more tests with time series management

Edit the error message for converting the child

Use OR instead of AND in ErrorIfUnsupportedAlterTableStmt

Edit/improve tests

Disable ddl prop when dropping default column definitions

Disable/enable ddl prop just before/after the command

Add comment

Add sequence test

Add trigger test

Remove NeedCascadeViaForeignKeys

Add one more insert to sequence test

Add comment

Style

Fix test output shard ids

Update comments

Disable creating fkey on partitions

Move partition check to CreateCitusLocalTable

Add comment

Add check for  attachingmulti-level  partition

Add test for pg_constraint

Check pg_dist_partition in tests

Add test inserting on the worker
2021-10-11 10:45:07 +03:00
Halil Ozan Akgul 9c9d4b5eeb Turn MX on by default 2021-10-08 18:17:21 +03:00
Naisila Puka d0390af72d
Add fix_partition_shard_index_names udf to fix currently broken names (#5291)
* Add udf to include shardId in broken partition shard index names

* Address reviews: rename index such that operations can be done on it

* More comprehensive index tests

* Final touches and formatting
2021-10-07 19:34:52 +03:00
Marco Slot 91b647024a Fixes CREATE INDEX deparsing issue 2021-10-06 13:08:16 +02:00
Onur Tirtir 5d8f74bd0b
(Share) Lock buffer page when reading from columnar storage (#5338)
Under high write concurrency, we were sometimes reading columnar
metapage as all zeros.

In `WriteToBlock()`, if `clear == true`, then it will clear the page before
writing the new one, rather than just adding data to the page. That
means any concurrent connection that is holding only a pin will be
able to see the all-zero state between the `InitPage()` and the
`memcpy_s()`.

Moreover, postgres/storage/buffer/README states that:

> Buffer access rules:
>
> 1. To scan a page for tuples, one must hold a pin and either shared or
> exclusive content lock.  To examine the commit status (XIDs and status bits)
> of a tuple in a shared buffer, one must likewise hold a pin and either shared
> or exclusive lock.

For those reasons, we have to make sure to never keep a pin on the
page without (at least) the shared lock, to avoid having such problems.
2021-10-06 11:57:02 +03:00
Halil Ozan Akgul 43d5853b6d Fixes function names in comments 2021-10-06 09:24:43 +03:00
Hanefi Onaldi a74409f24c
Bump Citus to 11.0devel 2021-10-01 22:21:22 +03:00
Onur Tirtir fe72e8bb48
Discard index deletion requests made to columnarAM (#5331)
A write operation might trigger index deletion if index already had
dead entries for the key we are about to insert.
There are two ways of index deletion:
  a) simple deletion
  b) bottom-up deletion (>= pg14)

Since columnar_index_fetch_tuple never sets all_dead to true,
columnarAM doesn't ever expect to receive simple deletion requests
(columnar_index_delete_tuples) as we don't mark any index entries
as dead.

However, since columnarAM doesn't delete any dead entries via simple
deletion, postgres might ask for a more comprehensive deletion
(i.e.: bottom-up) at some point when pg >= 14.

So with this commit, we start gracefully ignoring bottom-up deletion
requests made to columnar_index_delete_tuples.

Given that users can anyway "VACUUM FULL" their columnar tables,
we don't see any problem in ignoring deletion requests.
2021-10-01 14:32:47 +03:00
Önder Kalacı c2311b4c0c
Make (columnar.stripe) first_row_number index a unique constraint (#5324)
* Make (columnar.stripe) first_row_number index a unique constraint

Since stripe_first_row_number_idx is required to scan a columnar
table, we need to make sure that it is created before doing anything
with columnar tables during pg upgrades.

However, a plain btree index is not a dependency of a table, so
pg_upgrade cannot guarantee that stripe_first_row_number_idx gets
created when creating columnar.stripe, unless we make it a unique
"constraint".

To do that, drop stripe_first_row_number_idx and create a unique
constraint with the same name to keep the code change at minimum.

* Add more pg upgrade tests for columnar

* Fix a logic error in uprade_columnar_after test

Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
2021-09-30 10:51:56 +03:00
Jeff Davis d49d321eac Columnar: only call BuildStripeMetadata() with heap tuple.
BuildStripeMetadata() calls HeapTupleHeaderGetXmin(), which must only
be called on a proper heap tuple with MVCC information. Make sure the
caller passes the heap tuple, and not a datum tuple.

Fixes #5318.
2021-09-23 15:51:01 -07:00
tejeswarm a1604a87e6 Parition shards to be colocated with the parent shards 2021-09-22 14:47:04 -07:00
Onur Tirtir 77a2dd68da
Revoke read access to columnar.chunk from unprivileged user (#5313)
Since this could expose chunk min/max values to unprivileged users.
2021-09-22 16:23:02 +03:00
Onur Tirtir 68335285b4 Columnar CustomScan: Pushdown BoolExpr's as we do before 2021-09-22 10:51:34 +03:00
Onur Tirtir e6ed764f63
Check if xact id is in progress before checking if aborted (#5312) 2021-09-21 21:20:31 +03:00
Onur Tirtir f8b1ff7214
Add CheckCitusVersion() calls to columnarAM (#5308)
Considering all code-paths that we might interact with a columnar table,
add `CheckCitusVersion` calls to tableAM callbacks:
- initializing table scan (`columnar_beginscan` & `columnar_index_fetch_begin`)
- setting a new filenode for a relation (storage initializiation or a table rewrite)
- truncating the storage
- inserting tuple (single and multi)

Also add `CheckCitusVersion` call to:
- drop hook (`ColumnarTableDropHook`)
- `alter_columnar_table_set` & `alter_columnar_table_reset` UDFs
2021-09-20 17:26:41 +03:00
Onder Kalaci cea937f52f Add missing version checks for citus_internal_XXX functions 2021-09-20 09:54:35 +02:00
SaitTalhaNisanci 35ff513dfe
Give proper error while distributing a temp table (#5269) 2021-09-17 14:34:40 +03:00
jeff-davis 6e8b19984e
Columnar: separate plan and runtime quals. (#5261)
* Columnar: separate plain and exec quals.

Make a clear separation between plain quals, which contain constants
or extern params; and exec quals, which contain exec params and can't
be evaluated until a rescan.

Fixes #5258.

* more vanilla tests

Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
2021-09-13 10:54:53 -07:00
jeff-davis d48ceee238
Columnar: add method ReparameterizeCustomPathByChild. (#5275)
When performing a partition-wise join, the planner will adjust paths
parameterized by the parent rel to instead parameterize by the child
rel directly. When this reparameterization happens, we also need to
adjust the join quals to reference the child rather than the parent.

Fixes #5257.
2021-09-13 10:33:48 -07:00
Onur Tirtir ea61efb63a
Not flush writes until need to read them when doing index-scan on columnar (#5247)
Not flush pending writes if given tid belongs to a "flushed" or
"aborted" stripe write, or to an "in-progress" stripe write of
another backend.

That way, we would reduce the cases where we flush single-tuple
stripes during index scan.

To do that, we follow below steps for index look-up's:

- Do not flush any pending writes and do stripe metadata look-up for
  given tid.
  If tuple with tid is found, then no need to do another look-up
  since we already found the tuple without needing to flush pending
  writes.

- If tuple is not found without flushing pending writes, then we have two
  scenarios:

  -  If given tid belongs to a pending write of my backend, then do stripe
     metadata look-up for given tid. But this time first **flush any pending
     writes**.
     
  -  Otherwise, just return false from `index_fetch_tuple` since flushing
      pending writes wouldn't help.
2021-09-13 18:41:20 +02:00
Onur Tirtir 4ee0fb2758
Make sure to skip aborted writes when reading the first tuple (#5274)
With 5825c44d5f, we made the changes to
skip aborted writes when scanning a columnar table.

However, looks like we forgot to handle such cases for the very first
call made to columnar_getnextslot. That means, that commit only
considered the intermediate stripe read operations.

However, functions called by columnar_getnextslot to find first stripe
to read (ColumnarBeginRead & ColumnarRescan) were not caring about
those aborted writes.

To fix that, we teach AdvanceStripeRead to find the very first stripe
to read, and then start using it where were blindly calling
FindNextStripeByRowNumber.
2021-09-13 11:50:53 +03:00
Burak Velioglu ceec5d72e3
Swallow errors while aborting remote transactions 2021-09-10 11:06:16 +03:00
Naisila Puka a69abe3be0
Fixes bug about int and smallint sequences on MX (#5254)
* Introduce worker_nextval udf for int&smallint column defaults

* Fix current tests and add new ones for worker_nextval
2021-09-09 23:41:07 +03:00
Onur Tirtir be74518965
Improve memset calls made to reset bool arrays (#5262) 2021-09-09 17:56:03 +03:00
Halil Ozan Akgul 19af1cef2f Errors for CTEs with search clause
Relevant PG commit:
3696a600e2292d43c00949ddf0352e4ebb487e5b
2021-09-09 13:48:24 +03:00
Marco Slot f84164a000 Avoid switch to superuser in worker_merge_files_into_table 2021-09-09 11:00:29 +02:00
Marco Slot 4faa49775b Perform copy command as regular user in worker_append_table_to_shard 2021-09-09 11:00:29 +02:00
Hanefi Onaldi 9ae912a8c8
Prevent C-style comments in all directories (#5250) 2021-09-09 11:54:58 +03:00
SaitTalhaNisanci e3e0a028c7
return early in case we want to skip outer vars (#5259) 2021-09-09 10:53:36 +03:00
Onur Tirtir 32e3e51ed4
Fix a compiler warning that we get on debian (#5260) 2021-09-08 20:03:59 +03:00
Onur Tirtir be3914ae28 Prevent generating index-only "Path"s for columnar tables
Previously, even when `EXPLAIN` output tells that we will do
index-only scan, it was never the case since columnar tables
don't have the visibility fork that postgres is looking for.

For this reason, visibility check done in
`IndexOnlyNext->VM_ALL_VISIBLE`
code-path was always returning false and postgres was reading
the tuple from the columnar relation itself.
2021-09-08 14:14:24 +03:00
Onur Tirtir cc49e63222
Not read heaptuple after closing pg_rewrite (#5255) 2021-09-08 13:03:17 +02:00
Onur Tirtir 3340f17c4e
Prevent planner from choosing parallel scan for columnar tables (#5245)
Previously, for regular table scans, we were setting `RelOptInfo->partial_pathlist`
to `NIL` via `set_rel_pathlist_hook` to discard scan `Path`s that need to use any
parallel workers, this was working nicely.

However, when building indexes, this hook doesn't get called so we were not
able to prevent spawning parallel workers when building an index. For this
reason, 9b4dc2f804 added basic
implementation for `columnar_parallelscan_*` callbacks but also made some
changes to skip using those workers when building the index.

However, now that we are doing stripe reservation in two stages, we call 
`heap_inplace_update` at some point to complete stripe reservation.
However, postgres throws an error if we call `heap_inplace_update` during
a parallel operation, even if we don't actually make use of those workers.

For this reason, with this pr, we make sure to not generate scan `Path`s that
need to use any parallel workers by using `get_relation_info_hook`.

This is indeed useful to prevent spawning parallel workers during index builds.
2021-09-08 13:53:43 +03:00
Onur Tirtir 5825c44d5f
Handle aborted writes properly when scanning a columnar table (#5244)
If it is certain that we will not use any `parallel_worker`s for a columnar table,
then stripe entries inserted by aborted transactions become visible to
`SnapshotAny` and that causes `REINDEX` to fail by throwing a duplicate key
error.

To fix that:
* consider three states for a stripe write operation:
   "flushed", "aborted", or "in-progress",
* make sure to have a clear separation between them, and
* act according to those three states when reading from a columnar table
2021-09-08 13:26:11 +03:00
Jelte Fennema bb5c494104 Enable binary encoding by default on PG14
Since PG14 we can now use binary encoding for arrays and composite types
that contain user defined types. This was fixed in this commit in
Postgres: 670c0a1d47

This change starts using that knowledge, by not necessarily falling back
to text encoding anymore for those types.

While doing this and testing a bit more I found various cases where
binary encoding would fail that our checks didn't cover. This fixes
those cases and adds tests for those. It also fixes EXPLAIN ANALYZE
never using binary encoding, which was a leftover of workaround that
was not necessary anymore.

Finally, it changes the default for both `citus.enable_binary_protocol`
and `citus.binary_worker_copy_format` to `true` for PG14 and up. In our
cloud offering `binary_worker_copy_format` already was true by default.
`enable_binary_protocol` had some bug with MX and user defined types,
this bug was fixed by the above mentioned fixes.
2021-09-06 10:27:29 +02:00
Burak Velioglu c3895f35cd
Add helper UDFs for easy time partition management
- get_missing_time_partition_ranges: Gets the ranges of missing partitions for the given table, interval and range unless any existing partition conflicts with calculated missing ranges.

- create_time_partitions: Creates partitions by getting range values from get_missing_time_partition_ranges.

- drop_old_time_partitions: Drops partitions of the table older than given threshold.
2021-09-03 23:03:13 +03:00
Onur Tirtir 2b71263e40
Align columnar path costing functions (#5239)
* Rename RecostColumnarPaths to CostColumnarPaths

* Rename RecostColumnarIndexPath to CostColumnarIndexPath

* Reorder args of CostColumnarScan to align with other two costing functions

* Not adjust index scan start-up cost

* Rename ColumnarIndexScanAddTotalCost to ColumnarIndexScanAdditionalCost

* Reflect that index scan will at least read one stripe in totalCost calculation

* Organize declarations in columnar_customscan.c
2021-09-03 19:37:42 +03:00
jeff-davis cc58b58f73
Columnar: reserve metapage flag for UNLOGGED support. (#5237)
Reserve space in the metapage for a flag to support UNLOGGED tables in
the future without a metapage upgrade.
2021-09-03 08:40:55 -07:00
Halil Ozan Akgul 7fadfb74bb Adds error message for REINDEX TABLE queries on distributed partitioned tables 2021-09-03 16:46:42 +03:00
Sait Talha Nisanci 0b67fcf81d Fix style 2021-09-03 16:09:59 +03:00
Halil Ozan Akgul e1f5520e1a Adds propagation of ALTER TABLE .. ALTER COLUMN .. SET COMPRESSION .. 2021-09-03 15:44:28 +03:00
Ahmet Gedemenli 2b263f9a2a ALTER STATISTICS .. OWNER TO CURRENT_ROLE (#5225)
(cherry picked from commit 42322caf90ca094777aa01376e02d1187afc1560)
2021-09-03 15:44:28 +03:00
Onder Kalaci 5844ab286c Support OUT parameters in procedure pushdown delegation
In PG 14, procedures can have OUT parameters. In Citus' procedure
delegation framework, we need to adjust the function expression
to get the outargs parameters.

Releven PG change:
e56bce5d43
2021-09-03 15:44:28 +03:00
Ahmet Gedemenli 1ff7186d20 Extended statistics on expressions - PG14 a4d75c8 (#5224)
(cherry picked from commit 1268415f123b5d99cfacfe207c8670240efc1c00)
2021-09-03 15:44:28 +03:00
Halil Ozan Akgul 113d5d6615 Adds support for column compression in table distribution 2021-09-03 15:44:28 +03:00
Ahmet Gedemenli 6fbdeb38a8 ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY - PG14 #71f4c8c (#5223) 2021-09-03 15:44:28 +03:00
Onder Kalaci c431bb2e45 Add support for "COPY dist/ref tables FROM" progress report
Simply call Postgres' function to report the progress on
each row recieved.

Note that we currently do not support "COPY dist/ref TO .." progress
report nicely. Citus has some specialized logic to support
"COPY dist/ref TO .." such that it either converts the underlying
command into "COPY (SELECT * FROM dist/ref ) ..." or sends COPY
command to shards directly. In the former case, "tuples_processed"
is only updated when the executor returns all the tuples, so the
progress is not accurate. In the latter case, Citus can actually
implement the progress report. But, for the sake of consistency,
we prefer to not implement at all.

Added to PG 14 with https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=8a4f618e7ae3cb11b0b37d0f06f05c8ff905833f
2021-09-03 15:44:28 +03:00
Ahmet Gedemenli 66303785f3 Add option PROCESS_TOAST to VACUUM - PG14 #7cb3048 (#5219)
(cherry picked from commit e63bdfc49f9203db14ef77313c1d5e3461a84a32)
2021-09-03 15:44:28 +03:00
Sait Talha Nisanci 35a3f7240d CHANGELOG: Allow REINDEX to change the tablespace of the new index 2021-09-03 15:44:28 +03:00
Sait Talha Nisanci 2656d885f9 Rewrite AppendColumnNames for Pg14
Postgres changed stats expression types as of PG14. Hence we needed to
write the AppendColumnNames method. Also they removed the error on PG
side so we remove it as well.

Relevant commits on pg14:
a4d75c86bf15220df22de0a92c819ecef9db3849
388e75ad33489b77cfb9a8590a91e9287d8fb960
2021-09-03 15:41:28 +03:00
Sait Talha Nisanci d1c0403055 Disable Query Idenfifier calculation in tests
When queryId is not 0 and verbose is true, the query identifier is
emitted to the explain output. This is breaking Postgres outputs.
We disable de query identifier calculation in the tests.
Commit on PG that introduced the query identifier in the explain output:
4f0b0966c866ae9f0e15d7cc73ccf7ce4e1af84b
2021-09-03 15:41:28 +03:00
Halil Ozan Akgul 9fc4c27b08 Readds deleted resultRelInfo changes for previos PG versions
These changes were removed in commit: Introduces ExecSimpleRelationInsert_compat and modifyStateResultRelInfo macros
We shouldn't have removed them but instead kept them for before PG14
2021-09-03 15:41:28 +03:00
Halil Ozan Akgul 7823e49219 Introduces pg_get_statisticsobj_worker_compat macro
Relevant PG commit:
a4d75c86bf15220df22de0a92c819ecef9db3849
2021-09-03 15:41:28 +03:00
Sait Talha Nisanci abd3c1089b Use oid_hash in write state management 2021-09-03 15:41:28 +03:00
Halil Ozan Akgul 8ef94dc1f5 Changes array_cat argument type from anyarray to anycompatiblearray
Relevant PG commit:
9e38c2bb5093ceb0c04d6315ccd8975bd17add66

fix array_cat_agg for pg upgrades

array_cat_agg now needs to take anycompatiblearray instead of anyarray
because array_cat changed its type from anyarray to anycompatiblearray
with pg14.

To handle upgrades correctly, we drop the aggregate in
citus_pg_prepare_upgrade. To be able to drop it, we first remove the
dependency from pg_depend.

Then we create the right aggregate in citus_finish_pg_upgrade and we
also add the dependency back to pg_depend.
2021-09-03 15:41:28 +03:00
Sait Talha Nisanci a1bfb4f31b Fix unlimited copy size variable's value 2021-09-03 15:41:28 +03:00
Sait Talha Nisanci 29f5b99951 Use empty string instead of NULL for queryString
Postgres doesn't accept NULL for queryStrings in explain plans anymore.
Internally, there are some places in Postgres where they modified the
NULLS to ""(the empty string). So we do the same on citus side.

Commit on Postgres:
1111b2668d89bfcb6f502789158b1233ab4217a6
2021-09-03 15:27:25 +03:00
Sait Talha Nisanci 96833e2b8f Use HASH_STRINGS explicitly in hash functions
Postgres expects to set the HASH_STRINGS explicitly in case of the
default behaivor for string hash function.

Postgres Commit
b3817f5f774663d55931dd4fab9c5a94a15ae7ab
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul 5930378f61 Renames shadowing ruleutils_14.c variables 2021-09-03 15:27:25 +03:00
Halil Ozan Akgul b21a00e775 Introduces index_insert_compat macro
index_insert function now has a new parameter, indexUnchanged
This new macro give us the ability to use these new parameter for PG14 and they don't give the parameters for previous versions
Existing parameter is set to false

Relevant PG commit:
9dc718bdf2b1a574481a45624d42b674332e2903
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul fd2ca2825b Introduces ExecSimpleRelationInsert_compat and modifyStateResultRelInfo macros
es_result_relation_info is removed from Estate. In this commit we make some changes to handle that.
resultRelationInfo filed is added to ModifyState to support the removed field.

Relevant PG commits:
1375422c7826a2bf387be29895e961614f69de4b
a04daa97a4339c38e304cd6164d37da540d665a8
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul b644ac55c6 Introduces GetOldestNonRemovableTransactionId_compat macro
GetOldestXmin function is removed so we use GetOldestNonRemovableTransactionId functions instead
GetOldestNonRemovableTransactionId_compat picks the appropriate one

Relevant PG commit:
dc7420c2c9274a283779ec19718d2d16323640c0
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul cb3b76ed24 Introduces get_partition_parent_compat and RelationGetPartitionDesc_compat macros
get_partition_parent and RelationGetPartitionDesc functions now have new parameters to also include detached partitions
Thess new macros give us the ability to use these new parameter for PG14 and they don't give the parameters for previous versions
Existing parameters are set to not accept detached partitions

Relevant PG commit:
71f4c8c6f74ba021e55d35b1128d22fb8c6e1629
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul 898d3bb8d3 Introduces proc_statusflags_compat macro
In two commits vacuumFlags in PGXACT is moved and then renamed to status flags
This macro uses the appropriate version of the flag

Relevant PG commits:
5788e258bb26495fab65ff3aa486268d1c50b123
cd9c1b3e197a9b53b840dcc87eb41b04d601a5f9
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul 287706b717 Introduces SetTuplestoreDestReceiverParams_compat macro
SetTuplestoreDestReceiverParams function now has two new parameters
This new macro give us the ability to use this new parameter for PG14 and it doesn't give the parameter for previous versions
Existing parameters are set to NULL to keep previous behavior

Relevant PG commit:
2f48ede080f42b97b594fb14102c82ca1001b80c
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul b01e7e884c Pass NULL for plannerInfo as we don't generate PlaceHolderVars 2021-09-03 15:27:25 +03:00
Halil Ozan Akgul 86d9260781 Uses lfirst_node in ruleutils_14.c
Relevant PG commit:
2b00db4fb0c7f02f000276bfadaab65a14059168
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul 3b7bcf7555 Adds missing include_out_argument parameter to func_get_detail in ruleutils_14.c
Relevant PG commit:
e56bce5d43789cce95d099554ae9593ada92b3b7
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul 2990cfb6c9 Adds SQL-standard function body support to ruleutils_14.c
Relevant PG commit:
e717a9a18b2e34c9c40e5259ad4d31cd7e420750
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul 84f0be56c3 Adds EXTRACT cases to get_func_sql_syntax in ruleutils_14.c
Relevant PG commit:
a2da77cdb4661826482ebf2ddba1f953bc74afe4
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul 131062d6b5 Removes ModifyTable check from set_deparse_plan in ruleutils_14.c
Relevant PG commit:
86dc90056dfdbd9d1b891718d2e5614e3e432f35
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul f557bae64c Adds JOIN ... USING alias to ruleutils_14.c
Relevant PG commit:
055fee7eb4dcc78e58672aef146334275e1cc40d
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul c3f0528607 Extends statistics on expressions in ruleutils_14.c
Relevant PG commit:
a4d75c86bf15220df22de0a92c819ecef9db3849
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul af2853d1de Adds GROUP BY DISTINCT to ruleutils_14.c
Relevant PG commit:
be45be9c33a85e72cdaeb9967e9f6d2d00199e09
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul 5bb538543d Enhances cycle mark values at ruleutils_14.c
Relevant PG commit:
f4adc41c4f92cc91d507b19e397140c35bb9fd71
2021-09-03 15:27:25 +03:00
Halil Ozan Akgul 12b3c04fe3 Adds SEARCH and CYCLE clauses to ruleutils_14.c
Relevant PG commit:
3696a600e2292d43c00949ddf0352e4ebb487e5b
2021-09-03 15:27:25 +03:00