external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,068 Commits
645 Branches
162 Tags
103 MiB
Commit Graph

4750 Commits (a1ab60f5ad0108058f8032bdf34cedd8e87e4604)

Author SHA1 Message Date
gindibay 473f6cbf05 Fixes metadata sync 2023-10-27 08:37:46 +03:00
gindibay 641e413676 Adds metadatasync for create database 2023-10-26 20:36:43 +03:00
gindibay c8fcf080c2 Fixes out files for transaction message 2023-10-24 20:44:43 +03:00
gindibay c437850db5 Beatufies error message 2023-10-24 19:25:26 +03:00
gindibay 48d749e2e8 Fixes message for non-transaction-safe commands 2023-10-24 19:15:46 +03:00
gindibay c7da2cd122 Fixes cannotBeExecutedInTransction 2023-10-24 15:44:59 +03:00
Gürkan İndibay c5c7705dbe
Merge branch 'main' into create_alter_database 2023-10-24 17:09:54 +03:00
gindibay c7b040c3cd Fixes indentation 2023-10-24 15:19:50 +03:00
gindibay 2f52fbd13e Fixes missing ok issue 2023-10-24 15:14:05 +03:00
gindibay 3a0a5ae344 Fixes missin ok issue 2023-10-24 15:13:22 +03:00
gindibay b5cbc048b7 Fixes missing ok issue 2023-10-24 15:12:08 +03:00
gindibay f779947b2b Removes unnecessary logs 2023-10-24 14:57:36 +03:00
gindibay 1dcee370e1 Fixes create_drop_database_test 2023-10-24 14:44:34 +03:00
gindibay fb9f75bb24 Removes unnecessary logs 2023-10-24 14:43:12 +03:00
gindibay e9e64a69c1 Fixes indentation 2023-10-24 14:18:10 +03:00
gindibay 73f0db2aed Fixes create and drop database transaction use 2023-10-24 14:09:36 +03:00
gindibay c9dae2684f tests db as role 2023-10-24 10:09:13 +03:00
Naisila Puka 10198b18e8
Technical readme small fixes (#7261) 2023-10-23 13:43:43 +03:00
Naisila Puka 1fe16fa746
Remove unnecessary pre-fastpath code (#7262) 
This code was here because we first implemented
`fast path planner` via
[#2606](https://github.com/citusdata/citus/pull/2606)
and then later `deferred pruning`
[#3369](https://github.com/citusdata/citus/pull/3369)
So, for some years, this code was useful.
 2023-10-23 13:01:48 +03:00
gindibay 690276c516 Fixes indentation 2023-10-23 12:35:53 +03:00
gindibay ea0a908702 Fixes pg_dist_objects record in coordinators 2023-10-23 12:31:25 +03:00
gindibay dad81042a5 Adds undistribute sql 2023-10-23 10:01:18 +03:00
zhjwpku 2d1444188c
Fix wrong comments around HasDistributionKey() (#7223) 
HasDistributionKey & HasDistributionKeyCacheEntry returns true when the
corresponding table has a distribution key, the comments state the
opposite,
which should be fixed.

Signed-off-by: Zhao Junwang <zhjwpku@gmail.com>
Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
 2023-10-18 10:53:00 +02:00
Onur Tirtir db13afaa7b
Fix flaky columnar_create.sql test (#7266) 2023-10-17 16:58:17 +03:00
Gürkan İndibay 71a4633dad
Fixes typo and renames multi_process_utility (#7259) 2023-10-17 16:39:37 +03:00
Jelte Fennema-Nio 788e09a39a
Add a test for citus_shards where table names have spaces (#7224) 
There was a bug reported for previous versions of Citus where
shard\_size was returning NULL for tables with spaces in them. It works
fine on the main branch though, but I'm still adding a test for this to
the main branch because it seems a good test to have.
 2023-10-16 11:38:24 +02:00
gindibay 1e91b144e2 Fixed review issues 2023-10-14 20:49:26 +03:00
gindibay a497a78239 Fixes review comments 2023-10-14 20:39:39 +03:00
gindibay c6d1ef9e41 Rollbacks enable_create_database_propagation 2023-10-14 20:19:18 +03:00
gindibay 674fd3226c Sets enable_create_database_propagation true 2023-10-14 19:50:32 +03:00
gindibay 3a6fdada11 Changes if to switch statements 2023-10-14 05:35:34 +03:00
gindibay 30bbfe444d Fixes review notes 2023-10-13 18:45:58 +03:00
gindibay ba3123455e Fixes review notes 2023-10-13 18:43:06 +03:00
gindibay ef920b0252 Fixes compile errors 2023-10-13 14:49:34 +03:00
gindibay 7ab86cf4ae Fixes indentation 2023-10-13 14:30:50 +03:00
Gürkan İndibay c3a9a37ae2
Merge branch 'create_alter_database' into alter_database_additional_options 2023-10-16 14:44:32 +03:00
gindibay ca5e234821 Adds EnableCreateDatabasePropagation for drop db 2023-10-13 14:17:14 +03:00
gindibay abf9dd9e59 Adds EnableCreateDatabasePropagation check 2023-10-13 14:17:14 +03:00
gindibay 333f77a267 Fixes distributed_object management 2023-10-13 14:17:14 +03:00
gindibay c31479d2eb Fixes review notes 2023-10-13 14:17:14 +03:00
gindibay a274ffcb51 Adds If exists statement for drop database 2023-10-13 14:17:14 +03:00
gindibay 8cc67921cb Fixes review comments 2023-10-13 14:17:14 +03:00
gindibay 875f347238 Fixes indentation 2023-10-13 14:17:14 +03:00
gindibay f2092d2f44 Clarifies the code 2023-10-13 14:17:14 +03:00
gindibay 7bdeb2f866 Fixes indentation 2023-10-13 14:17:14 +03:00
gindibay fb6dd413ad Fixes PostprocessCreateDatabaseStmt comment 2023-10-13 14:17:14 +03:00
gindibay d88a1e7785 Fixes compile error 2023-10-13 14:17:14 +03:00
gindibay d0ef13dc89 Fixes review comments 2023-10-13 14:17:14 +03:00
gindibay 480c22acc2 Fixes the comment 2023-10-13 14:17:14 +03:00
gindibay d778bb1ed6 Adds PG_VERSION_15 constant 2023-10-13 14:17:14 +03:00
gindibay d46a8394a3 Fixes compile error 2023-10-13 14:17:14 +03:00
gindibay df715a0402 Fixes indent 2023-10-13 14:17:14 +03:00
gindibay 134ec8f428 Fixes compile error for PG 14 2023-10-13 14:17:14 +03:00
gindibay 6096a0ed57 Adds missing types 2023-10-13 14:17:14 +03:00
gindibay e7f8058016 Removes T_Boolean since it breaks pg14 compile 2023-10-13 14:17:14 +03:00
gindibay 82358fb37d Fixes indentation 2023-10-13 14:17:13 +03:00
gindibay 0bf1c2c139 Refactors the code to make more managable 2023-10-13 14:17:13 +03:00
gindibay 09d201d1a8 Fixes pg15 output 2023-10-13 14:17:13 +03:00
gindibay 7bccf93183 Fixes icu locale 2023-10-13 14:17:13 +03:00
gindibay d55ecc23ae Updates test results 2023-10-13 14:17:13 +03:00
gindibay 32d74d9521 Fixes collate and ctype 2023-10-13 14:17:13 +03:00
gindibay ddea1c897b Removes iculocale 2023-10-13 14:17:13 +03:00
gindibay da46878a8f Fixes Locale 2023-10-13 14:17:13 +03:00
gindibay 031b6480f2 Fixes locale 2023-10-13 14:17:13 +03:00
gindibay 576264ceca Fixes UTF error 2023-10-13 14:17:13 +03:00
gindibay 3050b0eab9 Fixes pg15 tests 2023-10-13 14:17:13 +03:00
gindibay 214ebc4374 Fixes encodings for pg16 2023-10-13 14:17:13 +03:00
gindibay 6f36556724 Revert "Removes pg15 changes to test unusal failures" 
This reverts commit 85963e9cb5.
 2023-10-13 14:17:13 +03:00
gindibay a7112fb965 Adds tablespace to test tablespace statement 2023-10-13 14:17:13 +03:00
gindibay 6913496f42 Fixes tests 2023-10-13 14:17:13 +03:00
gindibay 94abb07409 Fixes indentation 2023-10-13 14:17:13 +03:00
gindibay 1f578791bb Fixes pg_dist_object delete problem 2023-10-13 14:17:13 +03:00
gindibay 51d1560b34 Adds OID to see the error cause 2023-10-13 14:17:13 +03:00
gindibay c3d773ac6f Changes test order 2023-10-13 14:17:13 +03:00
gindibay 1d9c02fab9 Changes schedule location 2023-10-13 14:17:13 +03:00
gindibay 5eb87839b6 Rollbacks last test 2023-10-13 14:17:13 +03:00
gindibay acece9818c Rollbacks pg_regress_multi 2023-10-13 14:17:13 +03:00
gindibay c7304abd54 Fixes pg15 file 2023-10-13 14:17:13 +03:00
gindibay 14618878c8 Tests error cause 2023-10-13 14:17:13 +03:00
gindibay 29093e042a Removes pg15 changes to test unusal failures 2023-10-13 14:17:13 +03:00
gindibay aa47ca8bda Fixes tablespace results 2023-10-13 14:17:13 +03:00
gindibay 4c79788936 Fixes tablespace error 2023-10-13 14:17:13 +03:00
gindibay b01c9ffb45 Removes unnecessary fileds from selects 2023-10-13 14:17:13 +03:00
gindibay d0bd20c9b1 Fixes tests 2023-10-13 14:17:13 +03:00
gindibay 25488afccd Changes encoding 2023-10-13 14:17:13 +03:00
gindibay a5e0b9411a Adds tests for create and drop database 2023-10-13 14:17:13 +03:00
gindibay 8c98369e21 Fixes affected tests 2023-10-13 14:17:13 +03:00
gindibay 5c50d58757 Changes EnableCreateDatabasePropagation to false 
to fix failing tests
 2023-10-13 14:17:13 +03:00
gindibay 3735bef156 Fixes sql comments 2023-10-13 14:17:13 +03:00
gindibay cef9ac702e Fixes indentation 2023-10-13 14:17:13 +03:00
gindibay 1e7c18cb61 Removes unnecessary code 2023-10-13 14:17:13 +03:00
gindibay 40893e6ffc Removes unnecessary files 2023-10-13 14:17:13 +03:00
gindibay eb1f093381 Fixes runtime and compile errors 2023-10-13 14:17:13 +03:00
gindibay ca36833b5c Fix for Pg16 compile error 2023-10-13 14:16:43 +03:00
gindibay bd398a627b Fixes compile errors 2023-10-13 14:16:43 +03:00
gindibay 39cb7977ce Fixes additional compile errors 2023-10-13 14:16:43 +03:00
gindibay 0d5c2f9350 Adds first commit 2023-10-13 14:16:43 +03:00
gindibay cd4ffd1878 Fixes merge errors 2023-10-13 06:16:12 +03:00
Gürkan İndibay c38e856f13
Merge branch 'create_alter_database' into alter_database_additional_options 2023-10-14 08:15:45 +03:00
gindibay 0fdb3384d9 Adds EnableCreateDatabasePropagation for drop db 2023-10-13 04:46:53 +03:00
gindibay 06050a1411 Adds EnableCreateDatabasePropagation check 2023-10-13 04:36:49 +03:00
gindibay bc0a283221 Fixes distributed_object management 2023-10-13 04:26:03 +03:00
gindibay 761fb13ac8 Fixes review notes 2023-10-13 03:10:17 +03:00
gindibay e5a6b7880c Adds If exists statement for drop database 2023-10-13 01:36:01 +03:00
gindibay 4087d1941d Fixes review comments 2023-10-13 00:12:38 +03:00
gindibay f8962380a1 Adds expected test result 2023-10-12 19:36:13 +03:00
gindibay 8d6d27a241 Adds set tablespace support 2023-10-12 19:06:27 +03:00
gindibay a72e474c3a Fixes test error 2023-10-11 21:25:49 +03:00
gindibay 7278968365 Adds additional tests for set tablespace 2023-10-11 20:40:26 +03:00
gindibay 80e450b860 Adds tests 2023-10-11 20:21:08 +03:00
gindibay 959d26db89 Fixes code indents 2023-10-11 20:01:12 +03:00
gindibay 6a333ca742 Adds alter database rename stmt propagation 2023-10-11 19:59:33 +03:00
gindibay fd521f2792 Fixes indentation 2023-10-11 18:09:45 +03:00
gindibay 37d2a5f5a8 Adds seperate set tablespace on each node support 2023-10-11 18:09:00 +03:00
gindibay 2d9da93d3d Adds connection limit 2023-10-11 17:28:40 +03:00
Gürkan İndibay f7f768168b
Merge branch 'main' into create_alter_database 2023-10-10 11:02:40 +03:00
Emel Şimşek e9035f6d32
Send keepalive messages in split decoder periodically to avoid wal receiver timeouts during large shard splits. (#7229) 
DESCRIPTION: Send keepalive messages during the logical replication
phase of large shard splits to avoid timeouts.

During the logical replication part of the shard split process, split
decoder filters out the wal records produced by the initial copy. If the
number of wal records is big, then split decoder ends up processing for
a long time before sending out any wal records through pgoutput. Hence
the wal receiver may time out and restarts repeatedly causing our split
driver code catch up logic to fail.

Notes: 

1. If the wal_receiver_timeout is set to a very small number e.g. 600ms,
it may time out before receiving the keepalives. My tests show that this
code works best when the` wal_receiver_timeout `is set to 1minute, which
is the default value.

2. Once a logical replication worker time outs, a new one gets launched.
The new logical replication worker sets the pg_stat_subscription columns
to initial values. E.g. the latest_end_lsn is set to 0. Our driver logic
in `WaitForGroupedLogicalRepTargetsToCatchUp` can not handle LSN value
to go back. This is the main reason for it to get stuck in the infinite
loop.
 2023-10-09 22:33:08 +03:00
gindibay 723c2b154f Fixes indentation 2023-10-09 20:27:50 +03:00
gindibay 78f8ae22c4 Clarifies the code 2023-10-09 20:24:34 +03:00
gindibay 1a26316cef Fixes indentation 2023-10-09 20:03:07 +03:00
gindibay be9f660fad Fixes PostprocessCreateDatabaseStmt comment 2023-10-09 20:01:29 +03:00
gindibay b04c52abb9 Fixes compile error 2023-10-09 19:44:36 +03:00
gindibay 5dd6ed6935 Fixes review comments 2023-10-09 19:42:17 +03:00
gindibay d573910f2a Fixes the comment 2023-10-09 18:10:28 +03:00
gindibay 3547eeeb6f Adds PG_VERSION_15 constant 2023-10-09 16:05:07 +03:00
gindibay 4b4aae2d4a Fixes compile error 2023-10-09 16:00:18 +03:00
gindibay 69ccf7dd97 Fixes indent 2023-10-09 15:57:34 +03:00
gindibay 45103629dc Fixes compile error for PG 14 2023-10-09 15:56:05 +03:00
gindibay 7022344870 Adds missing types 2023-10-09 15:14:13 +03:00
Nils Dijk 6d8725efb0
Fix leaking of memory and memory contexts in Foreign Constraint Graphs (#7236) 
DESCRIPTION: Fix leaking of memory and memory contexts in Foreign
Constraint Graphs

Previously, every time we (re)created the Foreign Constraint
Relationship Graph, we created a new Memory Context while loosing a
reference to the previous context. This old context could still have
left over memory in there causing a memory leak.

With this patch we statically have one memory context that we lazily
initialize the first time we create our foreign constraint relationship
graph. On every subsequent creation, beside destroying our previous
hashmap we also reset our memory context to remove any left over
references.
 2023-10-09 13:05:51 +02:00
gindibay b99674dcf9 Removes T_Boolean since it breaks pg14 compile 2023-10-09 13:46:54 +03:00
gindibay 2b04873159 Fixes indentation 2023-10-09 13:28:23 +03:00
gindibay 6846b7fbc7 Refactors the code to make more managable 2023-10-09 13:27:12 +03:00
Onur Tirtir 858d99be33
Take improvement_threshold into the account in citus_add_rebalance_strategy() (#7247) 
DESCRIPTION: Makes sure to take improvement_threshold into the account
in `citus_add_rebalance_strategy()`.

Fixes https://github.com/citusdata/citus/issues/7188.
 2023-10-09 13:13:08 +03:00
gindibay 93ccbf74d8 Fixes pg15 output 2023-10-06 19:14:21 +03:00
gindibay 8d59e891f0 Fixes icu locale 2023-10-06 18:58:52 +03:00
gindibay c4f8acb0d9 Updates test results 2023-10-06 18:33:10 +03:00
gindibay 35c04c5d48 Fixes collate and ctype 2023-10-06 18:14:00 +03:00
gindibay 92c1b2bc2c Removes iculocale 2023-10-06 18:02:43 +03:00
gindibay 48359e5b3b Fixes Locale 2023-10-06 17:50:10 +03:00
gindibay a19b8216a2 Fixes locale 2023-10-06 17:03:57 +03:00
gindibay d218c31893 Fixes UTF error 2023-10-06 17:01:27 +03:00
gindibay 8251a22bee Fixes pg15 tests 2023-10-06 16:41:07 +03:00
gindibay fc67b6748d Fixes encodings for pg16 2023-10-06 16:10:47 +03:00
gindibay 2162c89efd Revert "Removes pg15 changes to test unusal failures" 
This reverts commit 85963e9cb5.
 2023-10-06 15:47:42 +03:00
gindibay 791fe0db48 Adds tablespace to test tablespace statement 2023-10-06 15:45:05 +03:00
gindibay 4dff8f3c67 Fixes tests 2023-10-06 15:24:23 +03:00
gindibay 6d429521b6 Fixes indentation 2023-10-06 15:22:15 +03:00
gindibay 8ded5bc256 Fixes pg_dist_object delete problem 2023-10-06 15:18:16 +03:00
Önder Kalacı 7d6c401dd3
Update technical readme (#7248) 
Fix a wrong query, reported by @naisila
 2023-10-06 13:37:37 +03:00
gindibay 8f08074720 Adds OID to see the error cause 2023-10-05 22:30:06 +03:00
gindibay 76f1d242c9 Changes test order 2023-10-05 22:14:12 +03:00
gindibay eaf26ee601 Changes schedule location 2023-10-05 21:48:28 +03:00
gindibay e75c820ccb Rollbacks last test 2023-10-05 21:34:14 +03:00
gindibay 0c1de3484e Rollbacks pg_regress_multi 2023-10-05 20:49:19 +03:00
gindibay fd9a8f3192 Fixes pg15 file 2023-10-05 20:36:47 +03:00
gindibay 1ab31646d8 Tests error cause 2023-10-05 20:35:22 +03:00
gindibay 85963e9cb5 Removes pg15 changes to test unusal failures 2023-10-05 19:44:57 +03:00
gindibay 04c46b4099 Fixes tablespace results 2023-10-05 17:51:01 +03:00
gindibay 7ccd8c8390 Fixes tablespace error 2023-10-05 17:48:36 +03:00
gindibay 8bce0386d7 Removes unnecessary fileds from selects 2023-10-05 17:27:07 +03:00
gindibay 0b562db7df Fixes tests 2023-10-05 17:11:07 +03:00
gindibay e46c9cff9b Changes encoding 2023-10-04 23:11:48 +03:00
gindibay 236ea1d1b9 Adds tests for create and drop database 2023-10-04 21:20:19 +03:00
gindibay a708a7810d Fixes affected tests 2023-10-04 15:33:14 +03:00
gindibay f25b173ad5 Changes EnableCreateDatabasePropagation to false 
to fix failing tests
 2023-10-04 15:06:22 +03:00
gindibay 1b8c400821 Fixes sql comments 2023-10-04 13:40:37 +03:00
gindibay 90ead5dd19 Fixes indentation 2023-10-04 13:32:39 +03:00
gindibay c1aca5ef89 Removes unnecessary code 2023-10-04 13:23:28 +03:00
gindibay 90547e6cd5 Removes unnecessary files 2023-10-04 13:23:28 +03:00
gindibay 84826dcc98 Fixes runtime and compile errors 2023-10-04 13:23:28 +03:00
gindibay 8435509347 Fix for Pg16 compile error 2023-10-04 13:23:28 +03:00
gindibay 8c93c65351 Fixes compile errors 2023-10-04 13:23:28 +03:00
gindibay ae9cd9c895 Fixes additional compile errors 2023-10-04 13:23:28 +03:00
gindibay 226696e42a Adds first commit 2023-10-04 13:23:28 +03:00
Önder Kalacı 0dca65c84d
Addd missing image to Technical Readme (#7243) 
DESCRIPTION: PR description that will go into the change log, up to 78
characters
 2023-09-29 22:24:10 +02:00
Önder Kalacı 185ac5e01e
Citus Technical Readme (#7207) 
This commit aims to add a comprehensive guide that covers all essential
aspects of Citus, including planning, execution, locking mechanisms,
shard moves, 2PC, and many other major components of Citus.

Co-authored-by: Marco Slot <marco.slot@gmail.com>
 2023-09-29 16:50:52 +03:00
dependabot[bot] c323f49e83
Bump cryptography from 41.0.3 to 41.0.4 in /src/test/regress (#7231) 
Bumps [cryptography](https://github.com/pyca/cryptography) from 41.0.3
to 41.0.4.

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Nils Dijk <nils@citusdata.com>
 2023-09-27 15:36:58 +02:00
Onur Tirtir 27ac44eb2a
Fix mixed Citus upgrade tests (#7218) 
When testing rolling Citus upgrades, coordinator should not be upgraded
until we upgrade all the workers.

---------

Co-authored-by: Jelte Fennema-Nio <github-tech@jeltef.nl>
 2023-09-26 17:52:52 +03:00
Nils Dijk b87fbcbf79
Shard moves/isolate report LSN's in lsn format (#7227) 
DESCRIPTION: Shard moves/isolate report LSN's in lsn format

While investigating an issue with our catchup mechanism on certain
postgres versions we noticed we print LSN's in the format of the native
long type. This is an uncommon representation for LSN's in postgres
logs.

This patch changes the output of our log message to go from the long
type representation to the native LSN type representation. Making it
easier for postgres users to recognize and compare LSN's with other
related reports.

example of new output:
```
2023-09-25 17:28:47.544 CEST [11345] LOG:  The LSN of the target subscriptions on node localhost:9701 have increased from 0/0 to 0/E1ED20F8 at 2023-09-25 17:28:47.544165+02 where the source LSN is 1/415DCAD0
```
 2023-09-26 13:47:50 +02:00
Gürkan İndibay 7fa109c977
Adds alter user missing features (#7204) 
DESCRIPTION: Adds alter user rename propagation and enriches alter user
tests

---------

Co-authored-by: Jelte Fennema <github-tech@jeltef.nl>
 2023-09-26 12:28:07 +03:00
Onur Tirtir 111b4c19bc
Make sure to disallow creating a replicated distributed table concurrently (#7219) 
See explanation in https://github.com/citusdata/citus/issues/7216.
Fixes https://github.com/citusdata/citus/issues/7216.

DESCRIPTION: Makes sure to disallow creating a replicated distributed
table concurrently
 2023-09-25 11:14:35 +03:00
Nils Dijk 0f28a69f12
Use the $(DLSUFFIX) instead of hard coded extensions for cdc (#7221) 
When cdc got added the makefiles hardcoded the `.so` extension instead
of using the platform specifc `$(DLSUFFIX)` variable used by `pgxs.mk`.
Also don't remove installed cdc artifacts on `make clean`.
 2023-09-22 16:24:18 +02:00
Jelte Fennema-Nio 71e556e090
Remove useless test output (#7209) 
This was sometimes failing when running locally due to some local shard
still existing due to. This fixes that. We normally silence all
`drop schema cascade` output like this anyway to avoid unnecessary
diffs when modifying a test later on.
 2023-09-19 14:12:46 +02:00
Naisila Puka 4e46708789
Adds PostgreSQL 16.0 Support (#7201) 
This commit concludes PG16.0 Support in Citus.

The main PG16 support work has been done for 16beta3
https://github.com/citusdata/citus/pull/6952
There was some extra work needed for 16rc1
https://github.com/citusdata/citus/pull/7173
And this PR yet introduces some extra work needed to 16.0 :)

`pgstat_fetch_stat_local_beentry` has been renamed to
`pgstat_get_local_beentry_by_index` in PG16.0

Relevant PG commit:
8dfa37b797
8dfa37b797843a83a5756ea3309055e8953e1a86

Sister PR
https://github.com/citusdata/the-process/pull/150
 2023-09-15 12:23:04 +03:00
Gürkan İndibay 7c0b289761
Adds alter database set option (#7181) 
DESCRIPTION: Adds support for ALTER DATABASE <db_name> SET .. statement
propagation
SET statements in Postgres has a common structure which is already being
used in Alter Function
statement. 
In this PR, I added a util file; citus_setutils and made it usable for
both for
alter database<db_name>set .. and alter function ... set ... statements.
With this PR, below statements will be propagated
```sql
ALTER DATABASE name SET configuration_parameter { TO | = } { value | DEFAULT }
ALTER DATABASE name SET configuration_parameter FROM CURRENT
ALTER DATABASE name RESET configuration_parameter
ALTER DATABASE name RESET ALL
```
Additionally, there was a bug in processing float values in the common
code block.
I fixed this one as well

Previous
```C
case T_Float:
			{
				appendStringInfo(buf, " %s", strVal(value));
				break;
			}
```
Now
```C
case T_Float:
			{
				appendStringInfo(buf, " %s", nodeToString(value));
				break;
			}
```
 2023-09-14 16:29:16 +03:00
aykut-bozkurt 26dc407f4a
bump citus and columnar into 12.2devel (#7200) 2023-09-14 12:03:09 +03:00
Gürkan İndibay e5e64b7454
Adds alter database propagation - with and refresh collation (#7172) 
DESCRIPTION: Adds ALTER DATABASE WITH ... and REFRESH COLLATION VERSION
support

This PR adds supports for basic ALTER DATABASE statements propagation 
support. Below statements are supported:
ALTER DATABASE <database_name> with IS_TEMPLATE <true/false>;
ALTER DATABASE <database_name> with CONNECTION LIMIT <integer_value>;
ALTER DATABASE <database_name> REFRESH COLLATION VERSION;

---------

Co-authored-by: Jelte Fennema-Nio <jelte.fennema@microsoft.com>
 2023-09-12 14:09:15 +03:00
Naisila Puka 1da99f8423
PG16 - Don't propagate GRANT ROLE with INHERIT/SET option (#7190) 
We currently don't support propagating these options in Citus
Relevant PG commits:
https://github.com/postgres/postgres/commit/e3ce2de
https://github.com/postgres/postgres/commit/3d14e17

Limitation:
We also need to take care of generated GRANT statements by dependencies
in attempt to distribute something else. Specifically, this part of the
code in `GenerateGrantRoleStmtsOfRole`:
```
grantRoleStmt->admin_opt = membership->admin_option;
```
In PG16, membership also has `inherit_option` and `set_option` which
need to properly be part of the `grantRoleStmt`. We can skip for now
since #7164 will take care of this soon, and also this is not an
expected use-case.
 2023-09-12 12:47:37 +03:00
Naisila Puka c1dc378504
Fix WITH ADMIN FALSE propagation (#7191) 2023-09-11 15:58:24 +03:00
Onur Tirtir d628a4c21a
Add citus_schema_move() function (#7180) 
Add citus_schema_move() that can be used to move tenant tables within a distributed
schema to another node. The function has two variations as simple wrappers around
citus_move_shard_placement() and citus_move_shard_placement_with_nodeid() respectively.
They pick a shard that belongs to the given tenant schema and resolve the source node
that contain the shards under given tenant schema. Hence their signatures are quite
similar to underlying functions:

```sql
-- citus_schema_move(), using target node name and node port
CREATE OR REPLACE FUNCTION pg_catalog.citus_schema_move(
	schema_id regnamespace,
	target_node_name text,
	target_node_port integer,
	shard_transfer_mode citus.shard_transfer_mode default 'auto')
RETURNS void
LANGUAGE C STRICT
AS 'MODULE_PATHNAME', $$citus_schema_move$$;

-- citus_schema_move(), using target node id
CREATE OR REPLACE FUNCTION pg_catalog.citus_schema_move(
	schema_id regnamespace,
	target_node_id integer,
	shard_transfer_mode citus.shard_transfer_mode default 'auto')
RETURNS void
LANGUAGE C STRICT
AS 'MODULE_PATHNAME', $$citus_schema_move_with_nodeid$$;
```
 2023-09-08 12:03:53 +03:00
Naisila Puka 8894c76ec0
PG16 - Add rules option to CREATE COLLATION (#7185) 
Relevant PG commit:
https://github.com/postgres/postgres/commit/30a53b7
30a53b7
 2023-09-07 13:50:47 +03:00
Naisila Puka 2df88042b3
Add tests with JSON_ARRAYAGG and JSON_OBJECTAGG aggregates (#7186) 
Relevant PG commit:
7081ac46ac
7081ac46ace8c459966174400b53418683c9fe5c
 2023-09-07 13:29:39 +03:00
Naisila Puka 7e5136f2de
Add tests with publications with schema and table of the same schema (#7184) 
Relevant PG commit:
https://github.com/postgres/postgres/commit/13a185f
13a185f

It was backpatched through PG15 so I added this test in publication.sql
instead of pg16.sql
 2023-09-06 16:40:36 +03:00
Naisila Puka b2fc763bc3
PG16 - Add tests with random_normal (#7183) 
Relevant PG commit:
https://github.com/postgres/postgres/commit/38d8176
 2023-09-06 14:57:24 +03:00
Naisila Puka 5c658b4eb7
PG16 - Add citus_truncate_trigger for Citus foreign tables (#7170) 
Since in PG16, truncate triggers are supported on foreign tables, we add
the citus_truncate_trigger to Citus foreign tables as well, such that the TRUNCATE
command is propagated to the table's single local shard as well.
Note that TRUNCATE command was working for foreign tables even before this
commit: see https://github.com/citusdata/citus/pull/7170#issuecomment-1706240593 for details

This commit also adds tests with user-enabled truncate triggers on Citus foreign tables:
both trigger on the shell table and on its single foreign local shard.

Relevant PG commit:
https://github.com/postgres/postgres/commit/3b00a94
 2023-09-05 19:42:39 +03:00
zhjwpku 205b159606
get rid of {Push/Pop}OverrideSearchPath (#7145) 2023-09-05 17:40:22 +02:00
aykut-bozkurt 8eb3360017
Fixes visibility problems with dependency propagation (#7028) 
**Problem:**
Previously we always used an outside superuser connection to overcome
permission issues for the current user while propagating dependencies.
That has mainly 2 problems:
1. Visibility issues during dependency propagation, (metadata connection
propagates some objects like a schema, and outside transaction does not
see it and tries to create it again)
2. Security issues (it is preferrable to use current user's connection
instead of extension superuser)

**Solution (high level):**
Now, we try to make a smarter decision on whether should we use an
outside superuser connection or current user's metadata connection. We
prefer using current user's connection if any of the objects, which is
already propagated in the current transaction, is a dependency for a
target object. We do that since we assume if current user has
permissions to create the dependency, then it can most probably
propagate the target as well.

Our assumption is expected to hold most of the times but it can still be
wrong. In those cases, transaction would fail and user should set the
GUC `citus.create_object_propagation` to `deferred` to work around it.

**Solution:**
1. We track all objects propagated in the current transaction (we can
handle subtransactions),
2. We propagate dependencies via the current user's metadata connection
if any dependency is created in the current transaction to address
issues listed above. Otherwise, we still use an outside superuser
connection.


DESCRIPTION: Fixes some object propagation errors seen with transaction
blocks.

Fixes https://github.com/citusdata/citus/issues/6614

---------

Co-authored-by: Nils Dijk <nils@citusdata.com>
 2023-09-05 18:04:16 +03:00
Naisila Puka 9f067731c0
Adds PostgreSQL 16 RC1 support (#7173) 2023-09-05 14:32:41 +03:00
Emel Şimşek a849570f3f
Improve the performance of CitusHasBeenLoaded function for a database that does not do CREATE EXTENSION citus but load citus.so. (#7123) 
For a database that does not create the citus extension by running

`  CREATE EXTENSION citus;`

`CitusHasBeenLoaded ` function ends up querying the `pg_extension` table
every time it is invoked. This is not an ideal situation for a such a
database.

The idea in this PR is as follows:

### A new field in MetadataCache.
 Add a new variable `extensionCreatedState `of the following type:

```
typedef enum ExtensionCreatedState
{
        UNKNOWN = 0,
        CREATED = 1,
        NOTCREATED = 2,
} ExtensionCreatedState;
```
When the MetadataCache is invalidated, `ExtensionCreatedState` will be
set to UNKNOWN.
     
### Invalidate MetadataCache when CREATE/DROP/ALTER EXTENSION citus
commands are run.

- Register a callback function, named
`InvalidateDistRelationCacheCallback`, for relcache invalidation during
the shared library initialization for `citus.so`. This callback function
is invoked in all the backends whenever the relcache is invalidated in
one of the backends. (This could be caused many DDLs operations).

- In the cache invalidation callback,`
InvalidateDistRelationCacheCallback`, invalidate `MetadataCache` zeroing
it out.
 
- In `CitusHasBeenLoaded`, perform the costly citus is loaded check only
if the `MetadataCache` is not valid.
 
### Downsides

Any relcache invalidation (caused by various DDL operations) will case
Citus MetadataCache to get invalidated. Most of the time it will be
unnecessary. But we rely on that DDL operations on relations will not be
too frequent.
 2023-09-05 13:29:35 +03:00