external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,098 Commits
645 Branches
162 Tags
103 MiB
Commit Graph

2872 Commits (15014f405f3385cc2472635e26b6d36d3a13dd32)

Author SHA1 Message Date
aykutbozkurt 9d232d7b00 change address method to return list of addresses 2022-07-19 18:13:11 +03:00
Önder Kalacı 90b1afe31e
Merge branch 'main' into baby_step_pg_15 2022-07-18 15:02:39 +02:00
Nitish Upreti 5b3537cdff
Shard Split for Citus (#6029) 
* Blocking split setup

* Add missing type

* Missing API from Metadata Sync

* Shard Split e2e code

* Worker Split Copy DestReceiver skeleton

* Basic destreceiver code

* worker_split_copy UDF

* UDF calling

* Split points are text

* Isolate Tenant and Split Shard Unification

* Fixing executor and misc

* Reindent code

* Fixing UDF definitions

* Hello World Local Copy works

* Remote copy hello world works

* Local and Remote binary test

* Fixing text local copy and adding tests

* Hello World shard split works

* Negative tests

* Blocking Split workflow works

* Refactor

* Bug fix

* Reindent

* Cleaning up and adding comments

* Basic test for shard split workflow

* ReIndent

* Circle CI integration

* Removing include causing circle-ci build failure

* Remove SplitCopyDestReceiver and use PartitionedResultDestReceiver

* Add support for citus.enable_binary_protocol

* Reindent

* Fix build break

* Update Test

* Cleanup on catch

* Addressing open comments

* Update downgrade script and quote schema/table in COPY statement

* Fix metadata sync issue. Update regression test

* Isolation test and bug fix

* Add Isolation test, fix foreign constraint deadlock issue

* Misc code review comments

* Test name needing to be quoted

* Refactor code from review comments

* Explaining shardGroupSplitIntervalListList

* Fix upgrade & downgrade

* Fix broken test

* Test fix Round 2

* Fixing bug and modifying test appropriately

* Fully qualify copy udf name. Run Reindent

* Address PR comments

* Fix null handling when creating AuxiliaryStructures

* Ensure local copy is triggered in tests

* Limit max shards that can be created with split

* Test failure fix

* Remove split_mode and use shard_transfer_mode instead'

* Fix test failure

* Fix test failure

* Fixing permission issue when splitting non-superuser owned tables

* Fix test expected output

* Remove extra space

* Fix test

* attempt to fix test

* Addressing Marco's PR comment

* Only clean shards created by workflow

* Remove from merge

* Update test
 2022-07-18 02:54:15 -07:00
Sameer Awasekar 1243fe20ae Merge remote-tracking branch 'origin/users/niupre/BlockingSplitAPI' into users/saawasek/non_blocking_split_integrated 2022-07-18 14:27:59 +05:30
Sameer Awasekar 10ea295d6c Flow completed 2022-07-18 13:51:58 +05:30
Nitish Upreti 5fe9bac4b9
Merge branch 'main' into users/niupre/BlockingSplitAPI 2022-07-18 01:02:12 -07:00
Nitish Upreti 1c16060bd6 Only clean shards created by workflow 2022-07-18 00:47:42 -07:00
Sameer Awasekar b26bab32de Refactor subscriber code 2022-07-17 15:32:34 +05:30
Sameer Awasekar 1c617e7d1d Added code to drop publications and subscriptions 2022-07-16 15:38:51 +05:30
Onder Kalaci 3eaef027e2 Remove unused code 
Probably left over from removing old repartitioning code
 2022-07-15 10:28:46 +02:00
Onder Kalaci 483a3a5875 PG 15 Compat: Resolve compile issues + shmem requests 
Similar to #5897, one more step for running Citus with PG 15.

This PR at least make Citus run with PG 15. I have not tried running the tests with PG 15.

Shmem changes are based on 4f2400cb3f

Compile breaks are mostly due to #6008
 2022-07-15 10:11:39 +02:00
Nitish Upreti b1405d5eaf Addressing Marco's PR comment 2022-07-14 13:21:19 -07:00
Nitish Upreti cb3ddf18a7 Merge branch 'main' into users/niupre/BlockingSplitAPI 2022-07-14 09:49:41 -07:00
Sameer Awasekar 2fa1dac051 Add: 
1) Drop colocated shards
2) Swap Metadata
3) Drop dummy shards
 2022-07-14 16:23:00 +05:30
Sameer Awasekar 662a4aaec1 Merge remote-tracking branch 'origin/users/niupre/BlockingSplitAPI' into users/saawasek/non_blocking_split_integrated 2022-07-14 09:52:27 +05:30
ywj 1675519f93
Support citus_columnar as separate extension (#5911) 
* Support upgrade and downgrade and separate columnar as citus_columnar extension

Co-authored-by: Yanwen Jin <yanwjin@microsoft.com>
Co-authored-by: Jeff Davis <jeff@j-davis.com>
 2022-07-13 21:08:29 -07:00
Sameer Awasekar 028abaa3cb Add: 
1. Add publication/subscription logic
2. Add Catchup code
3. Blockwrites
4. Create Indexs, Replica Identity
 2022-07-14 09:25:07 +05:30
Nitish Upreti aca0a03d21 Fixing permission issue when splitting non-superuser owned tables 2022-07-13 14:09:02 -07:00
Onder Kalaci b2e9a5baf1 Make sure citus_is_coordinator works on read replicas 2022-07-13 14:11:18 +02:00
Onder Kalaci 8ab696f7e2 LOCK COMMAND does not require primaries at the start 2022-07-13 14:08:49 +02:00
Sameer Awasekar 51242a21d7 Added pub/sub 2022-07-13 13:16:01 +05:30
Nitish Upreti 8244db7d7c Fix test failure 2022-07-12 15:54:04 -07:00
Nitish Upreti 6fc7544f18 Remove split_mode and use shard_transfer_mode instead' 2022-07-12 15:42:20 -07:00
Nitish Upreti c4a0d55b45 Limit max shards that can be created with split 2022-07-12 13:42:35 -07:00
Nitish Upreti 0445d72cc7 Ensure local copy is triggered in tests 2022-07-12 10:38:18 -07:00
Nitish Upreti 8f8761ed40 Fix null handling when creating AuxiliaryStructures 2022-07-11 17:45:32 -07:00
Nitish Upreti cf8fea5a3f Address PR comments 2022-07-11 15:44:48 -07:00
Sameer Awasekar fd46d8011d Merge remote-tracking branch 'origin/users/niupre/BlockingSplitAPI' into users/saawasek/non_blocking_split_integrated 2022-07-10 19:35:36 +05:30
Sameer Awasekar 0d26c4c8c7 Non-Blocking Shard Split 2022-07-10 19:14:41 +05:30
Nitish Upreti 5a1a8d5dcb Fully qualify copy udf name. Run Reindent 2022-07-09 16:35:10 -07:00
Nitish Upreti a6ca23feda Fixing bug and modifying test appropriately 2022-07-09 16:27:53 -07:00
Nitish Upreti 7eafc95fd3 Fix broken test 2022-07-08 23:50:22 -07:00
Nitish Upreti f5f352c620 Fix upgrade & downgrade 2022-07-08 23:13:49 -07:00
Nitish Upreti eb1e2f1450 Merging with main 2022-07-08 22:57:11 -07:00
Nitish Upreti 5c635511d9 Explaining shardGroupSplitIntervalListList 2022-07-08 22:45:07 -07:00
Nitish Upreti a30c791eeb Refactor code from review comments 2022-07-08 22:23:38 -07:00
Nitish Upreti 7ef8ea948d Test name needing to be quoted 2022-07-08 21:44:35 -07:00
Nitish Upreti a515a49f4c Misc code review comments 2022-07-08 19:07:25 -07:00
Nitish Upreti f8c2c2555f Add Isolation test, fix foreign constraint deadlock issue 2022-07-08 15:03:46 -07:00
Nitish Upreti fb42c17ac3 Isolation test and bug fix 2022-07-08 11:52:36 -07:00
Nitish Upreti 55770d2816 Fix metadata sync issue. Update regression test 2022-07-06 21:27:25 -07:00
aykutbozkurt da089d72c5 we should check if relation is valid after fetching a relation 2022-07-06 16:35:01 +03:00
Halil Ozan Akgul 1490acbbe9 Removes incorrect parameter from get_all_active_transactions 2022-07-06 11:35:46 +03:00
Sameer Awasekar ea9eb103e0 Fix nits 2022-07-06 11:22:19 +05:30
Sameer Awasekar b6662ff872 Add hashmap, cleanup methods 2022-07-06 11:22:19 +05:30
Sameer Awasekar 213b15071b Rename methods 2022-07-06 11:22:19 +05:30
Sameer Awasekar 6d55154529 Remove pg_sleep and address nits 2022-07-06 11:22:19 +05:30
Sameer Awasekar 99a21757d4 Fix name 2022-07-06 11:22:19 +05:30
Sameer Awasekar 0b11a324a4 Change value of static variable 2022-07-06 11:22:15 +05:30
Sameer Awasekar 91c9394ec4 Remove '.so' of plugin and have it embedded within Citus 2022-07-06 11:21:46 +05:30
Sameer Awasekar 23237e50a2 Add UDF description 2022-07-06 11:21:46 +05:30
Sameer Awasekar a23beeb43f Change return type, shard range as text 2022-07-06 11:21:43 +05:30
Sameer Awasekar b66067d09f Addressing review comments 
1. Changed SM memory header
2. Changed API prototype
3. Refactored testcase using gset
 2022-07-06 11:21:04 +05:30
Sameer Awasekar 5b82fd2ea3 Address review comments 2022-07-06 11:20:37 +05:30
Sameer Awasekar 672d198019 Renamed UDF to worker_split_shard_replication_setup 2022-07-06 11:20:34 +05:30
Sameer Awasekar 01540d78d6 Rename method to IsCommitRecursive 2022-07-06 11:15:23 +05:30
Sameer Awasekar f03a0343bc fix indentations 2022-07-06 11:15:23 +05:30
Sameer Awasekar f1be4888b9 Address comments 
1) Created new schema for tests
2) Variable renaming
 2022-07-06 11:15:18 +05:30
Sameer Awasekar 65eb62e723 Change encoding of replication slot names. Include table owners 2022-07-06 11:14:52 +05:30
Sameer Awasekar 04c2a92455 Reduced test timings 2022-07-06 11:14:52 +05:30
Sameer Awasekar 5a711a9176 Address comments 
- Fix warnings
 2022-07-06 11:14:52 +05:30
Sameer Awasekar 60ee33cfcc Add testcase and handling of UPDATE for non-partition column. 
- Change nodeId to uint32
- some test enchancement
- fix comments
 2022-07-06 11:14:52 +05:30
Sameer Awasekar 3864cd3187 Rename file to split_shard_replication_setup.c 2022-07-06 11:14:52 +05:30
Sameer Awasekar cbc4a4a11c Change datatype from integer to bigint 2022-07-06 11:14:52 +05:30
Sameer Awasekar daae0bb10d Added colocated shard test 
- added a new test helper file
 2022-07-06 11:14:48 +05:30
Sameer Awasekar d71ead0e43 Remove unwanted headers 2022-07-06 11:14:02 +05:30
Sameer Awasekar 5da75b84ac Removed some methods. Handling review comments 2022-07-06 11:14:02 +05:30
Sameer Awasekar 4b064fa321 change name of logical decoding plugin 2022-07-06 11:14:02 +05:30
Sameer Awasekar 25bae46cd6 Fix identations 2022-07-06 11:14:02 +05:30
Sameer Awasekar 121ec8cd51 Change prototype of GetShardSplitInfoSMHeaderFromDSMHandle 2022-07-06 11:14:02 +05:30
Sameer Awasekar 59c3c93aa4 Addressing comments 
- Changed name of functions
- Moved some functions from .h to .c file
 2022-07-06 11:14:02 +05:30
Sameer Awasekar 75c6484e02 Address review comments(partially done) 
- Parameterize testcase methods
- remove code for handline 'UPDATE'.
- replication slot encoding/decoding logic changed now
 2022-07-06 11:14:02 +05:30
Sameer Awasekar a43cab7e71 Add DELETE scenario 2022-07-06 11:08:16 +05:30
Sameer Awasekar c9844abea9 Introducing 'split_shard_replication_setup' UDF to handle the catchup phase of 
citus_split_shard.
 2022-07-06 11:08:10 +05:30
aykutbozkurt d53a7760b0 * alter index/table rename weird syntax supported, 
* correct the wrong level of lock if the weird syntax is used
 2022-07-04 21:27:47 +03:00
aykutbozkurt ba62c0a148 auto is a valid option for vacuum index_cleanup. 2022-07-04 19:27:55 +03:00
Ahmet Gedemenli c8e1e243b8
Fix matviews for citus_add_local_table_to_metadata (#6023) 2022-07-04 17:00:07 +03:00
Nitish Upreti e29222458c Update downgrade script and quote schema/table in COPY statement 2022-07-01 13:35:27 -07:00
Hanefi Onaldi f60809a6c1
Fix downgrade scripts from 11.0-2 to 11.0-1 (#6039) 2022-06-29 22:43:50 +03:00
Nitish Upreti 4fcdff53ae Addressing open comments 2022-06-28 16:45:26 -07:00
Nitish Upreti ebd0e9a617 Cleanup on catch 2022-06-28 16:22:37 -07:00
Nitish Upreti 383fc2b6f0 Fix build break 2022-06-28 09:37:11 -07:00
Onder Kalaci bab4c0a8c3 Fixes a bug that prevents upgrades when there are no worker nodes 2022-06-28 15:54:49 +02:00
Onder Kalaci bd3a070369 Fixes a bug that prevents upgrades when there COMPRESSION and DEFAULT columns 2022-06-28 13:36:00 +02:00
Nitish Upreti 529f1bc8e3 Reindent 2022-06-27 22:24:23 -07:00
Nitish Upreti 928fa56558 Add support for citus.enable_binary_protocol 2022-06-27 22:21:05 -07:00
Nitish Upreti 75ae1d0265 Remove SplitCopyDestReceiver and use PartitionedResultDestReceiver 2022-06-27 13:45:58 -07:00
Nitish Upreti 0fde80c133 Removing include causing circle-ci build failure 2022-06-24 15:47:42 -07:00
Nitish Upreti 47e67e81e6 ReIndent 2022-06-24 13:48:43 -07:00
Nitish Upreti a836a322c4 Cleaning up and adding comments 2022-06-24 11:54:07 -07:00
Nitish Upreti 8b4956e9e9 Reindent 2022-06-24 09:56:06 -07:00
Nitish Upreti f3a391b80f Bug fix 2022-06-23 22:14:44 -07:00
Nitish Upreti 2f3f212746 Refactor 2022-06-23 18:38:55 -07:00
Nitish Upreti 686ce21e80 Blocking Split workflow works 2022-06-23 18:35:34 -07:00
aykutbozkurt 8194dc4c62 * Added isolation tests for vacuum, 
* Added more regression tests for more vacuum options,
* Fixed deadlock for unqualified vacuum when there is only 1 worker,
* Supported lock_skipped for vacuum.
 2022-06-23 15:33:14 +03:00
aykutbozkurt 1d6c81245c fix bug, which is column mismatch of shard tasks when specifying column names for citus tables in vacuum and analyze commands 2022-06-23 15:33:14 +03:00
Aykut Bozkurt 6986f53835 propagate unqualified vacuum and analyze to all worker nodes 2022-06-23 15:33:14 +03:00
Nitish Upreti aa047bda16 Negative tests 2022-06-22 16:42:27 -07:00
Nitish Upreti ac29dbabf4 Hello World shard split works 2022-06-22 15:01:00 -07:00
Nitish Upreti a8e3344a40 Fixing text local copy and adding tests 2022-06-22 13:51:30 -07:00