external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,118 Commits
645 Branches
162 Tags
103 MiB
Commit Graph

61 Commits (8a866a3d5e5adc3e81dd49cb85e316a7ad934d36)

Author SHA1 Message Date
Sameer Awasekar 8a866a3d5e Fix review comments 2022-08-04 18:22:01 +05:30
Sameer Awasekar 00c3830bee 1)Revert the original order of workflow 
2) Introduce GetNextShardIdForSplitChild method
 2022-08-04 16:26:15 +05:30
Sameer Awasekar ad240b2904 Reorder split workflow to create empty publications first and then alter to add actual shards. 2022-08-04 12:42:53 +05:30
Sameer Awasekar 9029107508 Fix upgrade test 2022-08-03 15:12:13 +05:30
Sameer Awasekar 15014f405f Merge remote-tracking branch 'origin' into users/saawasek/non_blocking_split_integrated 2022-08-03 08:17:19 +05:30
Sameer Awasekar d6258fe0db Address review comments. 
1) Add partitioning test
2) Refactor some methods
 2022-08-02 19:09:32 +05:30
Sameer Awasekar a430b0bffc Address review comments 2022-08-01 23:58:20 +05:30
Sameer Awasekar c56b79b6f7 Address review comments 2022-08-01 12:54:30 +05:30
Sameer Awasekar adf73387f4 1. Added an isolation test 
2. Added DisableAndDropShardSplitSubscription
3. Added creation of replica identities and validated through isolation test
 2022-07-30 13:36:10 +05:30
Onder Kalaci 5bc8a81aa7 Add colocation checks for shard splits 2022-07-27 10:01:19 +02:00
Sameer Awasekar 0f72d7f706 Rename connection params 2022-07-24 16:34:16 +05:30
Sameer Awasekar f6c11bf09c Reindent 2022-07-24 15:16:44 +05:30
Sameer Awasekar 9073564900 Adding comments to the workflow 2022-07-24 14:16:37 +05:30
Sameer Awasekar b4f8226f96 Add comments for the functions 2022-07-23 22:58:00 +05:30
Sameer Awasekar 5a5505db55 Add comments for functions 2022-07-23 15:47:14 +05:30
Sameer Awasekar 8c871bcd10 Adding comments and code for cleanup 2022-07-22 20:10:18 +05:30
Sameer Awasekar 9e88c14096 Merge remote-tracking branch 'origin' into users/saawasek/non_blocking_split_integrated 2022-07-21 11:19:27 +05:30
Nitish Upreti 3d569cc49a
Shard Split support for Columnar and Partitioned Table (#6067) 
DESCRIPTION:
This PR extends support for Partitioned and Columnar tables in blocking 'citus_split_shard_by_split_points' workflow.
Columnar Support : No special handling required. Just removing checks that fails split for columnar table and adding test coverage.
Partitioned Table Support :

Skip copying of parent table as they are empty, The partitions contain data and are treated as co-located shards that will be copied separately.
Attach partitions to parent on destination after inserting new shard metadata and before creating foreign key constraints.
MISC:
Fix Bug #4949 where Blocking shard moves fails if there is a foreign key between partitioned distributed tables (from child to parent).

TEST:
Added new test 'citus_split_shards_columnar_partitioned' for splitting 'partitioned' and 'columnar + partitioned' table.
Added new test 'shard_move_constraints_blocking' to add coverage for shard move bug fix.
Updated test 'citus_split_shard_by_split_points_negative' to allow columnar and partitioned table.
 2022-07-20 12:24:50 -07:00
Sameer Awasekar a4bf6a1dbb Change version of worker_split_shard_replication_setup UDF 2022-07-19 21:02:35 +05:30
Nitish Upreti 5b3537cdff
Shard Split for Citus (#6029) 
* Blocking split setup

* Add missing type

* Missing API from Metadata Sync

* Shard Split e2e code

* Worker Split Copy DestReceiver skeleton

* Basic destreceiver code

* worker_split_copy UDF

* UDF calling

* Split points are text

* Isolate Tenant and Split Shard Unification

* Fixing executor and misc

* Reindent code

* Fixing UDF definitions

* Hello World Local Copy works

* Remote copy hello world works

* Local and Remote binary test

* Fixing text local copy and adding tests

* Hello World shard split works

* Negative tests

* Blocking Split workflow works

* Refactor

* Bug fix

* Reindent

* Cleaning up and adding comments

* Basic test for shard split workflow

* ReIndent

* Circle CI integration

* Removing include causing circle-ci build failure

* Remove SplitCopyDestReceiver and use PartitionedResultDestReceiver

* Add support for citus.enable_binary_protocol

* Reindent

* Fix build break

* Update Test

* Cleanup on catch

* Addressing open comments

* Update downgrade script and quote schema/table in COPY statement

* Fix metadata sync issue. Update regression test

* Isolation test and bug fix

* Add Isolation test, fix foreign constraint deadlock issue

* Misc code review comments

* Test name needing to be quoted

* Refactor code from review comments

* Explaining shardGroupSplitIntervalListList

* Fix upgrade & downgrade

* Fix broken test

* Test fix Round 2

* Fixing bug and modifying test appropriately

* Fully qualify copy udf name. Run Reindent

* Address PR comments

* Fix null handling when creating AuxiliaryStructures

* Ensure local copy is triggered in tests

* Limit max shards that can be created with split

* Test failure fix

* Remove split_mode and use shard_transfer_mode instead'

* Fix test failure

* Fix test failure

* Fixing permission issue when splitting non-superuser owned tables

* Fix test expected output

* Remove extra space

* Fix test

* attempt to fix test

* Addressing Marco's PR comment

* Only clean shards created by workflow

* Remove from merge

* Update test
 2022-07-18 02:54:15 -07:00
Sameer Awasekar 1243fe20ae Merge remote-tracking branch 'origin/users/niupre/BlockingSplitAPI' into users/saawasek/non_blocking_split_integrated 2022-07-18 14:27:59 +05:30
Sameer Awasekar 10ea295d6c Flow completed 2022-07-18 13:51:58 +05:30
Nitish Upreti 1c16060bd6 Only clean shards created by workflow 2022-07-18 00:47:42 -07:00
Sameer Awasekar b26bab32de Refactor subscriber code 2022-07-17 15:32:34 +05:30
Sameer Awasekar 1c617e7d1d Added code to drop publications and subscriptions 2022-07-16 15:38:51 +05:30
Nitish Upreti b1405d5eaf Addressing Marco's PR comment 2022-07-14 13:21:19 -07:00
Sameer Awasekar 2fa1dac051 Add: 
1) Drop colocated shards
2) Swap Metadata
3) Drop dummy shards
 2022-07-14 16:23:00 +05:30
Sameer Awasekar 662a4aaec1 Merge remote-tracking branch 'origin/users/niupre/BlockingSplitAPI' into users/saawasek/non_blocking_split_integrated 2022-07-14 09:52:27 +05:30
Sameer Awasekar 028abaa3cb Add: 
1. Add publication/subscription logic
2. Add Catchup code
3. Blockwrites
4. Create Indexs, Replica Identity
 2022-07-14 09:25:07 +05:30
Nitish Upreti aca0a03d21 Fixing permission issue when splitting non-superuser owned tables 2022-07-13 14:09:02 -07:00
Sameer Awasekar 51242a21d7 Added pub/sub 2022-07-13 13:16:01 +05:30
Nitish Upreti c4a0d55b45 Limit max shards that can be created with split 2022-07-12 13:42:35 -07:00
Nitish Upreti 8f8761ed40 Fix null handling when creating AuxiliaryStructures 2022-07-11 17:45:32 -07:00
Nitish Upreti cf8fea5a3f Address PR comments 2022-07-11 15:44:48 -07:00
Sameer Awasekar fd46d8011d Merge remote-tracking branch 'origin/users/niupre/BlockingSplitAPI' into users/saawasek/non_blocking_split_integrated 2022-07-10 19:35:36 +05:30
Sameer Awasekar 0d26c4c8c7 Non-Blocking Shard Split 2022-07-10 19:14:41 +05:30
Nitish Upreti 5a1a8d5dcb Fully qualify copy udf name. Run Reindent 2022-07-09 16:35:10 -07:00
Nitish Upreti 5c635511d9 Explaining shardGroupSplitIntervalListList 2022-07-08 22:45:07 -07:00
Nitish Upreti a30c791eeb Refactor code from review comments 2022-07-08 22:23:38 -07:00
Nitish Upreti a515a49f4c Misc code review comments 2022-07-08 19:07:25 -07:00
Nitish Upreti f8c2c2555f Add Isolation test, fix foreign constraint deadlock issue 2022-07-08 15:03:46 -07:00
Nitish Upreti fb42c17ac3 Isolation test and bug fix 2022-07-08 11:52:36 -07:00
Nitish Upreti 55770d2816 Fix metadata sync issue. Update regression test 2022-07-06 21:27:25 -07:00
Nitish Upreti 4fcdff53ae Addressing open comments 2022-06-28 16:45:26 -07:00
Nitish Upreti ebd0e9a617 Cleanup on catch 2022-06-28 16:22:37 -07:00
Nitish Upreti 383fc2b6f0 Fix build break 2022-06-28 09:37:11 -07:00
Nitish Upreti a836a322c4 Cleaning up and adding comments 2022-06-24 11:54:07 -07:00
Nitish Upreti 8b4956e9e9 Reindent 2022-06-24 09:56:06 -07:00
Nitish Upreti f3a391b80f Bug fix 2022-06-23 22:14:44 -07:00
Nitish Upreti 2f3f212746 Refactor 2022-06-23 18:38:55 -07:00