Commit Graph

152 Commits (d71ead0e43cd6b9cc698483b1863cd6194e2508e)

Author SHA1 Message Date
Sameer Awasekar d71ead0e43 Remove unwanted headers 2022-07-06 11:14:02 +05:30
Sameer Awasekar 5da75b84ac Removed some methods. Handling review comments 2022-07-06 11:14:02 +05:30
Sameer Awasekar 4b064fa321 change name of logical decoding plugin 2022-07-06 11:14:02 +05:30
Sameer Awasekar 59c3c93aa4 Addressing comments
- Changed name of functions
- Moved some functions from .h to .c file
2022-07-06 11:14:02 +05:30
Sameer Awasekar 75c6484e02 Address review comments(partially done)
- Parameterize testcase methods
- remove code for handline 'UPDATE'.
- replication slot encoding/decoding logic changed now
2022-07-06 11:14:02 +05:30
Sameer Awasekar c9844abea9 Introducing 'split_shard_replication_setup' UDF to handle the catchup phase of
citus_split_shard.
2022-07-06 11:08:10 +05:30
Nitish Upreti e29222458c Update downgrade script and quote schema/table in COPY statement 2022-07-01 13:35:27 -07:00
Nitish Upreti 4fcdff53ae Addressing open comments 2022-06-28 16:45:26 -07:00
Nitish Upreti ebd0e9a617 Cleanup on catch 2022-06-28 16:22:37 -07:00
Nitish Upreti 383fc2b6f0 Fix build break 2022-06-28 09:37:11 -07:00
Nitish Upreti 529f1bc8e3 Reindent 2022-06-27 22:24:23 -07:00
Nitish Upreti 928fa56558 Add support for citus.enable_binary_protocol 2022-06-27 22:21:05 -07:00
Nitish Upreti 75ae1d0265 Remove SplitCopyDestReceiver and use PartitionedResultDestReceiver 2022-06-27 13:45:58 -07:00
Nitish Upreti 47e67e81e6 ReIndent 2022-06-24 13:48:43 -07:00
Nitish Upreti a836a322c4 Cleaning up and adding comments 2022-06-24 11:54:07 -07:00
Nitish Upreti 8b4956e9e9 Reindent 2022-06-24 09:56:06 -07:00
Nitish Upreti f3a391b80f Bug fix 2022-06-23 22:14:44 -07:00
Nitish Upreti 2f3f212746 Refactor 2022-06-23 18:38:55 -07:00
Nitish Upreti 686ce21e80 Blocking Split workflow works 2022-06-23 18:35:34 -07:00
Nitish Upreti aa047bda16 Negative tests 2022-06-22 16:42:27 -07:00
Nitish Upreti ac29dbabf4 Hello World shard split works 2022-06-22 15:01:00 -07:00
Nitish Upreti a8e3344a40 Fixing text local copy and adding tests 2022-06-22 13:51:30 -07:00
Nitish Upreti bb2f72f91e Remote copy hello world works 2022-06-20 14:20:33 -07:00
Nitish Upreti 91ca1fb0b4 Hello World Local Copy works 2022-06-20 11:53:22 -07:00
Nitish Upreti 58facee439 Fixing UDF definitions 2022-06-19 17:41:38 -07:00
Nitish Upreti 29f0081169 Reindent code 2022-06-19 17:18:28 -07:00
Nitish Upreti 4a05f1f1e8 Fixing executor and misc 2022-06-19 17:07:26 -07:00
Nitish Upreti c38de446bb Isolate Tenant and Split Shard Unification 2022-06-17 15:54:50 -07:00
Nitish Upreti b502526ef1 Split points are text 2022-06-17 11:53:16 -07:00
Nitish Upreti 3e7b3e8f2f Merge branch 'master' into users/niupre/BlockingSplitAPI 2022-06-17 09:54:29 -07:00
Nitish Upreti 0f1d2bbd1a UDF calling 2022-06-16 18:06:09 -07:00
Nitish Upreti 77253cdafb worker_split_copy UDF 2022-06-16 14:47:48 -07:00
Jelte Fennema 184c7c0bce
Make enterprise features open source (#6008)
This PR makes all of the features open source that were previously only
available in Citus Enterprise.

Features that this adds:
1. Non blocking shard moves/shard rebalancer
   (`citus.logical_replication_timeout`)
2. Propagation of CREATE/DROP/ALTER ROLE statements
3. Propagation of GRANT statements
4. Propagation of CLUSTER statements
5. Propagation of ALTER DATABASE ... OWNER TO ...
6. Optimization for COPY when loading JSON to avoid double parsing of
   the JSON object (`citus.skip_jsonb_validation_in_copy`)
7. Support for row level security
8. Support for `pg_dist_authinfo`, which allows storing different
   authentication options for different users, e.g. you can store
   passwords or certificates here.
9. Support for `pg_dist_poolinfo`, which allows using connection poolers
   in between coordinator and workers
10. Tracking distributed query execution times using
   citus_stat_statements (`citus.stat_statements_max`,
   `citus.stat_statements_purge_interval`,
   `citus.stat_statements_track`). This is disabled by default.
11. Blocking tenant_isolation
12. Support for `sslkey` and `sslcert` in `citus.node_conninfo`
2022-06-16 00:23:46 -07:00
Nitish Upreti ab9f92eaa3 Basic destreceiver code 2022-06-15 16:31:01 -07:00
Nitish Upreti d5ab34040b Worker Split Copy DestReceiver skeleton 2022-06-14 15:58:12 -07:00
Nitish Upreti f61a053f00 Shard Split e2e code 2022-06-14 12:17:50 -07:00
Nitish Upreti b5b14ed0bd Blocking split setup 2022-06-13 16:32:26 -07:00
jeff-davis a9f8a60007
Columnar: support relation options with ALTER TABLE. (#5935)
Columnar: support relation options with ALTER TABLE.

Use ALTER TABLE ... SET/RESET to specify relation options rather than
alter_columnar_table_set() and alter_columnar_table_reset().

Not only is this more ergonomic, but it also allows better integration
because it can be treated like DDL on a regular table. For instance,
citus can use its own ProcessUtility_hook to distribute the new
settings to the shards.

DESCRIPTION: Columnar: support relation options with ALTER TABLE.
2022-05-20 08:35:00 -07:00
Marco Slot ad5214b50c Allow distributed execution from run_command_on_* functions 2022-05-20 15:26:47 +02:00
Onder Kalaci b4dbd84743 Prevent distributed queries while disabling first worker node
First worker node has a special meaning for modifications on the replicated tables

It is used to acquire a remote lock, such that the modifications are serialized.

With this commit, we make sure that we do not let any distributed query to see a
different 'first worker node' while first worker node is disabled.

Note that, maybe implicitly mentioned above, when first worker node is disabled,
the first worker node changes, that's why we have to handle the situation.
2022-05-18 17:21:12 +02:00
Jeff Davis 26f5e20580 PG15: update integer parsing APIs.
Account for PG commits 3c6f8c011f and cfc7191dfe.
2022-05-02 10:12:03 -07:00
Marco Slot 9476f377b5 Remove old re-partitioning functions 2022-04-04 18:11:52 +02:00
jeff-davis c485a04139
Separate build of citus.so and citus_columnar.so. (#5805)
* Separate build of citus.so and citus_columnar.so.

Because columnar code is statically-linked to both modules, it doesn't
make sense to load them both at once.

A subsequent commit will make the modules entirely separate and allow
loading them both simultaneously.

Author: Yanwen Jin

* Separate citus and citus_columnar modules.

Now the modules are independent. Columnar can be loaded by itself, or
along with citus.

Co-authored-by: Jeff Davis <jefdavi@microsoft.com>
2022-03-31 19:47:17 -07:00
Gledis Zeneli b825232ecb
Handle rebalance / replication when a node is disabled (Fix #5664) (#5729)
The issue in question is caused when rebalance / replication call `FullShardPlacementList` which returns all shard placements (including those in disabled nodes with `citus_disable_node`).  Eventually, `FindFillStateForPlacement` looks for the state across active workers and fails to find a state for the placements which are in the disabled workers causing a seg fault shortly after.

Approach:
* `ActivePlacementHash` was not using the status of the shard placement's node to determine if the node it is active. Initially, I just fixed that.
* Additionally, I refactored the code which handles active shards in replication / rebalance to:
	* use a single function to determine if a shard placement is active. 
	* do the shard active shard filtering before calling `RebalancePlacementUpdates` and `ReplicationPlacementUpdates`, so test methods like `shard_placement_rebalance_array` and `shard_placement_replication_array` which have different shard placement active requirements can do their own filtering while using the same rebalance / replicate logic that `rebalance_table_shards` and `replicate_table_shards` use. 

Fix #5664
2022-02-25 19:54:30 +03:00
Halil Ozan Akgul f6cd4d0f07 Overrides pg_cancel_backend and pg_terminate_backend to accept global pid 2022-02-21 16:41:35 +03:00
Marco Slot 872f0a79db Remove random shard placement policy 2022-02-06 21:55:58 +01:00
Marco Slot 0cae8e7d6b Remove local-node-first shard placement 2022-02-06 21:36:34 +01:00
Onder Kalaci 72d7d92611 Apply code review feedback 2022-02-04 10:52:57 +01:00
Teja Mupparti f31bce5b48 Fixes the issue seen in https://github.com/citusdata/citus-enterprise/issues/745
With this commit, rebalancer backends are identified by application_name = citus_rebalancer
and the regular internal backends are identified by application_name = citus_internal
2022-02-03 09:40:46 -08:00
Burak Velioglu f88cc230bf
Handle tables and objects as metadata. Update UDFs accordingly
With this commit we've started to propagate sequences and shell
tables within the object dependency resolution. So, ensuring any
dependencies for any object will consider shell tables and sequences
as well. Separate logics for both shell tables and sequences have
been removed.

Since both shell tables and sequences logic were implemented as a
part of the metadata handling before that logic, we were propagating
them while syncing table metadata. With this commit we've divided
metadata (which means anything except shards thereafter) syncing
logic into multiple parts and implemented it either as a part of
ActivateNode. You can check the functions called in ActivateNode
to check definition of different metadata.

Definitions of start_metadata_sync_to_node and citus_activate_node
have also been updated. citus_activate_node will basically create
an active node with all metadata and reference table shards.
start_metadata_sync_to_node will be same with citus_activate_node
except replicating reference tables. stop_metadata_sync_to_node
will remove all the metadata. All of those UDFs need to be called
by superuser.
2022-01-31 16:20:15 +03:00