external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
citus/src/include/distributed
History
Ahmet Gedemenli 235047670d
Drop SHARD_STATE_TO_DELETE (#6494) 
DESCRIPTION: Drop `SHARD_STATE_TO_DELETE` and use the cleanup records
instead

Drops the shard state that is used to mark shards as orphaned. Now we
insert cleanup records into `pg_dist_cleanup` so "orphaned" shards will
be dropped either by maintenance daemon or internal cleanup calls. With
this PR, we make the "cleanup orphaned shards" functions to be no-op, as
they would not be needed anymore.

This PR includes some naming changes about placement functions. We don't
need functions that filter orphaned shards, as there will be no orphaned
shards anymore.

We will also be introducing a small script with this PR, for users with
orphaned shards. We'll basically delete the orphaned shard entries from
`pg_dist_placement` and insert cleanup records into `pg_dist_cleanup`
for each one of them, during Citus upgrade.

We also have a lot of flakiness fixes in this PR.

Co-authored-by: Jelte Fennema <github-tech@jeltef.nl>
 		2023-01-03 14:38:16 +03:00
..
commands Not drop default col exprs from shard when adding local table to metadata (#6323) 2022-09-23 03:05:08 -07:00
metadata Make sure to disallow triggers that depend on extensions (#6399) 2022-11-02 16:27:31 +03:00
utils Add non-blocking variant of create_distributed_table (#6087) 2022-08-30 15:35:40 +03:00
adaptive_executor.h Use intermediate results for re-partition joins 2022-02-23 19:40:21 +01:00
argutils.h Add the necessary changes for rebalance strategies on enterprise (#3325) 2019-12-19 15:23:08 +01:00
backend_data.h Address race condition in InitializeBackendData (#6285) 2022-09-02 14:23:47 +02:00
background_jobs.h add citus_task_wait udf to wait on desired task status (#6475) 2022-12-12 22:41:03 +03:00
cancel_utils.h add IsHoldOffCancellationReceived utility function (#3290) 2019-12-12 17:32:59 +03:00
causal_clock.h Remove unused macros 2022-10-28 10:38:07 -07:00
citus_acquire_lock.h remove copyright years (#3286) 2019-12-11 21:14:08 +03:00
citus_clauses.h Rename master evaluation to coordinator evaluation 2020-07-07 10:37:41 +02:00
citus_custom_scan.h Do not execute subplans multiple times with cursors 2020-11-20 10:43:56 +01:00
citus_depended_object.h we consider stat object as invalid if it is not owned by current user (#6130) 2022-08-09 20:59:30 +03:00
citus_nodefuncs.h refactor table ddl events scoped for shards (#4342) 2020-11-26 13:31:59 +01:00
citus_nodes.h refactor table ddl events scoped for shards (#4342) 2020-11-26 13:31:59 +01:00
citus_ruleutils.h Adds support for unlogged distributed sequences (#6292) 2022-09-13 10:53:39 +03:00
citus_safe_lib.h Introduce Non-Blocking Shard Split Workflow 2022-08-04 16:32:38 +02:00
colocation_utils.h Add non-blocking variant of create_distributed_table (#6087) 2022-08-30 15:35:40 +03:00
combine_query_planner.h verify shards if exists for insert, delete, update (#6280) 2022-09-06 15:29:14 +02:00
commands.h fix the problem #5763 (#6519) 2022-12-02 13:49:32 +01:00
connection_management.h Include gpid in all internal application names (#6431) 2022-11-25 11:16:33 +01:00
coordinator_protocol.h Deprecate citus.replicate_reference_tables_on_activate, make it always off (#6474) 2022-11-04 16:21:10 +01:00
cte_inline.h Remove citus.enable_cte_inlining GUC 2022-03-22 17:14:44 +01:00
deparse_shard_query.h Fix missing from entry 2021-11-04 18:54:52 +03:00
deparser.h fix the problem #5763 (#6519) 2022-12-02 13:49:32 +01:00
directed_acyclic_graph_execution.h Fill in jobIdList field of DistributedExecution 2020-02-05 17:32:22 +00:00
distributed_deadlock_detection.h Remove copyright years (#2918) 2019-10-15 17:44:30 +03:00
distributed_execution_locks.h Unify distributed execution logic for single replicated tables 2021-11-08 13:52:20 +01:00
distributed_planner.h Support MERGE Phase – I 2022-12-18 20:32:15 -08:00
distribution_column.h Add non-blocking variant of create_distributed_table (#6087) 2022-08-30 15:35:40 +03:00
enterprise.h remove copyright years (#3286) 2019-12-11 21:14:08 +03:00
error_codes.h Issue worker messages with the same log level 2020-04-14 21:08:25 +02:00
errormessage.h Make enterprise features open source (#6008) 2022-06-16 00:23:46 -07:00
extended_op_node_utils.h Fix some more master->coordinator comments 2020-07-07 10:37:53 +02:00
foreign_key_relationship.h Use CreateSimpleHash (and variants) whenever possible (#6177) 2022-08-17 13:01:59 +03:00
function_call_delegation.h Force-delegated functions' distribution argument must be reset as soon as the routine completes execution, 2022-02-17 10:48:30 -08:00
function_utils.h Semmle: Fix obvious issues (#3502) 2020-02-21 10:16:00 +01:00
hash_helpers.h Use CreateSimpleHash (and variants) whenever possible (#6177) 2022-08-17 13:01:59 +03:00
insert_select_executor.h Fix issues with insert..select casts and column ordering 2022-07-28 13:23:57 +02:00
insert_select_planner.h Move more from insert_select_executor to insert_select_planner 2020-06-26 08:08:26 -07:00
intermediate_result_pruning.h Feature: localhost guc (#4836) 2021-05-12 16:59:44 +02:00
intermediate_results.h Shard Split for Citus (#6029) 2022-07-18 02:54:15 -07:00
jsonbutils.h Fix node adding itself with citus_add_node leading to deadlock (Fix #5720) (#5758) 2022-03-10 17:46:33 +03:00
listutils.h Remove useless version compats 2022-07-29 10:31:55 +02:00
local_distributed_join_planner.h Phase - I: recursively plan non-recurring relations 2022-12-07 18:27:50 +03:00
local_executor.h Improve nested execution checks and add GUC to disable 2022-05-20 18:55:43 +02:00
local_multi_copy.h Allow local execution for intermediate results in COPY 2021-02-09 15:00:06 +01:00
local_plan_cache.h Deparse/parse the local cached queries 2021-06-21 12:24:29 +03:00
locally_reserved_shared_connections.h COPY uses adaptive connection management on local node 2021-02-04 09:45:07 +01:00
lock_graph.h Dumping wait edges becomes optionally scan all backends 2022-02-21 17:37:07 +01:00
log_utils.h Remove unused functions (#6220) 2022-08-22 11:53:25 +03:00
maintenanced.h PG 15 Compat: Resolve compile issues + shmem requests 2022-07-15 10:11:39 +02:00
memutils.h Implementation for asycn FinishConnectionListEstablishment (#2584) 2019-03-22 17:30:42 +01:00
metadata_cache.h Drop SHARD_STATE_TO_DELETE (#6494) 2023-01-03 14:38:16 +03:00
metadata_sync.h Add non-blocking variant of create_distributed_table (#6087) 2022-08-30 15:35:40 +03:00
metadata_utility.h Drop SHARD_STATE_TO_DELETE (#6494) 2023-01-03 14:38:16 +03:00
multi_executor.h Introduce GUC citus.skip_constraint_validation (#6281) 2022-09-08 18:13:18 +03:00
multi_explain.h Sort explain analyze output by task time 2020-09-24 11:38:40 +03:00
multi_join_order.h Rename ForceXxx functions to to XxxOrError 2020-09-01 11:19:17 +02:00
multi_logical_optimizer.h Fix union pushdown issue (#5079) 2021-07-29 13:52:55 +03:00
multi_logical_planner.h Drop postgres 12 support (#6040) 2022-07-20 17:49:36 +03:00
multi_logical_replication.h Drop cleanup on failure (#6584) 2022-12-28 15:48:44 +03:00
multi_partitioning_utils.h Add non-blocking variant of create_distributed_table (#6087) 2022-08-30 15:35:40 +03:00
multi_physical_planner.h remove unused boundary type (#6563) 2022-12-16 18:19:22 +03:00
multi_progress.h Add status column to get_rebalance_progress() (#6403) 2022-10-17 16:55:31 +03:00
multi_router_planner.h Support MERGE Phase – I 2022-12-18 20:32:15 -08:00
multi_server_executor.h Fix typos. Spurred spotting "connectios" in logs 2021-10-25 13:54:09 +00:00
namespace_utils.h Implement PushOverrideEmptySearchPath (#3874) 2020-06-05 19:23:59 +03:00
param_utils.h Fixes: #5787 In prepared statements, map any unused parameters 2022-05-13 19:31:05 -07:00
pg_dist_background_job.h Implement infrastructure to run sql jobs in the background (#6296) 2022-09-09 16:11:19 +03:00
pg_dist_background_task.h Implement infrastructure to run sql jobs in the background (#6296) 2022-09-09 16:11:19 +03:00
pg_dist_backrgound_task_depend.h Implement infrastructure to run sql jobs in the background (#6296) 2022-09-09 16:11:19 +03:00
pg_dist_cleanup.h 'Deferred Drop' and robust 'Shard Cleanup' for Splits. (#6258) 2022-09-06 12:11:20 -07:00
pg_dist_colocation.h Add distributioncolumncollation to to pg_dist_colocation 2019-12-09 19:51:40 +00:00
pg_dist_local_group.h Remove copyright years (#2918) 2019-10-15 17:44:30 +03:00
pg_dist_node.h Replace master with citus in logs and comments (#5210) 2021-08-26 11:31:17 +03:00
pg_dist_node_metadata.h Remove copyright years (#2918) 2019-10-15 17:44:30 +03:00
pg_dist_partition.h Dont auto-undistribute user-added citus local tables (#5314) 2021-10-28 12:10:26 +03:00
pg_dist_placement.h Fix some more master->coordinator comments 2020-07-07 10:37:53 +02:00
pg_dist_rebalance_strategy.h Implement an improvement threshold in the rebalancer (#4927) 2021-05-11 14:24:59 +02:00
pg_dist_shard.h Remove cstore_fdw-related logic 2021-11-16 13:59:03 +01:00
pg_dist_transaction.h Remove copyright years (#2918) 2019-10-15 17:44:30 +03:00
pg_version_constants.h Pg15 support (#6085) 2022-08-24 17:59:17 +02:00
placement_access.h Remove real time router executors (#3142) 2019-11-05 12:48:10 +01:00
placement_connection.h Drop support for Inactive Shard placements 2021-10-22 18:03:35 +02:00
priority.h Support changing CPU priorities for backends and shard moves (#6126) 2022-08-16 13:07:17 +03:00
query_colocation_checker.h Refactor WrapRteRelationIntoSubquery 2020-12-15 18:18:36 +03:00
query_pushdown_planning.h Revert "Support JSON_TABLE on PG 15 (#6241)" 2022-09-12 15:20:17 +03:00
query_stats.h PG 15 Compat: Resolve compile issues + shmem requests 2022-07-15 10:11:39 +02:00
query_utils.h Fix partition column index issue (#4591) 2021-01-29 14:40:40 +03:00
recursive_planning.h Drop postgres 11 support 2021-03-25 09:20:28 +03:00
reference_table_utils.h Feature: run rebalancer in the background (#6215) 2022-09-12 20:46:53 +03:00
relation_access_tracking.h Detach relation access tracking from connection management 2022-07-28 11:27:59 +02:00
relation_restriction_equivalence.h Fix union pushdown issue (#5079) 2021-07-29 13:52:55 +03:00
relation_utils.h Refactor PreprocessIndexStmt (#4272) 2020-11-25 12:19:37 +03:00
relay_utility.h Drop SHARD_STATE_TO_DELETE (#6494) 2023-01-03 14:38:16 +03:00
remote_commands.h This implements a new UDF citus_get_cluster_clock() that returns a monotonically 2022-10-28 10:15:08 -07:00
remote_transaction.h Reuse connections for Splits and Logical Replication (#6314) 2022-09-26 13:37:40 +02:00
repartition_join_execution.h Enable re-partition joins after local execution 2022-02-23 19:40:21 +01:00
resource_lock.h Implement infrastructure to run sql jobs in the background (#6296) 2022-09-09 16:11:19 +03:00
run_from_same_connection.h Remove copyright years (#2918) 2019-10-15 17:44:30 +03:00
shard_cleaner.h Drop SHARD_STATE_TO_DELETE (#6494) 2023-01-03 14:38:16 +03:00
shard_pruning.h Enable citus-local distributed table joins 2020-12-15 18:17:10 +03:00
shard_rebalancer.h Introduce a new GUC to propagate local settings to new connections in rebalancer (#6396) 2022-10-18 12:50:30 +03:00
shard_split.h 'Deferred Drop' and robust 'Shard Cleanup' for Splits. (#6258) 2022-09-06 12:11:20 -07:00
shard_transfer.h Drop SHARD_STATE_TO_DELETE (#6494) 2023-01-03 14:38:16 +03:00
shard_utils.h Switch to sequential mode on long partition names 2021-04-14 15:27:50 +03:00
shardinterval_utils.h Remove unused functions that cppcheck found 2020-10-19 13:50:52 +03:00
shardsplit_logical_replication.h Add non-blocking variant of create_distributed_table (#6087) 2022-08-30 15:35:40 +03:00
shardsplit_shared_memory.h Share more code between splits and moves (#6152) 2022-08-15 20:21:51 +03:00
shared_connection_stats.h PG 15 Compat: Resolve compile issues + shmem requests 2022-07-15 10:11:39 +02:00
shared_library_init.h we supress notice log during looking up function oid to not break pg vanilla tests. (#6082) 2022-08-01 10:14:35 +03:00
statistics_collection.h Remove copyright years (#2918) 2019-10-15 17:44:30 +03:00
string_utils.h Make enterprise features open source (#6008) 2022-06-16 00:23:46 -07:00
subplan_execution.h Locally execute queries that don't need any data access (#3410) 2020-01-23 18:28:34 +01:00
task_execution_utils.h Remove task tracker executor (#3850) 2020-07-18 13:11:36 +03:00
tdigest_extension.h Feature: tdigest aggregate (#3897) 2020-06-12 13:50:28 +02:00
time_constants.h refactor some of hard coded values in citus gucs (#3137) 2019-10-30 10:35:39 +03:00
transaction_identifier.h Remove copyright years (#2918) 2019-10-15 17:44:30 +03:00
transaction_management.h Propagate BEGIN properties to worker nodes (#6483) 2022-11-10 18:08:43 +01:00
transaction_recovery.h Delete transactions when removing node 2020-12-07 11:35:20 +03:00
transmit.h Remove old re-partitioning functions 2022-04-04 18:11:52 +02:00
tuple_destination.h Local execution considers intermediate result size limit 2020-10-15 17:18:55 +02:00
tuplestore.h Remove copyright years (#2918) 2019-10-15 17:44:30 +03:00
type_utils.h This implements a new UDF citus_get_cluster_clock() that returns a monotonically 2022-10-28 10:15:08 -07:00
version_compat.h Drop postgres 12 support (#6040) 2022-07-20 17:49:36 +03:00
worker_create_or_replace.h Add support for TEXT SEARCH CONFIGURATION objects (#5685) 2022-02-17 13:12:46 +01:00
worker_log_messages.h Issue worker messages with the same log level 2020-04-14 21:08:25 +02:00
worker_manager.h Deprecate citus.replicate_reference_tables_on_activate, make it always off (#6474) 2022-11-04 16:21:10 +01:00
worker_protocol.h Deprecate citus.replicate_reference_tables_on_activate, make it always off (#6474) 2022-11-04 16:21:10 +01:00
worker_shard_copy.h Shard Split for Citus (#6029) 2022-07-18 02:54:15 -07:00
worker_shard_visibility.h Convert citus.hide_shards_from_app_name_prefixes to citus.show_shards_for_app_name_prefixes 2022-05-03 14:22:13 +02:00
worker_transaction.h Reuse connections for Splits and Logical Replication (#6314) 2022-09-26 13:37:40 +02:00