mirror of https://github.com/citusdata/citus.git
b3af63c8ce
* use adaptive executor even if task-tracker is set
* Update check-multi-mx tests for adaptive executor
Basically repartition joins are enabled where necessary. For parallel
tests max adaptive executor pool size is decresed to 2, otherwise we
would get too many clients error.
* Update limit_intermediate_size test
It seems that when we use adaptive executor instead of task tracker, we
exceed the intermediate result size less in the test. Therefore updated
the tests accordingly.
* Update multi_router_planner
It seems that there is one problem with multi_router_planner when we use
adaptive executor, we should fix the following error:
+ERROR: relation "authors_range_840010" does not exist
+CONTEXT: while executing command on localhost:57637
* update repartition join tests for check-multi
* update isolation tests for repartitioning
* Error out if shard_replication_factor > 1 with repartitioning
As we are removing the task tracker, we cannot switch to it if
shard_replication_factor > 1. In that case, we simply error out.
* Remove MULTI_EXECUTOR_TASK_TRACKER
* Remove multi_task_tracker_executor
Some utility methods are moved to task_execution_utils.c.
* Remove task tracker protocol methods
* Remove task_tracker.c methods
* remove unused methods from multi_server_executor
* fix style
* remove task tracker specific tests from worker_schedule
* comment out task tracker udf calls in tests
We were using task tracker udfs to test permissions in
multi_multiuser.sql. We should find some other way to test them, then we
should remove the commented out task tracker calls.
* remove task tracker test from follower schedule
* remove task tracker tests from multi mx schedule
* Remove task-tracker specific functions from worker functions
* remove multi task tracker extra schedule
* Remove unused methods from multi physical planner
* remove task_executor_type related things in tests
* remove LoadTuplesIntoTupleStore
* Do initial cleanup for repartition leftovers
During startup, task tracker would call TrackerCleanupJobDirectories and
TrackerCleanupJobSchemas to clean up leftover directories and job
schemas. With adaptive executor, while doing repartitions it is possible
to leak these things as well. We don't retry cleanups, so it is possible
to have leftover in case of errors.
TrackerCleanupJobDirectories is renamed as
RepartitionCleanupJobDirectories since it is repartition specific now,
however TrackerCleanupJobSchemas cannot be used currently because it is
task tracker specific. The thing is that this function is a no-op
currently.
We should add cleaning up intermediate schemas to DoInitialCleanup
method when that problem is solved(We might want to solve it in this PR
as well)
* Revert "remove task tracker tests from multi mx schedule"
This reverts commit
|
||
|---|---|---|
| .. | ||
| commands | ||
| metadata | ||
| adaptive_executor.h | ||
| argutils.h | ||
| backend_data.h | ||
| cancel_utils.h | ||
| citus_acquire_lock.h | ||
| citus_clauses.h | ||
| citus_custom_scan.h | ||
| citus_nodefuncs.h | ||
| citus_nodes.h | ||
| citus_ruleutils.h | ||
| citus_safe_lib.h | ||
| colocation_utils.h | ||
| combine_query_planner.h | ||
| commands.h | ||
| connection_management.h | ||
| coordinator_protocol.h | ||
| cte_inline.h | ||
| deparse_shard_query.h | ||
| deparser.h | ||
| directed_acyclic_graph_execution.h | ||
| distributed_deadlock_detection.h | ||
| distributed_execution_locks.h | ||
| distributed_planner.h | ||
| distribution_column.h | ||
| enterprise.h | ||
| error_codes.h | ||
| errormessage.h | ||
| extended_op_node_utils.h | ||
| foreign_key_relationship.h | ||
| function_call_delegation.h | ||
| function_utils.h | ||
| hash_helpers.h | ||
| insert_select_executor.h | ||
| insert_select_planner.h | ||
| intermediate_result_pruning.h | ||
| intermediate_results.h | ||
| listutils.h | ||
| local_executor.h | ||
| local_multi_copy.h | ||
| local_plan_cache.h | ||
| lock_graph.h | ||
| log_utils.h | ||
| maintenanced.h | ||
| memutils.h | ||
| metadata_cache.h | ||
| metadata_sync.h | ||
| metadata_utility.h | ||
| multi_client_executor.h | ||
| multi_executor.h | ||
| multi_explain.h | ||
| multi_join_order.h | ||
| multi_logical_optimizer.h | ||
| multi_logical_planner.h | ||
| multi_partitioning_utils.h | ||
| multi_physical_planner.h | ||
| multi_progress.h | ||
| multi_router_planner.h | ||
| multi_server_executor.h | ||
| namespace_utils.h | ||
| pg_dist_colocation.h | ||
| pg_dist_local_group.h | ||
| pg_dist_node.h | ||
| pg_dist_node_metadata.h | ||
| pg_dist_partition.h | ||
| pg_dist_placement.h | ||
| pg_dist_rebalance_strategy.h | ||
| pg_dist_shard.h | ||
| pg_dist_transaction.h | ||
| pg_version_constants.h | ||
| placement_access.h | ||
| placement_connection.h | ||
| query_colocation_checker.h | ||
| query_pushdown_planning.h | ||
| query_stats.h | ||
| query_utils.h | ||
| recursive_planning.h | ||
| reference_table_utils.h | ||
| relation_access_tracking.h | ||
| relation_restriction_equivalence.h | ||
| relay_utility.h | ||
| remote_commands.h | ||
| remote_transaction.h | ||
| repartition_join_execution.h | ||
| resource_lock.h | ||
| run_from_same_connection.h | ||
| shard_pruning.h | ||
| shard_utils.h | ||
| shardinterval_utils.h | ||
| shared_connection_stats.h | ||
| shared_library_init.h | ||
| statistics_collection.h | ||
| subplan_execution.h | ||
| task_execution_utils.h | ||
| tdigest_extension.h | ||
| time_constants.h | ||
| transaction_identifier.h | ||
| transaction_management.h | ||
| transaction_recovery.h | ||
| transmit.h | ||
| tuple_destination.h | ||
| tuplestore.h | ||
| version_compat.h | ||
| worker_create_or_replace.h | ||
| worker_log_messages.h | ||
| worker_manager.h | ||
| worker_protocol.h | ||
| worker_shard_visibility.h | ||
| worker_transaction.h | ||