external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
citus/src/test/regress/sql
History
Metin Doslu 1f838199f8 Use CustomScan API for query execution 
Custom Scan is a node in the planned statement which helps external providers
to abstract data scan not just for foreign data wrappers but also for regular
relations so you can benefit your version of caching or hardware optimizations.
This sounds like only an abstraction on the data scan layer, but we can use it
as an abstraction for our distributed queries. The only thing we need to do is
to find distributable parts of the query, plan for them and replace them with
a Citus Custom Scan. Then, whenever PostgreSQL hits this custom scan node in
its Vulcano style execution, it will call our callback functions which run
distributed plan and provides tuples to the upper node as it scans a regular
relation. This means fewer code changes, fewer bugs and more supported features
for us!

First, in the distributed query planner phase, we create a Custom Scan which
wraps the distributed plan. For real-time and task-tracker executors, we add
this custom plan under the master query plan. For router executor, we directly
pass the custom plan because there is not any master query. Then, we simply let
the PostgreSQL executor run this plan. When it hits the custom scan node, we
call the related executor parts for distributed plan, fill the tuple store in
the custom scan and return results to PostgreSQL executor in Vulcano style,
a tuple per XXX_ExecScan() call.

* Modify planner to utilize Custom Scan node.
* Create different scan methods for different executors.
* Use native PostgreSQL Explain for master part of queries.
 		2017-03-14 12:17:51 +02:00
..
.gitignore Add Regression Tests For Querying MX Tables from Workers 2017-01-24 10:36:59 +03:00
multi_agg_approximate_distinct.sql Add support for filters 2016-12-01 08:53:46 +03:00
multi_array_agg.sql Support PostgreSQL 9.6 2016-10-18 16:23:55 -06:00
multi_average_expression.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_basic_queries.sql Add LIMIT/OFFSET Support 2016-07-18 12:00:24 +03:00
multi_binary_master_copy_format.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_citus_tools.sql Add citus tools to default configuration 2017-01-10 17:53:27 +03:00
multi_cluster_management.sql Fix CloseNodeConnections to actually close connections 2017-01-11 01:13:58 +02:00
multi_colocated_shard_transfer.sql Improve regression tests for multi_colocated_shard_transfer 2016-12-20 14:09:35 +02:00
multi_colocation_utils.sql Modify tests to create clean workspace 2017-01-05 12:22:44 +03:00
multi_complex_expressions.sql Add support for filters 2016-12-01 08:53:46 +03:00
multi_count_type_conversion.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_create_fdw.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_create_insert_proxy.sql Add support for prepared statements with parameterized non-partition columns in router executor 2016-07-21 11:09:28 +03:00
multi_create_shards.sql Add syscols in queries; extend relnames in indexes 2016-09-07 11:54:55 -05:00
multi_create_table.sql Address review feedback in create_distributed_table data loading 2017-02-28 17:39:45 +01:00
multi_create_table_constraints.sql Always CASCADE while dropping a shard 2016-11-01 10:16:34 +01:00
multi_data_types.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_deparse_shard_query.sql Add ability to reorder target list for INSERT/SELECT queries 2016-10-26 10:00:03 +03:00
multi_distribution_metadata.sql Refactor get_shard_id_for_distribution_column() and other minor changes 2017-01-20 14:38:01 +02:00
multi_drop_extension.sql Switch from pg_worker_list.conf file to pg_dist_node metadata table. 2016-10-05 13:01:35 +03:00
multi_dropped_column_aliases.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_expire_table_cache.sql Add UDF master_expire_table_cache 2016-09-28 12:08:37 +03:00
multi_explain.sql Hack up PREPARE/EXECUTE for nearly all distributed queries. 2017-01-23 09:23:50 -08:00
multi_extension.sql Remove master_get_local_first_candidate_nodes 2017-03-07 11:50:59 +03:00
multi_foreign_key.sql Bugfix for creating foreign key 2017-02-07 09:34:24 +02:00
multi_function_evaluation.sql Evaluate functions on the master 2016-07-13 11:45:51 -07:00
multi_generate_ddl_commands.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_hash_pruning.sql Make router planner active at all times 2016-12-20 11:24:01 +03:00
multi_index_statements.sql Avoid error during CREATE INDEX IF NOT EXISTS 2016-11-01 14:51:19 -07:00
multi_insert_select.sql Use DEBUG2 instead of DEBUG4 in INSERT SELECT tests & debug message. 2017-02-20 12:56:16 +02:00
multi_join_order_additional.sql Use CustomScan API for query execution 2017-03-14 12:17:51 +02:00
multi_join_order_tpch_large.sql Use CustomScan API for query execution 2017-03-14 12:17:51 +02:00
multi_join_order_tpch_small.sql Add EXPLAIN for simple distributed queries 2016-04-30 00:11:02 +02:00
multi_join_pruning.sql Fix segmentation fault in case of joins with WHERE 1=0 2016-09-26 15:12:29 +02:00
multi_large_table_join_planning.sql Remove variant files 2016-06-13 12:12:06 +03:00
multi_large_table_pruning.sql Fix segmentation fault in case of joins with WHERE 1=0 2016-09-26 15:12:29 +02:00
multi_large_table_task_assignment.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_limit_clause.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_limit_clause_approximate.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_master_protocol.sql Remove unused master_get_round_robin_candidate_nodes 2017-03-07 11:51:24 +03:00
multi_metadata_access.sql GRANT SELECT access for metadata tables to public 2016-12-23 16:32:47 +03:00
multi_metadata_sync.sql Fix dependent tests 2017-01-25 19:19:39 +03:00
multi_modifications.sql Feature: INSERT INTO ... SELECT 2016-10-26 10:01:00 +03:00
multi_modifying_xacts.sql Add copy failure tests inside transactions 2017-01-26 11:54:40 +03:00
multi_mx_create_table.sql Add Regression Tests For Querying MX Tables from Workers 2017-01-24 10:36:59 +03:00
multi_mx_ddl.sql Use coordinator instead of schema node in terminology 2017-01-25 11:07:23 +01:00
multi_mx_explain.sql Add Regression Tests For Querying MX Tables from Workers 2017-01-24 10:36:59 +03:00
multi_mx_metadata.sql Use coordinator instead of schema node in terminology 2017-01-25 11:07:23 +01:00
multi_mx_modifications.sql Use coordinator instead of schema node in terminology 2017-01-25 11:07:23 +01:00
multi_mx_modifying_xacts.sql Use coordinator instead of schema node in terminology 2017-01-25 11:07:23 +01:00
multi_mx_reference_table.sql Use CustomScan API for query execution 2017-03-14 12:17:51 +02:00
multi_mx_repartition_join_w1.sql Add Regression Tests For Querying MX Tables from Workers 2017-01-24 10:36:59 +03:00
multi_mx_repartition_join_w2.sql Add Regression Tests For Querying MX Tables from Workers 2017-01-24 10:36:59 +03:00
multi_mx_repartition_udt_prepare.sql Add Regression Tests For Querying MX Tables from Workers 2017-01-24 10:36:59 +03:00
multi_mx_repartition_udt_w1.sql Add Regression Tests For Querying MX Tables from Workers 2017-01-24 10:36:59 +03:00
multi_mx_repartition_udt_w2.sql Add Regression Tests For Querying MX Tables from Workers 2017-01-24 10:36:59 +03:00
multi_mx_router_planner.sql Use CustomScan API for query execution 2017-03-14 12:17:51 +02:00
multi_mx_schema_support.sql Use CustomScan API for query execution 2017-03-14 12:17:51 +02:00
multi_mx_tpch_query1.sql Use coordinator instead of schema node in terminology 2017-01-25 11:07:23 +01:00
multi_mx_tpch_query3.sql Use coordinator instead of schema node in terminology 2017-01-25 11:07:23 +01:00
multi_mx_tpch_query6.sql Use coordinator instead of schema node in terminology 2017-01-25 11:07:23 +01:00
multi_mx_tpch_query7.sql Use coordinator instead of schema node in terminology 2017-01-25 11:07:23 +01:00
multi_mx_tpch_query7_nested.sql Use coordinator instead of schema node in terminology 2017-01-25 11:07:23 +01:00
multi_mx_tpch_query10.sql Add Regression Tests For Querying MX Tables from Workers 2017-01-24 10:36:59 +03:00
multi_mx_tpch_query12.sql Use coordinator instead of schema node in terminology 2017-01-25 11:07:23 +01:00
multi_mx_tpch_query14.sql Use coordinator instead of schema node in terminology 2017-01-25 11:07:23 +01:00
multi_mx_tpch_query19.sql Use coordinator instead of schema node in terminology 2017-01-25 11:07:23 +01:00
multi_name_lengths.sql Provides safe, idempotent shard-extended names to any object name 2016-10-03 17:02:34 -04:00
multi_null_minmax_value_pruning.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_partition_pruning.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_prepare_plsql.sql Hack up PREPARE/EXECUTE for nearly all distributed queries. 2017-01-23 09:23:50 -08:00
multi_prepare_sql.sql Hack up PREPARE/EXECUTE for nearly all distributed queries. 2017-01-23 09:23:50 -08:00
multi_prune_shard_list.sql Remove hash-pruning logic for NULL values 2016-07-06 17:04:21 -06:00
multi_query_directory_cleanup.sql Initial temp table removal implementation 2017-03-14 12:09:49 +02:00
multi_reference_table.sql Use CustomScan API for query execution 2017-03-14 12:17:51 +02:00
multi_remove_node_reference_table.sql Fix dependent tests 2017-01-25 19:19:39 +03:00
multi_repair_shards.sql Disallow SendCommandListToWorkerInSingleTransaction when modifications have occurred 2016-11-02 12:26:56 +01:00
multi_repartition_udt.sql During repartitions, the partitionColumnType argument sent to workers 2016-10-03 13:41:20 -04:00
multi_repartitioned_subquery_udf.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_replicate_reference_table.sql Convert DropShards to use new connection API 2017-01-23 21:08:41 +03:00
multi_router_planner.sql Use CustomScan API for query execution 2017-03-14 12:17:51 +02:00
multi_schema_support.sql Use CustomScan API for query execution 2017-03-14 12:17:51 +02:00
multi_shard_modify.sql Error on Unsupported Features on Workers 2017-01-02 16:03:45 +03:00
multi_simple_queries.sql Support PostgreSQL 9.6 2016-10-18 16:23:55 -06:00
multi_single_relation_subquery.sql Add HAVING support 2016-10-13 15:47:53 +03:00
multi_sql_function.sql Add regression tests for parameterized queries 2016-10-18 14:02:50 +03:00
multi_table_ddl.sql Enforce statement based replication on old APIs and non-hash tables 2017-02-16 10:37:53 -08:00
multi_task_assignment_policy.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_tpch_query1.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_tpch_query3.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_tpch_query6.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_tpch_query7.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_tpch_query7_nested.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_tpch_query10.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_tpch_query12.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_tpch_query14.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_tpch_query19.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
multi_transaction_recovery.sql Enable transaction recovery in connection API 2016-12-23 16:14:29 +01:00
multi_transactional_drop_shards.sql Add ORDER BY to some tests to have consistent output 2017-01-25 11:43:25 +02:00
multi_truncate.sql Convert DropShards to use new connection API 2017-01-23 21:08:41 +03:00
multi_unsupported_worker_operations.sql Allow dropping sequences on mx workers 2017-01-31 14:51:44 -08:00
multi_upgrade_reference_table.sql Add replication_model GUC 2017-01-23 09:05:14 -07:00
multi_upsert.sql Remove references to 9.4 2016-09-29 17:35:19 -06:00
multi_utilities.sql Add worker_hash() and a stub for isolate_tenant_to_new_shard() 2017-01-20 14:38:01 +02:00
multi_utility_statements.sql Use CustomScan API for query execution 2017-03-14 12:17:51 +02:00
multi_utility_warnings.sql Remove warnings on schema creation 2016-07-22 18:24:23 +03:00
multi_verify_no_subquery.sql Add LIMIT/OFFSET Support 2016-07-18 12:00:24 +03:00
multi_view.sql Add view support 2017-01-13 09:39:42 +03:00
multi_working_columns.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
task_tracker_assign_task.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
task_tracker_cleanup_job.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
task_tracker_create_table.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
task_tracker_partition_task.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
worker_binary_data_partition.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
worker_check_invalid_arguments.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
worker_create_table.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
worker_hash_partition.sql During repartitions, the partitionColumnType argument sent to workers 2016-10-03 13:41:20 -04:00
worker_hash_partition_complex.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
worker_merge_hash_files.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
worker_merge_range_files.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
worker_null_data_partition.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00
worker_range_partition.sql Use single-quote interpolation in partition test 2016-10-10 13:03:43 -06:00
worker_range_partition_complex.sql Set Explicit ShardId/JobId In Regression Tests 2016-06-07 14:32:44 +03:00