external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,586 Commits
637 Branches
162 Tags
99 MiB
Commit Graph

1568 Commits (5459c01956aa2fe4dd17a590166c9acf9cff79e2)

Author SHA1 Message Date
Philip Dubé 5459c01956 multi_partitioning_utils: version_above_ten 2019-08-09 15:25:59 +00:00
Philip Dubé e0f19fb58c multi_partitioning_1.out 2019-08-09 15:25:59 +00:00
Philip Dubé 5e835e7565 Fix multi_repair_shards. There's already a group/shardid entry, pg11 gives us back the inserted one, pg12 gives us the preexisting one 2019-08-09 15:25:59 +00:00
Philip Dubé 66ce2d2d2d Materialize c1 to keep subplan ids in sync 2019-08-09 15:25:59 +00:00
Philip Dubé 9065ef429c foreign_key_to_reference_table: terse to avoid differing order of drop cascade details 2019-08-09 15:25:59 +00:00
Philip Dubé 0d9e5bde9c window_functions: 'ORDER BY time' when using lag(time) & coordinator_plan 2019-08-09 15:25:59 +00:00
Philip Dubé 7992077fd9 multi_modifying_xacts: don't differ in output if reference table select tries broken worker first 2019-08-09 15:25:59 +00:00
Philip Dubé 546b71ac18 multi_router_planner: be terse for ctes with false wheres 2019-08-09 15:25:59 +00:00
Philip Dubé a523a5b773 multi_null_minmax_value_pruning: no versioning & coordinator_plan 2019-08-09 15:25:59 +00:00
Philip Dubé 871dabdc63 Force CTE materialization in pg12 2019-08-09 15:25:59 +00:00
Philip Dubé 667c67891e intermediate_results: COSTS OFF 2019-08-09 15:25:59 +00:00
Philip Dubé b2ea806d8a extra_float_digits=0 2019-08-09 15:25:59 +00:00
Philip Dubé 705d1bf0e0 Use PG_JOB_CACHE_DIR 2019-08-09 15:25:59 +00:00
Onder Kalaci 060ac11476 Do not record relation accessess unnecessarily 
Before this commit, we've recorded the relation accesses in 3 different
places
    - FindPlacementListConnection         -- applies all executor in tx block
    - StartPlacementExecutionOnSession()  -- adaptive executor only
    - StartPlacementListConnection()      -- router/real-time only

This is different than Citus 8.2, and could lead to query execution times
increase considerably on multi-shard commands in transaction block
that are on partitioned tables.

Benchmarks:

```
1+8 c5.4xlarge cluster

Empty distributed partitioned table with 365 partitions: https://gist.github.com/onderkalaci/1edace4ed6bd6f061c8a15594865bb51#file-partitions_365-sql

./pgbench -f /tmp/multi_shard.sql -c10 -j10 -P 1 -T 120 postgres://citus:w3r6KLJpv3mxe9E-NIUeJw@c.fy5fkjcv45vcepaogqcaskmmkee.db.citusdata.com:5432/citus?sslmode=require

cat  /tmp/multi_shard.sql
BEGIN;
	DELETE FROM collections_list;
	DELETE FROM collections_list;
	DELETE FROM collections_list;
COMMIT;
cat  /tmp/single_shard.sql
BEGIN;
	DELETE FROM collections_list WHERE key = :aid;
	DELETE FROM collections_list WHERE key = :aid;
	DELETE FROM collections_list WHERE key = :aid;
COMMIT;

cat  /tmp/mix.sql
BEGIN;
	DELETE FROM collections_list WHERE key = :aid;
	DELETE FROM collections_list WHERE key = :aid;
	DELETE FROM collections_list WHERE key = :aid;

	DELETE FROM collections_list;
	DELETE FROM collections_list;
	DELETE FROM collections_list;
COMMIT;
```

The table shows `latency average` of pgbench runs explained above, so we have a pretty solid improvement even over 8.2.2.

| Test  | Citus 8.2.2  |  Citus 8.3.1   | Citus 8.3.2 (this branch)  | Citus 8.3.1 (FKEYs disabled via GUC)  |
| ------------- | ------------- | ------------- |------------- | ------------- |
|multi_shard |  2370.083 ms  |3605.040 ms |1324.094 ms |1247.255 ms  |
| single_shard  | 85.338 ms  |120.934 ms  |73.216 ms  | 78.765 ms |
| mix  | 2434.459 ms | 3727.080 ms  |1306.456 ms  | 1280.326 ms |
 2019-08-08 18:42:08 +02:00
Onder Kalaci 35ee896f3d Get rid of an unnecessary parameter 
targetPoolSize parameter for ExecuteUtilityTaskListWithoutResults
becomes obsolete, just remove it.
 2019-08-07 19:35:56 +02:00
Onder Kalaci b2e01d0745 Refactor switching to sequential mode 
We don't need to wait until the execution. As soon as we realize
that we need sequential execution, we should do it.
 2019-08-07 19:35:56 +02:00
Hadi Moshayedi b1ab805ce2 Fix a typo in foreign_key_restriction_enforcement 2019-08-02 16:06:52 -07:00
Philip Dubé b77c52f95b PlanRouterQuery: don't store list of list of shard intervals in relationShardList 2019-08-02 14:08:57 +00:00
Philip Dubé fdc0ef6392 Adaptive executor: use 2PC when replication_factor > 1 2019-08-01 23:55:12 +00:00
Philip Dubé 19bcb1b4f7 multi_modifications: extend to demonstrate issue in adaptive executor 2019-08-01 23:55:04 +00:00
Philip Dubé 064bd66a20 Avoid segfault in logging queries 2019-07-31 15:28:46 +00:00
Philip Dubé 3982b4635f CompareShardIntervals: if intervals are equal, compare id. Works around sort being unstable 2019-07-26 16:13:36 +00:00
Philip Dubé 0e233c63a3 multi_colocation_utils: sort by nodeport, not placementid 
multi_copy: replace smgr with aclitem, smgr is removed in pg12
 2019-07-25 14:33:43 +00:00
Marco Slot e2bc09838e Use ereport instead of elog in adaptive executor 2019-07-23 20:40:32 +02:00
Marco Slot bd111366b0 Skip CheckConnectionTimeout when checkForPoolTimeout is false 2019-07-23 20:40:32 +02:00
Marco Slot a3811b1e55 Avoid FindWorkerNode calls in adaptive executor 2019-07-23 20:40:32 +02:00
Marco Slot 4444d92dbc Set initial pool size to cached connection count 2019-07-23 20:40:32 +02:00
Marco Slot 4c0c33365e Avoid creating a redundant event set at the start 2019-07-23 20:40:32 +02:00
Marco Slot 32e7a80960 Avoid unnecessary calls to PQconsumeInput 2019-07-23 20:40:32 +02:00
Marco Slot 71ad5c095b Use ModifyWaitEvent when only wait flags changed 2019-07-23 20:40:32 +02:00
Philip Dubé 50144b75d0 Add check-empty to testing Makefile 
Don't create functions multiple times
Move ALTER TABLEs to their declaration
Remove DROP FUNCTIONS IF EXISTS, OR REPLACE
 2019-07-24 11:03:54 -07:00
Philip Dubé acbaa38a62 Squash migrations for versions 5/6, don't use WITH OIDS 2019-07-24 11:03:29 -07:00
Hanefi Onaldi 8127297999 update workerNodeList after sorting 2019-07-23 20:57:07 +00:00
Philip Dubé 6598c68993 Fix multi_prune_shard_list & don't set next_shard_id unnecessarily in multi_null_minmax_value_pruning 2019-07-23 19:44:18 +00:00
Marco Slot efbe58eab2 Fix SQL schema version, we skipped 8.3 2019-07-17 16:05:25 +02:00
Philip Dubé 0915027389 DistributedPlan: replace operation with modLevel 
This causes no behaviorial changes, only organizes better to implement modifying CTEs

Also rename ExtactInsertRangeTableEntry to ExtractResultRelationRTE,
as the source of this function didn't match the documentation

Remove Task's upsertQuery in favor of ROW_MODIFY_NONCOMMUTATIVE

Split up AcquireExecutorShardLock into more internal functions

Tests: Normalize multi_reference_table multi_create_table_constraints
 2019-07-16 13:58:18 -07:00
Hanefi Onaldi 0bdec52761
Fix default_version in citus.control file (#2840) 2019-07-11 14:24:51 +03:00
Philip Dubé befd0caddd Tests: normalize sql_procedure and custom_aggregate_support 
Also fix typo in multi_insert_select
 2019-07-10 14:36:17 +00:00
Hanefi Onaldi 5a6eba6ba9
Bump Citus to 8.4devel 2019-07-10 15:26:10 +03:00
Nils Dijk 791cc26a86
Fix an issue with subquery map merge jobs as non-root 
Also automated all manual tests around multi user isolation for internal citus udf's

automate upgrade_to_reference_table tests
add negative tests for lock_relation_if_exists
add tests for permissions on worker_cleanup_job_schema_cache
add tests for worker_fetch_partition_file
add tests for worker_merge_files_into_table
fix problem with worker_merge_files_and_run_query when run as non-super user and add tests for behaviour
 2019-07-10 12:40:05 +02:00
Hadi Moshayedi 46608e42f9 Add hyperscale tutorial to the regression tests. 2019-07-10 10:47:55 +02:00
Hadi Moshayedi 91d8a41ecd Don't modify cache entry in RelationShardListForShardCreate() 2019-07-09 12:44:48 -07:00
Marco Slot 70434bc716 Increase slow start time in test to make valgrind tests pass 2019-07-08 06:04:13 +02:00
Hadi Moshayedi 032167c553 Fix Assert() in ProcessVariableSetStmt() 2019-07-05 14:11:22 -07:00
Marco Slot 07d2266e11 Fix RESET and other types of SET 2019-07-05 19:30:48 +02:00
Marco Slot 97334ff1ec Copy WorkerNode before returning in FindWorkerNode 2019-07-05 09:35:53 +02:00
Hadi Moshayedi 5d59aab38d Increase valgrind's max-stackframe 2019-07-04 14:19:41 +02:00
Hadi Moshayedi d233887d68 Fix multi_extension in check-multi-vg 2019-07-04 13:03:46 +02:00
Hadi Moshayedi 47aa95d00d Fix a NULL dereference. 2019-07-03 16:26:49 -07:00
Hadi Moshayedi 805a2ac602 Fix a use after free in adaptive executor 2019-07-02 10:12:13 -07:00