external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,588 Commits
636 Branches
162 Tags
98 MiB
Commit Graph

2588 Commits (009d8b74013fde1394ec075ce6cce8a34f2fa655)

Author SHA1 Message Date
Hadi Moshayedi 009d8b7401 Some cleanup 2019-08-12 15:38:52 -07:00
Philip Dubé 03ef456c50
Merge pull request #2871 from citusdata/pg12-test-prep 
Update tests in preparation for pg12
 2019-08-09 15:32:19 +00:00
Philip Dubé 5459c01956 multi_partitioning_utils: version_above_ten 2019-08-09 15:25:59 +00:00
Philip Dubé e0f19fb58c multi_partitioning_1.out 2019-08-09 15:25:59 +00:00
Philip Dubé 5e835e7565 Fix multi_repair_shards. There's already a group/shardid entry, pg11 gives us back the inserted one, pg12 gives us the preexisting one 2019-08-09 15:25:59 +00:00
Philip Dubé 66ce2d2d2d Materialize c1 to keep subplan ids in sync 2019-08-09 15:25:59 +00:00
Philip Dubé 9065ef429c foreign_key_to_reference_table: terse to avoid differing order of drop cascade details 2019-08-09 15:25:59 +00:00
Philip Dubé 0d9e5bde9c window_functions: 'ORDER BY time' when using lag(time) & coordinator_plan 2019-08-09 15:25:59 +00:00
Philip Dubé 7992077fd9 multi_modifying_xacts: don't differ in output if reference table select tries broken worker first 2019-08-09 15:25:59 +00:00
Philip Dubé 546b71ac18 multi_router_planner: be terse for ctes with false wheres 2019-08-09 15:25:59 +00:00
Philip Dubé a523a5b773 multi_null_minmax_value_pruning: no versioning & coordinator_plan 2019-08-09 15:25:59 +00:00
Philip Dubé 871dabdc63 Force CTE materialization in pg12 2019-08-09 15:25:59 +00:00
Philip Dubé 667c67891e intermediate_results: COSTS OFF 2019-08-09 15:25:59 +00:00
Philip Dubé b2ea806d8a extra_float_digits=0 2019-08-09 15:25:59 +00:00
Philip Dubé 705d1bf0e0 Use PG_JOB_CACHE_DIR 2019-08-09 15:25:59 +00:00
Hanefi Onaldi ef33282de4
Update Changelog for v8.3.2 2019-08-09 12:32:38 +03:00
Önder Kalacı e21578f3af
Merge pull request #2866 from citusdata/fix_83_regression 
Do not record relation accessess unnecessarily
 2019-08-08 18:47:58 +02:00
Onder Kalaci 060ac11476 Do not record relation accessess unnecessarily 
Before this commit, we've recorded the relation accesses in 3 different
places
    - FindPlacementListConnection         -- applies all executor in tx block
    - StartPlacementExecutionOnSession()  -- adaptive executor only
    - StartPlacementListConnection()      -- router/real-time only

This is different than Citus 8.2, and could lead to query execution times
increase considerably on multi-shard commands in transaction block
that are on partitioned tables.

Benchmarks:

```
1+8 c5.4xlarge cluster

Empty distributed partitioned table with 365 partitions: https://gist.github.com/onderkalaci/1edace4ed6bd6f061c8a15594865bb51#file-partitions_365-sql

./pgbench -f /tmp/multi_shard.sql -c10 -j10 -P 1 -T 120 postgres://citus:w3r6KLJpv3mxe9E-NIUeJw@c.fy5fkjcv45vcepaogqcaskmmkee.db.citusdata.com:5432/citus?sslmode=require

cat  /tmp/multi_shard.sql
BEGIN;
	DELETE FROM collections_list;
	DELETE FROM collections_list;
	DELETE FROM collections_list;
COMMIT;
cat  /tmp/single_shard.sql
BEGIN;
	DELETE FROM collections_list WHERE key = :aid;
	DELETE FROM collections_list WHERE key = :aid;
	DELETE FROM collections_list WHERE key = :aid;
COMMIT;

cat  /tmp/mix.sql
BEGIN;
	DELETE FROM collections_list WHERE key = :aid;
	DELETE FROM collections_list WHERE key = :aid;
	DELETE FROM collections_list WHERE key = :aid;

	DELETE FROM collections_list;
	DELETE FROM collections_list;
	DELETE FROM collections_list;
COMMIT;
```

The table shows `latency average` of pgbench runs explained above, so we have a pretty solid improvement even over 8.2.2.

| Test  | Citus 8.2.2  |  Citus 8.3.1   | Citus 8.3.2 (this branch)  | Citus 8.3.1 (FKEYs disabled via GUC)  |
| ------------- | ------------- | ------------- |------------- | ------------- |
|multi_shard |  2370.083 ms  |3605.040 ms |1324.094 ms |1247.255 ms  |
| single_shard  | 85.338 ms  |120.934 ms  |73.216 ms  | 78.765 ms |
| mix  | 2434.459 ms | 3727.080 ms  |1306.456 ms  | 1280.326 ms |
 2019-08-08 18:42:08 +02:00
Onder Kalaci 35ee896f3d Get rid of an unnecessary parameter 
targetPoolSize parameter for ExecuteUtilityTaskListWithoutResults
becomes obsolete, just remove it.
 2019-08-07 19:35:56 +02:00
Onder Kalaci b2e01d0745 Refactor switching to sequential mode 
We don't need to wait until the execution. As soon as we realize
that we need sequential execution, we should do it.
 2019-08-07 19:35:56 +02:00
Hanefi Onaldi 263faffb27
Update CONTRIBUTING.md (#2865) 
* Update dependency versions
* Add libcurl and autoconf to required dependencies
* Add Clang/LLVM instructions for CentOS/RHEL setup
 2019-08-06 17:52:44 +03:00
Hadi Moshayedi f9efb21f1b
Merge pull request #2863 from citusdata/fix_typo 
Fix a typo in foreign_key_restriction_enforcement
 2019-08-02 16:13:43 -07:00
Hadi Moshayedi b1ab805ce2 Fix a typo in foreign_key_restriction_enforcement 2019-08-02 16:06:52 -07:00
Hadi Moshayedi a1a7d95c0a
Merge pull request #2861 from citusdata/less_polymorphic_plan_router_query 
PlanRouterQuery: don't store list of list of shard intervals in relationShardList
 2019-08-02 09:16:09 -07:00
Philip Dubé b77c52f95b PlanRouterQuery: don't store list of list of shard intervals in relationShardList 2019-08-02 14:08:57 +00:00
Philip Dubé 9b4ba2f5b2
Merge pull request #2858 from citusdata/multi_modifications_bug 
Use 2PC in adaptive executor when dealing with replication factors above 1
 2019-08-02 00:20:22 +00:00
Philip Dubé fdc0ef6392 Adaptive executor: use 2PC when replication_factor > 1 2019-08-01 23:55:12 +00:00
Philip Dubé 19bcb1b4f7 multi_modifications: extend to demonstrate issue in adaptive executor 2019-08-01 23:55:04 +00:00
Hadi Moshayedi b81d5947e4
Merge pull request #2859 from citusdata/no_null_percent_s 
Avoid segfault in logging queries
 2019-08-01 09:36:52 -07:00
Philip Dubé 064bd66a20 Avoid segfault in logging queries 2019-07-31 15:28:46 +00:00
Hanefi Onaldi e88cb8335f
Update Changelog for v8.3.1 2019-07-29 13:10:28 +03:00
Philip Dubé dc67fa36c6
Merge pull request #2854 from citusdata/compare_shard_intervals_id_tie_breaker 
CompareShardIntervals: if intervals are equal, compare id
 2019-07-26 16:27:35 +00:00
Philip Dubé 3982b4635f CompareShardIntervals: if intervals are equal, compare id. Works around sort being unstable 2019-07-26 16:13:36 +00:00
Philip Dubé 6f1a8dfdbe
Merge pull request #2852 from citusdata/update_tests_colocation_utils_copy 
Update two tests, useful fallout from pg12 branch
 2019-07-25 17:36:16 +00:00
Marco Slot c471d9680c
Merge pull request #2848 from citusdata/adaptive_executor_tuning 
Adaptive executor performance improvements
 2019-07-25 16:52:36 +02:00
Philip Dubé 0e233c63a3 multi_colocation_utils: sort by nodeport, not placementid 
multi_copy: replace smgr with aclitem, smgr is removed in pg12
 2019-07-25 14:33:43 +00:00
Hadi Moshayedi cd2905ec23
Merge pull request #2845 from citusdata/squash_migrations_56 
Squash migrations for versions 5/6, don't use WITH OIDS
 2019-07-24 11:10:15 -07:00
Philip Dubé 50144b75d0 Add check-empty to testing Makefile 
Don't create functions multiple times
Move ALTER TABLEs to their declaration
Remove DROP FUNCTIONS IF EXISTS, OR REPLACE
 2019-07-24 11:03:54 -07:00
Philip Dubé acbaa38a62 Squash migrations for versions 5/6, don't use WITH OIDS 2019-07-24 11:03:29 -07:00
Philip Dubé 3e6f3e4f3b
Merge pull request #2849 from citusdata/sort-list-is-not-in-place 
Update workerNodeList after sorting
 2019-07-23 21:04:22 +00:00
Hanefi Onaldi 8127297999 update workerNodeList after sorting 2019-07-23 20:57:07 +00:00
Philip Dubé 6c5866cc4d
Merge pull request #2850 from citusdata/fix_a_couple_shardid_tests 
Fix multi_prune_shard_list
 2019-07-23 20:19:55 +00:00
Philip Dubé 6598c68993 Fix multi_prune_shard_list & don't set next_shard_id unnecessarily in multi_null_minmax_value_pruning 2019-07-23 19:44:18 +00:00
Marco Slot e2bc09838e Use ereport instead of elog in adaptive executor 2019-07-23 20:40:32 +02:00
Marco Slot bd111366b0 Skip CheckConnectionTimeout when checkForPoolTimeout is false 2019-07-23 20:40:32 +02:00
Marco Slot a3811b1e55 Avoid FindWorkerNode calls in adaptive executor 2019-07-23 20:40:32 +02:00
Marco Slot 4444d92dbc Set initial pool size to cached connection count 2019-07-23 20:40:32 +02:00
Marco Slot 4c0c33365e Avoid creating a redundant event set at the start 2019-07-23 20:40:32 +02:00
Marco Slot 32e7a80960 Avoid unnecessary calls to PQconsumeInput 2019-07-23 20:40:32 +02:00
Marco Slot 71ad5c095b Use ModifyWaitEvent when only wait flags changed 2019-07-23 20:40:32 +02:00