external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,691 Commits
637 Branches
162 Tags
99 MiB
Commit Graph

1046 Commits (e75eb1713033a62234affdf1f4dd1daa8e010de3)

Author SHA1 Message Date
Marco Slot af201a2f6d Allow intermediate results to be used in parallel workers 2017-12-18 19:05:08 +01:00
Marco Slot 7dab078e67 Set cost estimates for read_intermediate_result 2017-12-18 16:23:44 +01:00
Marco Slot e49254f876 Revert "Add EXPLAIN regression test with subplans" 
This reverts commit 8b6d641227.
 2017-12-17 22:34:31 +01:00
Marco Slot 74bd33d0cc Revert "Plan CTEs when subquery pushdown is on" 
This reverts commit e3b953b8e3.
 2017-12-17 22:34:20 +01:00
Marco Slot aca5f35ab9 Revert "Show distributed subplan ID in EXPLAIN output" 
This reverts commit 686b079272.
 2017-12-17 22:34:04 +01:00
Marco Slot 8b6d641227 Add EXPLAIN regression test with subplans 2017-12-17 22:00:25 +01:00
Marco Slot e3b953b8e3 Plan CTEs when subquery pushdown is on 2017-12-17 21:49:36 +01:00
Marco Slot 686b079272 Show distributed subplan ID in EXPLAIN output 2017-12-16 11:32:01 +01:00
Marco Slot ea6b98fda4 Allow count(distinct) in queries with a subquery 2017-12-15 15:24:26 +01:00
Marco Slot 9ee0e68882 Do not take extra access exclusive lock partitioned tables 2017-12-15 13:02:31 +01:00
Marco Slot 5a69fc1b17 Relax checks on recurring tuples in FROM with sublinks 2017-12-15 11:56:06 +01:00
Marco Slot a64f0060ba Reduce the frequency of FinishConnectionIO calls during COPY (#1864) 2017-12-14 13:21:59 -05:00
Marco Slot a811aad264 Deparallelise multi_modifying_xacts tests 2017-12-14 10:27:17 +01:00
mehmet furkan şahin 5851f71bfb Add CTE regression tests 2017-12-14 09:32:55 +01:00
Marco Slot fa73abe6d4 Regression test output changes after CTE support 2017-12-14 09:32:55 +01:00
Marco Slot 2e2b4e81fa Add support for CTEs in distributed queries 2017-12-14 09:32:55 +01:00
Marco Slot d0335ec818 Send BEGIN for SELECTs in the router executor 2017-12-14 09:32:55 +01:00
Marco Slot cbbd418af2 Add citus.copy_format OIDs to metadata cache 2017-12-14 09:32:55 +01:00
Marco Slot 66f9f1d6cd Make some intermediate results functions public 2017-12-14 09:32:55 +01:00
Marco Slot 36ee21c323 Make CanUseBinaryCopyFormatForType public 2017-12-14 09:32:55 +01:00
Marco Slot 7d1191954d Add DistributedSubPlan node 2017-12-14 09:32:55 +01:00
Onder Kalaci 86b2d9420c Treat recurring tuples as reference table for GROUP BY checks 
read_intermediate_results() and immutable functions are implemented.
Empty join trees seems not applicable here.
 2017-12-13 14:55:42 +02:00
Marco Slot d1a470a52e Fix issue with multiple ANALYZE in transaction block 2017-12-12 10:28:48 +01:00
mehmet furkan şahin 3c941aedf1 adds citus.enable_repartition_joins GUC 
The new GUC allows Citus to switch between task executors
when necessary
 2017-12-11 09:36:37 +03:00
Marco Slot 5895c88552 Add materialized view regression tests 2017-12-07 16:20:23 +01:00
Marco Slot 60a1e31671 Allow queries with local tables in NeedsDistributedPlanning 2017-12-07 16:20:23 +01:00
Marco Slot f8550b8c85 Fix issues with read_intermediate_result signature 2017-12-07 13:47:56 +01:00
Marco Slot d8fea4efb8 Revert "Allow queries with local tables in NeedsDistributedPlanning" 
This reverts commit d2bac081e8.
 2017-12-07 11:19:11 +01:00
Marco Slot d2bac081e8 Allow queries with local tables in NeedsDistributedPlanning 2017-12-07 11:02:16 +01:00
Onder Kalaci c42a92afd2 Fix bug related to incrementing an index not properly 2017-12-07 08:50:57 +02:00
Marco Slot eab15aa035 Avoid deadlock in ColocatedTableId 2017-12-06 11:49:34 +01:00
metdos 12d5974d97 Increase sleep time in a regression test to give Valgrind tests enough time 2017-12-05 14:59:37 +02:00
Marco Slot 7279d42849 Treat read_intermediate_result as recurring tuples 2017-12-04 14:50:11 +01:00
Marco Slot 716448ddef Add regression tests for intermediate results 2017-12-04 14:50:11 +01:00
Marco Slot 4cdadfcab6 Add intermediate results infrastructure 2017-12-04 14:50:11 +01:00
Marco Slot bfcc76df69 Make several COPY-related functions public 2017-12-04 13:12:03 +01:00
Marco Slot 73989b07eb Refactor query execution functions 2017-12-04 13:12:03 +01:00
Murat Tuncer 2d66bf5f16
Fix hard coded formatting strings for 64 bit numbers (#1831) 
Postgres provides OS agnosting formatting macros for
formatting 64 bit numbers. Replaced %ld %lu with
INT64_FORMAT and UINT64_FORMAT respectively.

Also found some incorrect usages of formatting
flags and fixed them.
 2017-12-04 14:11:06 +03:00
Hadi Moshayedi ff706cf556 Test that COPY blocks UPDATE/DELETE/INSERT...SELECT when rep factor 2. 2017-11-30 14:52:29 -05:00
Marco Slot acbc0fe0de Use RowExclusiveLock shard resource lock in COPY 2017-11-30 09:15:45 -05:00
Onder Kalaci a273711500 The common attribute equivalance class always includes the input relations 
We added the ability to filter out the planner restriction information
for specific parts of the query. This might lead to situations where
the common restriction includes some other relations that we're searching
for. The reason is that while filtering for join restrictions, we add the
restriction as soon as we find the relation.

With this commit we make sure that the common attribute
equivalance class always includes the input relations.
 2017-11-30 16:00:26 +02:00
Marco Slot 0d6a7f5884 Add real-time BEGIN regression tests 2017-11-30 12:59:09 +01:00
Marco Slot d6dd0b3a81 Send BEGIN in the real-time executor when in a transaction 2017-11-30 12:59:09 +01:00
Marco Slot 3a4d5f8182 Remove filter checks on leaf queries 2017-11-30 12:25:14 +01:00
Marco Slot 3f03cb6a6a Support UNION with joins in the subqueries 2017-11-30 10:37:56 +01:00
Marco Slot a9933deac6 Make real time executor work in transactions 2017-11-30 09:59:32 +03:00
mehmet furkan şahin 6041f85b70
Add tests for non-propagated VACUUM/ANALYZE 2017-11-29 16:06:50 -07:00
Jason Petersen 0eacf6bd95
Refactor VacuumStmt checker to be single-return 
Decided this would be safer for the future (defaults to unsupported).
 2017-11-29 16:06:50 -07:00
Jason Petersen b12e77ab0e
Ensure unsupported VACUUMs don't go to workers 
Apparently these two blocks have been incorrect for nearly a year…
 2017-11-29 16:06:50 -07:00
Marco Slot 7ea718fd8d Round-robin over worker nodes for 0-shard router queries 2017-11-29 15:52:22 +01:00
Marco Slot ae67fa0e52 Do not run multi_mx_modifications in parallel with multi_mx_transaction_recovery 2017-11-29 15:35:21 +01:00
mehmet furkan şahin b6eb0c2823 multi_subquery_behavioral_analytics.sql query fix by adding proper order by 2017-11-28 14:15:46 +03:00
mehmet furkan şahin 1b06b2b306 The data used in regression tests is reduced 
This commit reduces the size of the data in users_table.data
and events_table.data from 10K rows to 100 rows.
 2017-11-28 14:15:46 +03:00
Onder Kalaci 05fb0dd020 Add infrastructure for filtering restriction contexts based on the input query 
In subquery pushdown, we first ensure that each relation is joined with at least
on another relation on the partition keys. That's fine given that the decision
is binary: pushdown the query at all or not.

With recursive planning, we'd want to check whether any specific part
of the query can be pushded down or not. Thus, we need the ability to
understand which part(s) of the subquery is safe to pushdown. This commit
adds the infrastructure for doing that.
 2017-11-28 09:58:21 +02:00
Onder Kalaci 26d9b58e9e Make sure that ExtractRangeTableRelationWalker never misses RTE_RELATION 2017-11-28 09:27:34 +02:00
Onder Kalaci 32def06ebd Split assigning RTE identities and partitioning related query modifications 
Note that we used to iterate over the RTEs once for performance reasons.
However, keeping an extra copy of original query seems more costly and
hard to maintain/explain.
 2017-11-28 09:27:34 +02:00
Marco Slot feffe86440 Subqueries containing functions go through subquery pushdown 2017-11-27 22:13:02 +01:00
Onder Kalaci 48f96bf3e5 Enable non equi joins in subquery pushdown 
Subquery pushdown planning is based on relation restriction
equivalnce. This brings us the opportuneatly to allow any
other joins as long as there is an already equi join between
the distributed tables.

We already allow that for joins with reference tables and
this commit allows that for joins among distributed tables.
 2017-11-23 16:13:46 +02:00
mehmet furkan şahin 032b34ea52 some more parallelization 2017-11-23 14:10:42 +03:00
Onder Kalaci 16421f089f Register citus custom scan nodes 2017-11-23 11:38:33 +02:00
Onder Kalaci 83c1143505 Refactor custom scan related codes 
In this commit, we don't change any codes, only create a new
file and move the related functions and types there.
 2017-11-23 11:38:12 +02:00
Marco Slot 20a526d5c4 Fix memory leak in ListToHashSet 2017-11-22 11:26:58 +01:00
Marco Slot f4ceea5a3d Enable 2PC by default 2017-11-22 11:26:58 +01:00
Marco Slot 8486f76e15 Auto-recover 2PC transactions 2017-11-22 11:26:58 +01:00
Marco Slot 6ba3f42d23 Rename MultiPlan to DistributedPlan 2017-11-22 09:36:24 +01:00
Marco Slot 0ad39b36fe Treat immutable table functions and constant subqueries as reference tables 2017-11-21 14:15:22 +01:00
Onder Kalaci d558ebb923 Relax the checks on ensuring distribution columns for target entries 
With this commit, we allow pushing down subqueries with only
reference tables where GROUP BY or DISTINCT clause or Window
functions include only columns from reference tables.
 2017-11-21 12:28:14 +02:00
Andres Freund d063658d6d Protect some initializations from being called during backend startup. 
On EXEC_BACKEND builds these functions shouldn't be called at every
backend start.
 2017-11-20 15:29:51 -08:00
Brian Cloutier d267e0f9fa EXEC_BACKEND: don't put pointers to shared hashes into shared memory 
Store pointers to shared hashes in process-local variables. Previously
pointers to shared hashes were put into shared memory. This causes
problems on EXEC_BACKEND because everybody calls execve and receives a
brand new address space; the shared hash will be in a different place
for every backend. (normally we call fork, which gives you a copy of the
address space, so these pointers remain constant)
 2017-11-20 15:29:51 -08:00
Brian Cloutier 30a2365d81 Rename CreateDirectory to CitusCreateDirectory 2017-11-20 14:38:26 -08:00
Brian Cloutier aa2ab023a2 Rename RemoveDirectory -> CitusRemoveDirectory 2017-11-20 14:21:52 -08:00
Brian Cloutier 06f756b0a1 Rename DeleteFile -> CitusDeleteFile 2017-11-20 13:30:11 -08:00
mehmet furkan şahin 34709c2a16 Regression tests parallelization PART-1 2017-11-20 18:03:37 +03:00
Marco Slot 9793218122 Do not commit already-committed prepared transactions in recovery 2017-11-20 13:18:48 +01:00
Marco Slot fe798cf0f9 Add recovery vs. recovery isolation test 2017-11-20 12:26:25 +01:00
Marco Slot ae47df01ea Observe prepared xacts twice in RecoverWorkerTransactions to avoid race condition 2017-11-20 11:44:08 +01:00
Marco Slot 2410c2e450 Rewrite recover_prepared_transactions to be fast, non-blocking 2017-11-20 11:27:40 +01:00
mehmet furkan şahin 314fc09d90 regression test shard_count is changed from 32 to 4 2017-11-20 12:47:49 +03:00
mehmet furkan şahin 8d55754b4d the tests are separated and some more added 2017-11-20 11:45:48 +03:00
mehmet furkan şahin 636faadc47 create_distributed_table vs create_distributed_table, master_append_table_to_shard vs master_apply_delete_command, master_apply_delete_command vs master_apply_delete_command are added 2017-11-20 11:45:48 +03:00
mehmet furkan şahin 0722334e50 concurrent master_append_table test is added 2017-11-20 11:45:48 +03:00
mehmet furkan şahin f45988962f multi-shard update affecting the same/different rows 2017-11-20 11:45:48 +03:00
Onder Kalaci 5bea95009b Skip autovacuum processes for distributed deadlock detection 
Autovacuum process cancels itself if any modification starts
on the table in order to avoid blocking your regular Postgres
sessions. That's normal and expected. Thus, any locks held by
autovacuum process cannot involve in a distributed deadlock
since it'll be released if needed.
 2017-11-15 14:32:16 +02:00
Onder Kalaci c65c153a46 Skip speculative locks for distributed deadlock detection 
These locks are held for a very short duration time and cannot
contribute to a deadlock. Speculative locks are used by Postgres
for internal notification mechanism among transactions.
 2017-11-15 12:43:45 +02:00
Marco Slot bbbadd6d1b Bump Citus version to 7.2devel 2017-11-15 10:32:49 +01:00
Marco Slot ea306c6cfe Use citus.next_placement_id where practical in regression tests 2017-11-15 10:12:06 +01:00
Marco Slot d3b634b301 Allow generating placement IDs without using the sequence 2017-11-15 10:12:06 +01:00
Marco Slot 89eb833375 Use citus.next_shard_id where practical in regression tests 2017-11-15 10:12:05 +01:00
Marco Slot c24a0875a5 Allow generating shard IDs without using the sequence 2017-11-15 10:12:05 +01:00
Brian Cloutier 0f3230170f Pull in INT32_MAXINT and INT32_MININT 2017-11-14 14:03:46 -08:00
Brian Cloutier 0db8277266 remove unused errno import 2017-11-14 13:09:34 -08:00
Brian Cloutier 5d9f3ae7fd Remove unused poll import from multi_real_time_executor 2017-11-14 13:09:34 -08:00
Marco Slot 533a533565 Only drop sequences on workers with metadata 2017-11-14 16:01:56 +01:00
velioglu be28ba8e70 Add stub UDF to run pg_upgrade flawlessly 2017-11-13 16:14:45 +02:00
metdos 111c04c2bd Warn on CLUSTER command for distributed tables 2017-11-10 12:14:45 +02:00
Burak Yücesoy 863df0b874
Merge branch 'master' into fix_partitioning_in_schema 2017-11-09 12:49:35 +02:00
Burak Yucesoy 17229ed7bd Fix attaching partition to a distributed table in schema 
While attaching a partition to a distributed table in schema, we mistakenly
used unqualified name to find partitioned table's oid. This caused problems
while using partitioned tables with schemas. We are fixing this issue in
this PR.
 2017-11-09 13:20:29 +03:00
Onder Kalaci 94921a2be1 Skip page-level locks on distributed deadlock detection 
Short-term share/exclusive page-level locks are used for
read/write access. Locks are released immediately after
each index row is fetched or inserted.

Since those locks may not lead to any deadlocks, it's safe
to ignore them in the distributed deadlock detection.
 2017-11-09 10:37:23 +02:00
Marco Slot f71728f634 Add GUC for specifying sslmode in connections to workers 2017-11-08 14:15:58 +01:00
Murat Tuncer 4e3d633ebf
Add check for connection failures during multishard update (#1765) 2017-11-07 12:33:25 +02:00