Commit Graph

1821 Commits (f8cfe07fd11e5caeddac93460e94cd72a23b1d2d)

Author SHA1 Message Date
Marco Slot f8cfe07fd1 Support intermediate results in distributed INSERT..SELECT 2018-05-03 14:42:28 +02:00
Marco Slot 90cdfff602 Implement recursive planning for DML statements 2018-05-03 14:42:28 +02:00
Murat Tuncer 42a8082721 PG11 compatibility refresh
adds a shim for a changed function api
2018-05-03 13:21:15 -06:00
Mehmet Furkan ŞAHİN 711128671a
Merge pull request #2141 from citusdata/regression_tests_large_shard_count2
shard count for some of the tests are increased
2018-05-03 11:05:55 +03:00
mehmet furkan şahin ef90122cd3 shard count for some of the tests are increased 2018-05-03 10:44:43 +03:00
Önder Kalacı 7fd4383886
Merge pull request #2075 from citusdata/hash_single_repartition
Implement hash-repartitioning for single repartition joins (e.g., enable single repartition joins among hash distributed tables)
2018-05-02 22:07:31 +03:00
Onder Kalaci 317dd02a2f Implement single repartitioning on hash distributed tables
* Change worker_hash_partition_table() such that the
     divergence between Citus planner's hashing and
     worker_hash_partition_table() becomes the same.

   * Rename single partitioning to single range partitioning.

   * Add single hash repartitioning. Basically, logical planner
     treats single hash and range partitioning almost equally.
     Physical planner, on the other hand, treats single hash and
     dual hash repartitioning almost equally (except for JoinPruning).

   * Add a new GUC to enable this feature
2018-05-02 18:50:55 +03:00
Burak Velioglu 7850f93127
Merge pull request #2124 from citusdata/ot_msud_subquery
Support UPDATE/DELETE with joins and subqueries
2018-05-02 17:25:07 +03:00
velioglu 32bcd610c1 Support modify queries with multiple tables
With this commit we begin to support modify queries with multiple
tables if these queries are pushdownable.
2018-05-02 16:22:26 +03:00
Burak Velioglu 38bdd51dea
Merge pull request #2104 from citusdata/refactor_subquery
Refactor query pushdown related logic
2018-05-02 15:27:31 +03:00
velioglu d9fa69c031 Refactor query pushdown related logic 2018-05-02 15:03:09 +03:00
Brian Cloutier f8fb7a27fb Don't copyObject into the wrong memory context
utilityStmt sometimes (such as when it's inside of a plpgsql function)
comes from a cached plan, which is kept in a child of the
CacheMemoryContext. When we naively call copyObject we're copying it into
a statement-local context, which corrupts the cached plan when it's
thrown away.
2018-05-01 15:34:32 -07:00
Marco Slot 663c9b2a39
Merge pull request #2138 from citusdata/drop_shards_users
Make DROP TABLE connect as current user instead of superuser
2018-05-01 11:22:40 +02:00
Marco Slot 2559b84049 Drop shards as current user instead of super user 2018-05-01 09:57:20 +02:00
Marco Slot 64280a2329
Merge pull request #1991 from citusdata/remove_broadcast_pr4
Remove usages of large_table_shard_count GUC
2018-04-30 18:22:31 +02:00
Önder Kalacı b1e6636398
Merge pull request #2107 from citusdata/simplify_optimizer_phase_3
Simplify optimizer [Phase 3]  - Move processing each part of the query into its own functions
2018-04-29 12:56:17 +03:00
velioglu 121ff39b26 Removes large_table_shard_count GUC 2018-04-29 10:34:50 +02:00
Onder Kalaci 832c91e28c Move processing each part of the query into its own functions
This commit doesn't change any of the logic at all.

Instead, the goal is to:

 * Get rid of any code duplication
 * Incremental changes to the optimizer made it slightly hard
   to follow the code, improve that and make it easier to
   implement new features
 * Simplify the code by moving each part of query processing (e.g.,
   DISTINCT, LIMIT etc) into its own function
 * Make the interaction between each part of the query more
   obvious (e.g., How DISTINCT affects LIMIT etc)
2018-04-27 17:32:38 +03:00
Mehmet Furkan ŞAHİN ca2818b569
Merge pull request #2121 from citusdata/vacuum_analyze_verbose_support
Add support for (VACUUM | ANALYZE) VERBOSE
2018-04-27 16:04:35 +03:00
mehmet furkan şahin f2555317b6 ProcessVacuumStmt update on names 2018-04-27 14:37:01 +03:00
mehmet furkan şahin a4153c6ab1 notice handler is implemented 2018-04-27 14:37:01 +03:00
Marco Slot 304b3a41ba Cache the partition column Var 2018-04-26 14:58:16 -06:00
Jason Petersen 643059860a
Merge pull request #2123 from citusdata/improve_connection_errors
Improve connection error reporting

cr: @jasonmp85
2018-04-26 13:59:59 -06:00
Marco Slot 3d3c19a717
Improve messages for essential connection failures 2018-04-26 12:58:47 -06:00
Marco Slot 88f64d22db
Prevent connection pointer is NULL details 2018-04-26 12:49:57 -06:00
Marco Slot 394732b6be
Add a connection failure error code 2018-04-26 12:49:57 -06:00
Önder Kalacı ebb8f902c8 Relax assertion on transaction rollback failure (#2052)
In case a failure happens when a transaction is rollbacked,
we used to hit an assertion for ensuring there is no pending
activity on the connection. However, that's not true after the
changes in #2031. Thus, we've replaced the assertion with a more
generic function call to consume any pending activity, if exists.
2018-04-26 13:39:03 -04:00
Hadi Moshayedi 24659a97dc
Fail task in real-time executor if no placements found. (#2133) 2018-04-26 12:05:24 -04:00
Murat Tuncer 9610fe70f8
Merge pull request #2118 from citusdata/pg11_compat
PG 11 compatibility refresh
2018-04-26 11:54:00 +03:00
Murat Tuncer a6fe5ca183 PG11 compatibility update
- changes in ruleutils_11.c is reflected
- vacuum statement api change is handled. We now allow
  multi-table vacuum commands.
- some other function header changes are reflected
- api conflicts between PG11 and earlier versions
  are handled by adding shims in version_compat.h
- various regression tests are fixed due output and
  functionality in PG1
- no change is made to support new features in PG11
  they need to be handled by new commit
2018-04-26 11:29:43 +03:00
Brian Cloutier 49255213d4 Configure appveyor to run regression tests
- Add install.pl to instal .sql files on Windows
- Remove a hack to PGDLLIMPORT some variables
- Add citus_version.o to the Makefile
- Fix pg_regress_multi's PATH generation on Windows
- Output regression.diffs when the tests fail
- Fix permissions in data directory, make sure postgres can play with it
2018-04-25 18:02:07 -07:00
Önder Kalacı f0b50f2f99
Merge pull request #2090 from citusdata/simplify_optimizer_phase_2
Refactor logical optimizer [Phase 2] - Eliminate code duplication in `WorkerExtendedOpNode()`
2018-04-25 09:49:05 +03:00
Onder Kalaci ac8f2f1e6d Eliminate code duplication in WorkerExtendedOpNode()
Before this commit, we had code duplication in the
WorkerExtendedOpNode(). The duplication was
noticeable and any change is prone to bugs.

The PR consists of 4 commits. Each commit incrementally
fixes the problem by moving certain parts of the duplicated
code into smaller, better-documented functions.
2018-04-25 08:54:59 +03:00
Brian Cloutier 8d4c4d5c58 Close all files before trying to remove them 2018-04-24 14:35:20 -07:00
Brian Cloutier c5f1235090 Turn the crashes on Windows into WARNINGs 2018-04-24 14:35:20 -07:00
Önder Kalacı 18cb93c107
Merge pull request #2070 from citusdata/simplify_optimizer
Refactor logical optimizer [Phase 1] - Unify `extendedOpNode` creation
2018-04-24 14:14:16 +03:00
Onder Kalaci ee748d9140 Unify extendedOpNode Processing
Before this commit, we had a divergence among
the creation of master/worker extended op nodes.

This commit moves the related parts into a single place
and allows the creation of master/extended op nodes to
share a common data structure.
2018-04-24 11:56:38 +03:00
Hadi Moshayedi 966f01fad3
Fix write and copy functions for TaskExecution. (#2120)
We were missing criticalErrorOccurred from CopyNodeTaskExecution() and OutTaskExecution(). This PR fixes it.
2018-04-23 09:07:52 -04:00
Önder Kalacı 6c6cddff04
Merge pull request #2105 from citusdata/fix_null_access
Citus should not consider subqueries that are removed by PostgreSQL planner
2018-04-21 04:23:47 +03:00
Onder Kalaci 814f0e3acc Ensure Citus never try to access a not planned subquery
PostgreSQL might remove some of the subqueries when they do not
contribute to the query result at all. Citus should not try to
access such subqueries during planning.
2018-04-20 13:52:00 +03:00
Brian Cloutier b0b130f064 Fix Windows crash in multi_copy test
Without this change we crash on Windows with COPYing into a table with
62 shards, and we ERROR when COPYing into a table with >62 shards:

ERROR: WaitForMutipleObjects() failed: error code 87
2018-04-17 15:48:02 -07:00
Brian Cloutier d02f761d8e Change intermediate_results test to not crash 2018-04-17 15:14:02 -07:00
Brian Cloutier 0104790385 Fix hard-coded temp directory in multi_copy
/tmp does not exist on Windows, use :temp_dir instead
2018-04-17 15:01:22 -07:00
Brian Cloutier a59c1c634e Fix cancellation of real time queries
Without this change multi_real_time_transaction blocks forever (on
Windows) in the block where it repeatedly calls pg_advisory_lock(15).
This happens because the deadlock detector tries to cancel the backend
but the backend never processes that signal.
2018-04-17 14:26:22 -07:00
Mehmet Furkan ŞAHİN cd43d48608
Merge pull request #2109 from citusdata/capital_schema_support
Quotation needy schema name support
2018-04-17 22:21:55 +03:00
mehmet furkan şahin 00e786af00 Capital named schema support is added 2018-04-17 17:17:42 +03:00
Mehmet Furkan ŞAHİN de6d3f2d33
Merge pull request #2102 from citusdata/push_down_multiple_having
This is a fairly simple PR that changes the AND clauses in having explicit for worker queries for pushdown planner. Since, they are going to be switched back to be implicit in worker itself, we should provide them in explicit form. Otherwise, the worker errors-out saying the query syntax is wrong.
2018-04-16 15:30:05 +03:00
mehmet furkan şahin e5a5502b16 Adds support for multiple ANDs in Having
This PR adds support for multiple AND expressions in Having
for pushdown planner. We simply make a call to make_ands_explicit
from MultiLogicalPlanOptimize for the having qual in
workerExtendedOpNode.
2018-04-16 14:14:48 +03:00
Brian Cloutier 42ddfa176d Fix crash on Windows where there is no detail 2018-04-13 12:54:22 -07:00
Burak Velioglu f54ed5d3b0
Merge pull request #1980 from citusdata/remove_boradcast_pr3_v2
Convert broadcast join to reference join
2018-04-13 16:44:55 +03:00