Commit Graph

2842 Commits (86a4c0925bbbe9fdfb27103f9d223c0a2bc0dbf8)

Author SHA1 Message Date
Halil Ozan Akgül e32592221a
Merge pull request #3060 from citusdata/mx_isolation_dis2ref_foreign_key
MX Isolation Tests for Foreign keys from Distributrion to Reference tables vs DML
2019-10-03 09:52:02 +03:00
Halil Ozan Akgul bda8f6f87b Created tests for distribution to reference table foreign keys on mx 2019-10-03 09:31:13 +03:00
SaitTalhaNisanci 19bdca14d8
Add jobs to run tests with pg 12 (#3033)
* Add PG12 test outputs

* Add jobs to run tests with pg 12

* use POSIX collate for compatibility between pg10/pg11/pg12

* do not override the new default value when running vanilla tests

* fix 2 problems with pg12 tests

* update pg12 images with pg12 rc1

* remove pg10 jobs

* Revert "Add PG12 test outputs"

This reverts commit f3545b92ef.

* change images to use latest instead of dev

* add missing coverage flags
2019-10-02 15:33:12 +03:00
Halil Ozan Akgül 3f83b726e0
Merge pull request #3044 from citusdata/mx_ref_isolation_update_delete_upsert
MX Isolation Tests for Update, Delete, Upsert on Reference Tables
2019-10-02 10:27:47 +03:00
Halil Ozan Akgul e5906bead2 Created isolation tests for update, delete, upsert on reference tables with MX. 2019-10-02 10:11:21 +03:00
Hanefi Onaldi bd416ef68f Fix empty FROM clauses in PG12 2019-10-01 19:54:11 +00:00
Halil Ozan Akgül 9fa4a20148
Merge pull request #3045 from citusdata/mx_ref_isolation_select_for_update
MX Isolation Tests for SELECT FOR UPDATE on Reference Tables
2019-10-01 16:48:56 +03:00
Halil Ozan Akgul 1d7030a651 Created isolation tests for select for update on reference tables with MX. 2019-10-01 16:29:15 +03:00
Jelte Fennema ec4a165eec Improve isolation test block detection (#3055) 2019-10-01 14:10:15 +02:00
Jelte Fennema 40f785e6d8 Move citus_isolation_test_session_is_blocked to separate udf sql file 2019-10-01 14:10:15 +02:00
Philip Dubé 89d35e9692 Attempt to force custom plans for prepared statements when trying to delegate function calls
We discern between PARAM_EXEC & PARAM_EXTERN:
d52eaa0948/src/include/nodes/primnodes.h (L211)
According to primnodes.h we should only run into PARAM_EXEC or PARAM_EXTERN
2019-09-30 23:49:14 +00:00
Philip Dubé 29f1ea079b PG_VERSION_NUM > 110000 should be PG_VERSION_NUM >= 110000
Also fix a > 12000 typo
2019-09-30 23:37:43 +00:00
Hadi Moshayedi 1989ff85b5
Merge pull request #3059 from citusdata/cte_function_calls
Don't push down queries when in subqueries/ctes
2019-09-30 15:56:24 -07:00
Hadi Moshayedi 5e97e5c98e Don't push down queries when in subqueries/ctes 2019-09-30 14:22:05 -07:00
Marco Slot 2d1639ad9d
Merge pull request #3052 from citusdata/fix/internal_connection_caching
Avoid caching connections from backends that service internal connections
2019-09-30 17:07:03 +02:00
Nils Dijk 01b26cf91a
Disallow distributed functions for functions depending on an extension (#3049)
DESCRIPTION: Disallow distributed functions for functions depending on an extension

Functions depending on an extension cannot (yet) be distributed by citus. If we would allow this it would cause issues with our dependency following mechanism as we stop following objects depending on an extension.

By not allowing functions to be distributed when they depend on an extension as well as not allowing to make distributed functions depend on an extension we won't break the ability to add new nodes. Allowing functions depending on extensions to be distributed at the moment could cause problems in that area.
2019-09-30 15:19:47 +02:00
Jelte Fennema 4d991f281c
Update CircleCI codecov orb (#3050)
Hopefully this fixes the long runtime for coverage uploads that we sometimes have (> 1 minute).
2019-09-30 15:04:49 +02:00
Nils Dijk 473cbc0115
Propagate CREATE OR REPLACE FUNCTION to workers for distributed functions (#3043)
DESCRIPTION: Propagate CREATE OR REPLACE FUNCTION

Distributed functions could be replaced, which should be propagated to the workers to keep the function in sync between all nodes.

Due to the complexity of deparsing the `CreateFunctionStmt` we actually produce the plan during the processing phase of our utilityhook. Since the changes have already been made in the catalog tables we can reuse `pg_get_functiondef` to get us the generated `CREATE OR REPLACE` sql.
2019-09-30 12:41:17 +02:00
Jelte Fennema 82ec918b29
Add explain summary support (#3046)
Fixes #2922 and also adds explain analyze regression tests
2019-09-30 10:58:49 +02:00
Marco Slot 35bef0f3db Avoid caching connections from backends that servicei internal connections 2019-09-28 08:32:10 +02:00
Nils Dijk 9c2c50d875
Hookup function/procedure deparsing to our utility hook (#3041)
DESCRIPTION: Propagate ALTER FUNCTION statements for distributed functions

Using the implemented deparser for function statements to propagate changes to both functions and procedures that are previously distributed.
2019-09-27 22:06:49 +02:00
Philip Dubé 363409a0c2 Propagate REINDEX TABLE & REINDEX INDEX 2019-09-27 18:14:53 +00:00
Hanefi Onaldi 66b9f2e887 Deparsing and qualifiying for FUNCTION/PROCEDURE statements (#3014)
This PR aims to add all the necessary logic to qualify and deparse all possible `{ALTER|DROP} .. {FUNCTION|PROCEDURE}` queries.

As Procedures are introduced in PG11, the code contains many PG version checks. I tried my best to make it easy to clean up once we drop PG10 support.


Here are some caveats:
- I assumed that the parse tree is a valid one. There are some queries that are not allowed, but still are parsed successfully by postgres planner. Such queries will result in errors in execution time. (e.g. `ALTER PROCEDURE p STRICT` -> `STRICT` action is valid for functions but not procedures. Postgres decides to parse them nevertheless.)
2019-09-27 19:02:52 +02:00
Hadi Moshayedi 28acab9d02
Merge pull request #3021 from citusdata/distribute-select
Distribute select function
2019-09-27 09:35:24 -07:00
Marco Slot 2868e02a3d Implement SELECT function call delegation.
When a function is marked as colocated with a distributed table,
we try delegating queries of kind "SELECT func(...)" to workers.

We currently only support this simple form, and don't delegate
forms like "SELECT f1(...), f2(...)", "SELECT f1(...) FROM ...",
or function calls inside transactions.

As a side effect, we also fix the transactional semantics of DO blocks.
Previously we didn't consider a DO block a multi-statement transaction.
Now we do.

Co-authored-by: Marco Slot <marco@citusdata.com>
Co-authored-by: serprex <serprex@users.noreply.github.com>
Co-authored-by: pykello <hadi.moshayedi@microsoft.com>
2019-09-27 09:13:25 -07:00
Jelte Fennema dab16be283
Set default threshold on get_rebalance_table_shards_plan to 0, like rebalance_table_shards (#3039)
In this PR the default `threshold` of `rebalance_table_shards` was set to 0: https://github.com/citusdata/shard_rebalancer/pull/73
However, the default for get_rebalance_table_shards_plan was not updated. This
can cause the confusing situation where the actual steps run by
`rebalance_table_shards` are not the same as the ones returned by
`get_rebalance_table_shards_plan`.
2019-09-27 17:21:36 +02:00
Jelte Fennema 6adf64efdb
Hotfix for circleci problem where directory is owned by root (#3048) 2019-09-27 17:07:30 +02:00
Halil Ozan Akgül bbe3ec0493
Merge pull request #3015 from citusdata/mx_isolation_test_insert_select
MX Isolation tests for Insert Select
2019-09-26 18:15:22 +03:00
Halil Ozan Akgul 824a69587c Created isolation tests for insert select on MX 2019-09-26 17:40:36 +03:00
Marco Slot 32a11bdf6c Return early for common commands in the utility hook (#3031)
We started copying parse trees by default further on in `multi_ProcessUtility`. That's not a problem for maintenance command, but might register for things like `PREPARE` and `EXECUTE`, which might happen thousands of times per second. Add a few common commands to the check at the start.
2019-09-26 11:43:35 +02:00
Halil Ozan Akgül 3e465a6449
Merge pull request #3028 from citusdata/mx_isolation_test_drop_alter_index_select_for_update
MX Isolation Tests for Drop, Alter, Index and Select For Update
2019-09-26 10:56:08 +03:00
Halil Ozan Akgul d56ab6274c Created isolation tests for drop, alter, index and select for update on MX. 2019-09-26 10:47:14 +03:00
SaitTalhaNisanci e3dcc9504f
Update all docker images with pg 11.5 (#3012) 2019-09-26 10:44:18 +03:00
SaitTalhaNisanci 24a56d2257
Add postgres upgrade job (#2973) 2019-09-25 17:31:19 +03:00
Halil Ozan Akgül cc5d68577a
Merge pull request #3025 from citusdata/mx_isolation_test_truncate
MX Isolation Tests for Truncate
2019-09-25 17:14:42 +03:00
Halil Ozan Akgul d426fb2159 Created isolation tests for truncate on MX. 2019-09-25 16:51:20 +03:00
Halil Ozan Akgül 198535b752
Merge pull request #3010 from citusdata/mx_isolation_test_copy
MX Isolation Tests for Copy
2019-09-25 15:54:45 +03:00
Halil Ozan Akgul 62b6852923 Created isolation tests for copy on MX. 2019-09-25 15:36:05 +03:00
Önder Kalacı cefedf5b00
Merge pull request #3026 from citusdata/improve_test
Improve some tests around local execution and CTE inlining on pg 12
2019-09-25 11:07:28 +02:00
Onder Kalaci 219f3676a0 Improve some tests around local execution and CTE inlining on pg 12 2019-09-25 10:53:19 +02:00
Philip Dubé 4f60e3a149 Feedback 2019-09-24 17:31:09 +00:00
Marco Slot c1e43b25da Use the new create_distributed_function API in some call tests 2019-09-24 17:31:09 +00:00
Marco Slot ca478defeb Deparse CALL statement instead of using original query string 2019-09-24 17:31:09 +00:00
Philip Dubé 90e1f1442a Annotated tests for multi_mx_call.
Co-authored-by: pykello <hadi.moshayedi@microsoft.com>
2019-09-24 17:31:09 +00:00
Marco Slot e269d990c9 Cast the distribution argument value when possible 2019-09-24 17:31:09 +00:00
Philip Dubé c95d46b4f3 Extend multi_mx_call with some of Hadi's suggestions for better test coverage 2019-09-24 17:31:09 +00:00
Philip Dubé 432a8ef85b Hadi's feedback
Co-authored-by: pykello <hadi.moshayedi@microsoft.com>
Co-authored-by: serprex <serprex@users.noreply.github.com>
2019-09-24 17:31:09 +00:00
Philip Dubé 16b8d17aba Test: multi_mx_call 2019-09-24 17:31:09 +00:00
Philip Dubé bc1ad67eb5 Distribute CALL on distributed procedures to metadata workers
Lots taken from https://github.com/citusdata/citus/pull/2829
2019-09-24 17:31:09 +00:00
Önder Kalacı 932a407f07
Merge pull request #3029 from citusdata/relax_colocation_checks
Relax the colocation checks for distributed functions
2019-09-24 16:37:36 +02:00