Commit Graph

1357 Commits (f5b038e2bc9d60ddac59b03ecca857a0e8a8244b)

Author SHA1 Message Date
Philip Dubé f5b038e2bc Fixes from testing. Seems to be working 2019-10-18 18:45:40 +00:00
Philip Dubé 070ffc785f Fix compile issues 2019-10-18 18:45:40 +00:00
Philip Dubé 1545cb312c Sketched out rest of how mark_aggregate_for_distributed_execution should look like 2019-10-18 18:45:40 +00:00
Philip Dubé 020921f1eb Fixes from testing with create_distributed_function 2019-10-18 18:45:40 +00:00
Philip Dubé 47318ee7e0 Fix logging messages a bit 2019-10-18 18:45:40 +00:00
Philip Dubé 79f4fdbc89 Use pg_dist_object, remove pg_dist_enabled_custom_aggregates 2019-10-18 18:45:40 +00:00
Philip Dubé dd5916ef3a Move custom aggregates into a not-default path, add pg_dist_enabled_custom_aggregates to have a set of allowed aggregates as it's possible for aggregates to not be compatible with this implementation 2019-10-18 18:45:40 +00:00
Philip Dubé 9d19f2d810 aggregate_utils: remove elog debugging 2019-10-18 18:45:40 +00:00
Philip Dubé d1aefe240b Now able to figure out that the average of 1,2,3 is 2. Next: clean up all the logspam, make sure value_init is being handled properly everywhere 2019-10-18 18:45:40 +00:00
Philip Dubé 5b25204dda Gets as far as calling array_send, which seems to be getting an invalid array, maybe a memory context issue 2019-10-18 18:45:40 +00:00
Philip Dubé 22c6dc7f84 Fix compiling against pg12 2019-10-18 18:45:40 +00:00
Philip Dubé 17ed25e09a Fix multi_logical_optimizer merge regression. aggregate_utils still needs fixing 2019-10-18 18:45:40 +00:00
Philip Dubé 5d6ee5104b Get things working enough that code is executing & giving nonsense like avg() wanting to create arrays with 16777216 dimensions 2019-10-18 18:45:40 +00:00
Philip Dubé 1a7bf4ca2c Initial sketch of aggregate expression transforms 2019-10-18 18:45:40 +00:00
Philip Dubé c801889e08 return the right bytes 2019-10-18 18:45:40 +00:00
Philip Dubé baef1d2f85 Clean up code to point that it compiles 2019-10-18 18:45:40 +00:00
Philip Dubé 186f632f88 ReleaseSysCache in logical_optimizer 2019-10-18 18:45:40 +00:00
Philip Dubé a82fa8831c First rough draft of aggregate logic done. Still need query rewriting logic before being able to test 2019-10-18 18:45:40 +00:00
Philip Dubé 77fc5f91ab Pseudocode mostly restored from deleting my repo yesterday, started writing broken pseudo-C 2019-10-18 18:45:40 +00:00
Philip Dubé 40a5e3128b Initial custom aggregates. Very WIP. Only a branch because I like backups. Will squash someday 2019-10-18 18:45:40 +00:00
Philip Dubé c32bd459f4 create_distributed_function: accept aggregates 2019-10-18 18:45:40 +00:00
Jelte Fennema 7abedc38b0
Support subqueries in HAVING (#3098)
Areas for further optimization:
- Don't save subquery results to a local file on the coordinator when the subquery is not in the having clause
- Push the the HAVING with subquery to the workers if there's a group by on the distribution column
- Don't push down the results to the workers when we don't push down the HAVING clause, only the coordinator needs it

Fixes #520
Fixes #756
Closes #2047
2019-10-16 16:40:14 +02:00
Onur TIRTIR 3bfb2a078b
Make changes on if-statement in ExtractRangeTableList for furhter walker types (#3110) 2019-10-16 15:50:09 +03:00
Onur TIRTIR d5f83dc110
Refactor range table walkers (#3109) 2019-10-16 01:20:49 +03:00
SaitTalhaNisanci 94a7e6475c
Remove copyright years (#2918)
* Update year as 2012-2019

* Remove copyright years
2019-10-15 17:44:30 +03:00
Philip Dubé 74cb168205 Remove Postgres 10 support 2019-10-11 21:56:56 +00:00
Hadi Moshayedi b50d216536 Fix a typo 2019-10-10 10:44:41 -07:00
Philip Dubé 4063e7ca67 CALL delegation: apply strip_implicit_coercions to distribution argument 2019-10-10 17:42:43 +00:00
Philip Dubé dd490b6376 Cache whether an object is in pg_dist_object. Avoids redundant lookups for non-distributed objects 2019-10-10 14:50:38 +00:00
Nils Dijk 4a4a220945
Fix enum add value order and pg12 (#3082)
DESCRIPTION: Fix order for enum values and correctly support pg12

PG 12 introduces `ALTER TYPE ... ADD VALUE ...` during transactions. Earlier versions would error out when called in a transaction, hence we connect to workers outside of the transaction which could cause inconsistencies on pg12 now that postgres doesn't error with this syntax anymore.

During the implementation of this fix it became apparent there was an error with the ordering of enum labels when the type was recreated. A patch and test have been included.
2019-10-07 17:16:19 +02:00
Jelte Fennema 01da11f264
Change citus truncate trigger to AFTER and add more upgrade tests (#3070)
* Add more upgrade tests

* Fix citus trigger generation after upgrade

citus_truncate_trigger runs before truncate when created by create_distributed_table:
492d1b2cba/src/backend/distributed/commands/create_distributed_table.c (L1163)

* Remove pg_dist_jobid_seq
2019-10-07 16:43:04 +02:00
Onder Kalaci 3be72ce42f Make sure that distributed functions always have the correct user
Objectives:

(a) both super user and regular user should have the correct owner for the function on the worker
(b) The transactional semantics would work fine for both super user and regular user
(c) non-super-user and non-function owner would get a reasonable error message if tries to distribute the function

Co-authored-by: @serprex
2019-10-04 21:38:49 +00:00
Marco Slot 1a3a174f67 Grant usage on schema citus to public 2019-10-04 12:26:08 +02:00
Marco Slot 89377ee578 Move RowExclusiveLock to start in SyncMetadataToNodes 2019-10-04 12:07:41 +02:00
Hadi Moshayedi 217db2a03e Don't block for locks in SyncMetadataToNodes() 2019-10-03 16:53:36 -07:00
Hadi Moshayedi ae915493e6 Don't send metadata commands to not-synced workers.
Otherwise some of the dependencies might not exist yet and
commands will error out.
2019-10-03 16:52:25 -07:00
Marco Slot 0b4b63e647 Drop the rebalancer before creating new UDFs 2019-10-03 16:08:58 +02:00
Marco Slot 2e50306cf8 Check command type in TryToDelegateFunctionCall 2019-10-03 15:37:15 +02:00
Hanefi Onaldi bd416ef68f Fix empty FROM clauses in PG12 2019-10-01 19:54:11 +00:00
Jelte Fennema ec4a165eec Improve isolation test block detection (#3055) 2019-10-01 14:10:15 +02:00
Jelte Fennema 40f785e6d8 Move citus_isolation_test_session_is_blocked to separate udf sql file 2019-10-01 14:10:15 +02:00
Philip Dubé 89d35e9692 Attempt to force custom plans for prepared statements when trying to delegate function calls
We discern between PARAM_EXEC & PARAM_EXTERN:
d52eaa0948/src/include/nodes/primnodes.h (L211)
According to primnodes.h we should only run into PARAM_EXEC or PARAM_EXTERN
2019-09-30 23:49:14 +00:00
Philip Dubé 29f1ea079b PG_VERSION_NUM > 110000 should be PG_VERSION_NUM >= 110000
Also fix a > 12000 typo
2019-09-30 23:37:43 +00:00
Hadi Moshayedi 5e97e5c98e Don't push down queries when in subqueries/ctes 2019-09-30 14:22:05 -07:00
Marco Slot 35bef0f3db Avoid caching connections from backends that servicei internal connections 2019-09-28 08:32:10 +02:00
Nils Dijk 01b26cf91a
Disallow distributed functions for functions depending on an extension (#3049)
DESCRIPTION: Disallow distributed functions for functions depending on an extension

Functions depending on an extension cannot (yet) be distributed by citus. If we would allow this it would cause issues with our dependency following mechanism as we stop following objects depending on an extension.

By not allowing functions to be distributed when they depend on an extension as well as not allowing to make distributed functions depend on an extension we won't break the ability to add new nodes. Allowing functions depending on extensions to be distributed at the moment could cause problems in that area.
2019-09-30 15:19:47 +02:00
Nils Dijk 473cbc0115
Propagate CREATE OR REPLACE FUNCTION to workers for distributed functions (#3043)
DESCRIPTION: Propagate CREATE OR REPLACE FUNCTION

Distributed functions could be replaced, which should be propagated to the workers to keep the function in sync between all nodes.

Due to the complexity of deparsing the `CreateFunctionStmt` we actually produce the plan during the processing phase of our utilityhook. Since the changes have already been made in the catalog tables we can reuse `pg_get_functiondef` to get us the generated `CREATE OR REPLACE` sql.
2019-09-30 12:41:17 +02:00
Jelte Fennema 82ec918b29
Add explain summary support (#3046)
Fixes #2922 and also adds explain analyze regression tests
2019-09-30 10:58:49 +02:00
Nils Dijk 9c2c50d875
Hookup function/procedure deparsing to our utility hook (#3041)
DESCRIPTION: Propagate ALTER FUNCTION statements for distributed functions

Using the implemented deparser for function statements to propagate changes to both functions and procedures that are previously distributed.
2019-09-27 22:06:49 +02:00
Philip Dubé 363409a0c2 Propagate REINDEX TABLE & REINDEX INDEX 2019-09-27 18:14:53 +00:00