Commit Graph

7043 Commits (main)

Author SHA1 Message Date
Andres Freund 4cf0a4e48e citus_indent fixups 2016-07-13 11:45:51 -07:00
Brian Cloutier 0cad3b22cc Simplify code and fix include guards in citus_clauses 2016-07-13 11:45:51 -07:00
Brian Cloutier 08384ddc71 cosmetic changes 2016-07-13 11:45:51 -07:00
Brian Cloutier af9515f669 Only reparse queries if the planner flags them for reparsing 2016-07-13 11:45:51 -07:00
Brian Cloutier 4820366a6f citus_indent and some renaming 2016-07-13 11:45:51 -07:00
Brian Cloutier ae91768c96 Evaluate functions on the master
- Enables using VOLATILE functions (like nextval()) in INSERT queries
- Enables using STABLE functions (like now()) targetLists and joinTrees

UPDATE and INSERT can now contain non-immutable functions. INSERT can contain any kind of
expression, while UPDATE can contain any STABLE function, so long as a Var is not passed
into the STABLE function, even indirectly. UPDATE TagetEntry's can now also include Vars.

There's an exception, CASE/COALESCE statements may not contain mutable functions.

Functions calls in master_modify_multiple_shards are also evaluated.
2016-07-13 11:45:51 -07:00
Burak Yücesoy b01d19db3d Merge pull request #611 from citusdata/fix/fix_463_copy_on_array_of_user_defined_types
Fix COPY produces error when using array of user-defined types
2016-07-13 11:43:21 +03:00
Burak Yucesoy cab03a6274 Fix COPY produces error when using array of user-defined types
Fixes #463

OID of user-defined types may be different in master and worker nodes. This causes errors
while sending data between nodes with binary nodes. Because binary copy format adds OID
of the element if it is in an array. The code adding OID is in PostgreSQL code, therefore
we cannot change it. Instead we decided to use text format if we try to send array of
user-defined type.
2016-07-13 11:12:24 +03:00
Jason Petersen 800cb26ffd Merge pull request #639 from citusdata/feature/no_is_null_pruning
Remove hash-pruning logic for NULL values

cr: @sumedhpathak
2016-07-06 17:08:11 -06:00
Jason Petersen 41ed433b0e
Remove hash-pruning logic for NULL values
It turns out some tests exercised this behavior, but removing it should
have no ill effects. Besides, both copy and INSERT disallow NULLs in a
table's partition column.

Fixes a bug where anti-joins on hash-partitioned distributed tables
would incorrectly prune shards early, result in incorrect results (test
included).
2016-07-06 17:04:21 -06:00
Andres Freund 18236b18b5 Merge pull request #578 from citusdata/feature/returning
Support RETURNING
2016-07-01 13:15:36 -07:00
Andres Freund 4549e06884 Add regression tests for RETURNING. 2016-07-01 13:07:12 -07:00
Andres Freund cccba66f24 Support RETURNING for modification commands.
Fixes: #242
2016-07-01 13:07:12 -07:00
Andres Freund c38c1adce1 Combine router executor paths for select and modify commands.
The upcoming RETURNING support would otherwise require too much
duplication.  This contains most of the pieces required for RETURNING
support, except removing the planner checks and adjusting regression
test output.
2016-07-01 13:07:12 -07:00
Andres Freund e1282b6d70 Remember original targetlist in MultiQueryContainerNode().
The old targetlist wasn't used so far, but the upcoming RETURNING
support relies on it.

This also allows to get rid of some crufty code in
multi_executor.c:multi_ExecutorStart(), which used the worker query's
targetlist instead of the main statement's (which didn't have one up to
now).
2016-07-01 12:50:12 -07:00
Andres Freund f78c135e63 Fix definition of faux targetlist element inserted to prevent backward scans.
The targetlist contains TargetEntrys containing expressions, not
expressions directly. That didn't matter so far, but with the upcoming
RETURNING support, the targetlist is inspected to build a TupleDesc.
ExecCleanTypeFromTL hits an assert when looking at something that's not
a TargetEntry.

Mark the entry as resjunk, so it's not actually used.
2016-07-01 12:50:12 -07:00
Andres Freund d5ad8d7db9 Add tests verifying that updates return correct tuple counts.
This unfortunately requires adding a new table, triggering renumbering
of a number of shard ids.
2016-07-01 12:50:12 -07:00
Andres Freund 0b6819f773 Merge pull request #636 from citusdata/fix/634_null_check_sqlStateString
Add null check to SqlStateMatchesCategory()
2016-07-01 12:37:06 -07:00
Metin Doslu e5ecf92328 Add null check to SqlStateMatchesCategory()
Fixes #634
2016-07-01 12:28:46 -07:00
Jason Petersen e064cacea9
Minor formatting fix
Noticed that uncrustify doesn't like the array-of-struct literals, so
omitting them from formatting (at least here).
2016-06-28 13:09:57 -06:00
Jason Petersen 8b788eb899
Use literal instead of constant to fix 9.4 build
PG_UINT32_MAX doesn't exist before 9.5. Missed this because I removed
my assert-enabled builds during packaging work.

Fixes #619
2016-06-28 12:36:14 -06:00
Andres Freund e888b9f6c6 Merge pull request #574 from citusdata/fix/fix_439_missing_flex
Detect flex in citus configure script, instead of relying postgres'.
2016-06-22 11:10:01 -07:00
Andres Freund 700c076629 Provide our own psqlscan.l->psqlscan.l rule.
As postgres's generic .l -> .c Makefile rule uses ifdef - which is
evaluated early, not during rule evaluation - we have to override the
rule, in addition to the detection of FLEX in the previous commit.

Fixes: #439
2016-06-22 11:03:23 -07:00
Andres Freund e244fbb4d3 Detect flex in citus configure script, instead of relying postgres'.
If postgres was compiled without flex support - possible when building
from a tarball, because those contain the flex generated files - citus
compilation would fail, because FLEX is defined as missing.

Add detection for flex, overwriting postgres' detection if one was
found.

Fixes: #439
2016-06-22 11:03:22 -07:00
Jason Petersen 105f8a9132 Merge pull request #615 from citusdata/feature/purge_connections
Purge connection if re-raising error

cr: @anarazel
2016-06-21 10:01:33 -06:00
Jason Petersen 16fc92bf6b
Purge connection if re-raising error
The only way we re-raise an error is if the raiseError flag is true, so
might as well purge connection in that block rather than independently
checking errorLevel.
2016-06-21 09:51:12 -06:00
Murat Tuncer aa94a0b1ca Merge pull request #610 from citusdata/501_single_worker_queries
Refactor multi_planner to create router plan directly
2016-06-21 12:54:30 +03:00
Murat Tuncer fb99585ca5 Refactor multi_planner to create router plan directly
If router plan creation fails, it falls back to normal planner
2016-06-21 12:50:21 +03:00
Jason Petersen 88da69df8a
Add CHANGELOG entries for 5.1.1 release
cr: @sumedhpathak
2016-06-17 16:03:32 -06:00
Burak Yücesoy 65609c93ec Merge pull request #602 from citusdata/fix/fix_78_append_table_to_shard_with_schema
Fix master_append_table_to_shard to work with schemas
2016-06-17 04:46:20 +03:00
Burak Yucesoy 78aaad2738 Fix master_append_table_to_shard to work with schemas
Fixes #78

With this change, it is possible to append a table in any schema to shard. The function
master_append_table_to_shard now supports schema names.
2016-06-17 04:35:00 +03:00
Andres Freund e0d52a0e72 Merge pull request #608 from citusdata/fix_pruning_performance
Fix join pruning performance
2016-06-16 17:43:43 -07:00
Andres Freund 2e8e8d377e Store ShardInterval instead of shardId in RangeTableFragments.
For CITUS_RTE_RELATION type fragments, reloading shardIntervals from the
database is rather expensive. So store a pointer to the full shard
interval, instead of just the shard id.  There's no new memory lifetime
hazards here, because we already passed a pointer to the shardInterval's
->shardId field around.

The plan time for the query in issue #607 goes from 2889 ms to 106 ms.
with this change.
2016-06-16 17:31:35 -07:00
Andres Freund 211a9721a9 Use cached comparator in ShardIntervalsOverlap().
By far the most expensive part of ShardIntervalsOverlap() is computing
the function to use to determine overlap. Luckily we already have that
computed and cached.

The plan time for the query in issue #607 goes from 8764 ms to 2889 ms
with this change.
2016-06-16 17:21:19 -07:00
Andres Freund e697bf7ca7 Merge pull request #603 from citusdata/fix/outer-join-filter
Do not copy outer join clauses into WHERE
2016-06-16 17:16:49 -07:00
Andres Freund 38f4722f6f Add tests for LEFT JOIN ON clauses preventing matches left/right. 2016-06-16 16:53:02 -07:00
Marco Slot 52bc209c37 Do not copy outer join clauses into WHERE 2016-06-16 16:42:32 -07:00
Andres Freund bac1ac93ef Merge pull request #604 from citusdata/fix_alter_extension
Fix Alter Extension
2016-06-16 14:35:30 -07:00
Metin Doslu 7d18488676 Drop function from public and create in pg_catalog
Fixes #600
2016-06-16 14:08:40 -07:00
Hanna Wischmann 48b5d15741 Fix the documentation link from Citus 5.0 to Citus 5.1 (#593) 2016-06-16 10:45:36 -07:00
Murat Tuncer 4465b1d495 Merge pull request #585 from citusdata/faster_regression_tests
Faster regression tests
2016-06-15 16:48:51 +03:00
Murat Tuncer 9cb6a022a5 Reduce regression test runtime
-Added 2 more schedules for task-tracker and multi-binary
 instead of running multi_schedule 3 times
-set task-tracker-delay for each long running schedule
2016-06-15 16:35:07 +03:00
Metin Döşlü 936fab3610 Merge pull request #588 from citusdata/fix/fix_550_append_shardId_before_escaping_table_names
Append shardId before escaping the table name
2016-06-15 14:48:11 +03:00
Burak Yucesoy 4a718d293b Append shardId before escaping the table name
Fixes #550, fixes #545

If table name contains special characters, it needs to be escaped. However in some cases,
we escape table name before appending shardId, which causes syntax error in the queries
sent to worker nodes. With this change we now append shardId before escaping table names.
2016-06-15 04:15:40 +03:00
Murat Tuncer 27bf5f2ab2 Merge pull request #586 from citusdata/fix_275_remove_variant_files_from_tests
Remove variant files from regression tests
2016-06-13 12:26:51 +03:00
Murat Tuncer 31df82ba7a Remove variant files
This checkin removes variant files we needed
due to differences in outputs of pg94 and pg95 runs.
However, variant file for test multi_upsert stays
since this file tests for a feature that does not
exist in pg94, and outputs are drastically different.
2016-06-13 12:12:06 +03:00
Ahmet Eren Basak bbf13be508 Merge pull request #591 from citusdata/fix_compile_time_warning_in_multi_logical_optimizer
Eliminate compile time warnings in multi_logical_optimizer.c
2016-06-10 12:33:18 +03:00
Eren 57256b3476 Eliminate compile time warnings in multi_logical_optimizer.c
This change removes some issues about mixed declarations
and code in TablePartitioningSupportsDistinct() and
WorkerExtendedOpNode() functions.
2016-06-10 12:27:12 +03:00
Murat Tuncer 4e4aab0f05 Merge pull request #563 from citusdata/fix/345_task_cleanup_logic
make task cleanup logic less error prone
2016-06-09 17:53:35 +03:00
Murat Tuncer bb3eee63e7 Refactor task tracker cleanup to enable workers receive cleanup jobs
Long sleep is replaced by multiple small sleeps. Maximum timeout
is also increased since we do not have to wait for that long most
of the cases.
2016-06-09 17:03:54 +03:00