external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,091 Commits
636 Branches
162 Tags
98 MiB
Commit Graph

3091 Commits (70a332ffb68aaf321b2c46a4bc137a9d0d88abcc)

Author SHA1 Message Date
Jelte Fennema 70a332ffb6 Use planContext in createDistributedPlan 2020-01-06 13:05:35 +01:00
Jelte Fennema 7c76a5ab95 Rename planContext->parse to planContext->query 2020-01-06 12:56:08 +01:00
Jelte Fennema d4352646b9 Move check into TargetShardIntervalForFastPathQuery 2020-01-06 12:42:43 +01:00
Jelte Fennema 026db60075 Simplify some more 2020-01-06 12:42:43 +01:00
Jelte Fennema 093be287e2 Improve comments 2020-01-06 12:42:43 +01:00
Jelte Fennema 53020a7439 Separate function delagation from non distributed statements planning 2020-01-06 12:42:43 +01:00
Jelte Fennema 6251363bfb Rename some things based on reviews 2020-01-06 12:42:43 +01:00
Jelte Fennema 5f719b6358 Remove now unnecessary comment 2020-01-06 12:42:43 +01:00
Jelte Fennema aefda3edf2 Move functions around 2020-01-06 12:42:43 +01:00
Jelte Fennema d2f31a2dc2 Pack shared arguments in a struct 2020-01-06 12:42:43 +01:00
Jelte Fennema b4381df8be Refactor distributed_planner for better understandability 2020-01-06 12:42:43 +01:00
Onder Kalaci 5a1e752726 Apply feedback - add fastPath field to plan 2020-01-06 12:42:43 +01:00
Onder Kalaci 13a9b55695 Skip expensive checks when fast-path query 
The definition of fast-path query is very strict. So, we don't need
to do some extra checks.
 2020-01-06 12:42:43 +01:00
Onder Kalaci 7f3ab7892d Skip shard pruning when possible 
We're already traversing the queryTree and finding the distribution
key value, so pass it to the later stages of the planning.
 2020-01-06 12:42:43 +01:00
Onder Kalaci ca293116fa Reduce calls to FastPathRouterQuery() 
Before this commit, we called it twice durning planning. Instead,
we save the information and pass it.
 2020-01-06 12:42:43 +01:00
Önder Kalacı 270571c106
Merge pull request #3333 from citusdata/fix_wrong_data 
Make sure to update shard states of partitions on failures
 2020-01-06 11:37:40 +00:00
Onder Kalaci c8f14c9f6c Make sure to update shard states of partitions on failures 
Fixes #3331

In #2389, we've implemented support for partitioned tables with rep > 1.
The implementation is limiting the use of modification queries on the
partitions. In fact, we error out when any partition is modified via
EnsurePartitionTableNotReplicated().

However, we seem to forgot an important case, where the parent table's
partition is marked as INVALID. In that case, at least one of the partition
becomes INVALID. However, we do not mark partitions as INVALID ever.

If the user queries the partition table directly, Citus could happily send
the query to INVALID placements -- which are not marked as INVALID.

This PR fixes it by marking the placements of the partitions as INVALID
as well.

The shard placement repair logic already re-creates all the partitions,
so should be fine in that front.
 2020-01-06 12:26:08 +01:00
Jelte Fennema 3c770516eb
Commenting out flaky intermediate data leak test (#3359) 
check-multi apparently has an intermediate data leak, so commenting out
that test for now. This was introduced by #3349 

Examples:
- https://app.circleci.com/jobs/github/citusdata/citus/74675
- https://app.circleci.com/jobs/github/citusdata/citus/74683
- https://app.circleci.com/jobs/github/citusdata/citus/74763
 2020-01-06 11:55:01 +01:00
Jelte Fennema d29ce8965c
Actually check that test output normalization is applied in CI (#3358) 
Fixup of an issue with #3336 that caused CI not to check correctly that
normalized test output was committed.
 2020-01-06 10:37:34 +01:00
Jelte Fennema de75243000
Commit normalized test output for better diffs (#3336) 
We have a `normalize.sed` script that before diffing test output normalizes the
expected file and the actual file. This makes sure that we don't have random
test failures and that we have to update our test output all the time. This PR
takes that one step further and actually commits the normalized files. That way
whenever we DO have to update our test output files only relevant changes will
be visible in the diff.

The other change that this PR does it that it strips trailing whitespace during 
normalization. This works well with our editorconfig settings.

As an added benefit of committing these files it's also much more visible what
new normalization rules will result in. The original changes that were proposed
here were a bit to wide and were changing output that was not intentended to
be changed: https://github.com/citusdata/citus/pull/3161#discussion_r360928922
Because these changes are now in the diff of the commit they are much easier to
spot.

Finally the Plan number normalization rules were also added to this PR, because
they are useful even without the CTE inlining PR.
 2020-01-06 09:56:31 +01:00
Jelte Fennema 4a20ba3bfc Merge remote-tracking branch 'origin/master' into normalized-test-output 2020-01-06 09:36:04 +01:00
Jelte Fennema 2e4e1c030f Make sure the expected .out file always exists when running diff on it 2020-01-06 09:32:03 +01:00
Jelte Fennema 16bcf15e16 Remove unused normalization rule 2020-01-06 09:32:03 +01:00
Jelte Fennema 634ea80009 Add a basic testing README including normalization explanation 2020-01-06 09:32:03 +01:00
Jelte Fennema 7c3e8e150e Normalize tests: s/Subplan [0-9]+\_/Subplan XXX\_/g 2020-01-06 09:32:03 +01:00
Jelte Fennema acd12a6de5 Normalize tests: s/read_intermediate_result\('[0-9]+_/read_intermediate_result('XXX_/g 2020-01-06 09:32:03 +01:00
Jelte Fennema 21dbd4e55d Normalize tests: s/generating subplan [0-9]+\_/generating subplan XXX\_/g 2020-01-06 09:32:03 +01:00
Jelte Fennema 58723dd8b0 Normalize tests: s/DEBUG: Plan [0-9]+/DEBUG: Plan XXX/g 2020-01-06 09:32:03 +01:00
Jelte Fennema 34c5532e9c Add commented out rules to normalize Plan numbers 2020-01-06 09:32:03 +01:00
Jelte Fennema 38ac28b4b8 Normalize tests: intermediate_results 2020-01-06 09:32:03 +01:00
Jelte Fennema 0c6983a80e Normalize tests: pg12 changes 2020-01-06 09:32:03 +01:00
Jelte Fennema 7730bd449c Normalize tests: Remove trailing whitespace 2020-01-06 09:32:03 +01:00
Jelte Fennema 6353c9907f Normalize tests: Line info varies between versions 2020-01-06 09:32:03 +01:00
Jelte Fennema bf2c203908 Normalize tests: solation_ref2ref_foreign_keys 2020-01-06 09:32:03 +01:00
Jelte Fennema 7b2c769a5d Normalize tests: normalize file names for partitioned files 2020-01-06 09:32:03 +01:00
Jelte Fennema 98bab9caab Normalize tests: ignore WAL warnings 2020-01-06 09:32:03 +01:00
Jelte Fennema 5c0f955ab9 Normalize tests: ignore could not consume warnings 2020-01-06 09:32:03 +01:00
Jelte Fennema dc3cff991f Normalize tests: normalize failed task ids 2020-01-06 09:32:03 +01:00
Jelte Fennema d0ade90cd0 Normalize tests: pkey constraints for multi_insert_select 2020-01-06 09:32:03 +01:00
Jelte Fennema 704e1d2bc8 Normalize tests: shard table names for multi_name_lengths 2020-01-06 09:32:03 +01:00
Jelte Fennema 1c4ea6836b Normalize tests: shard table names for multi_insert_select_conflict 2020-01-06 09:32:03 +01:00
Jelte Fennema 27997c054e Normalize tests: shard table names for foreign_key_restrection_enforcement 2020-01-06 09:32:03 +01:00
Jelte Fennema 432b5baac7 Normalize tests: shard table names for custom_aggregate_support 2020-01-06 09:32:03 +01:00
Jelte Fennema 0c23caeb75 Normalize tests: shard table names for multi_subtransactions 2020-01-06 09:32:03 +01:00
Jelte Fennema 883ee9121f Normalize tests: shard table names in foreign_key_to_reference_table 2020-01-06 09:32:03 +01:00
Jelte Fennema 7f3de68b0d Normalize tests: header separator length 2020-01-06 09:32:03 +01:00
Philip Dubé 51a7e661f9
Merge pull request #3349 from citusdata/ensure-no-intermediate-data-leak-at-end 
End regression tests with ensure_no_intermediate_data_leak
 2020-01-03 19:20:28 +00:00
Philip Dubé 566246ecd4 End regression tests with ensure_no_intermediate_data_leak 
Also update tests to clean up jobs when they're directly testing job udfs
 2020-01-03 18:59:02 +00:00
Önder Kalacı 0c70a5470e
Allow RETURNING in fast-path queries (#3352) 
* Allow RETURNING in fast-path queries

Because there is no specific reason for that.
 2020-01-03 13:42:50 +00:00
Önder Kalacı a174eb4f7b
Do not go through standard_planner() for INSERTs (#3348) 
That seems unnecessary. We already have the notion of FastPath queries,
simply add it there.
 2020-01-03 12:15:22 +00:00