external
/
citus
mirror of https://github.com/citusdata/citus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,112 Commits
644 Branches
162 Tags
101 MiB
Commit Graph

1931 Commits (f1a058297354c4210a28d24384a4cbb254232dc8)

Author SHA1 Message Date
Marco Slot f1a0582973 Make ApplyLogRedaction a macro and redefine ereport 2020-01-13 18:24:36 +01:00
Marco Slot 06709ee108 Always use NOTICE in log_remote_commands and avoid redaction when possible 2020-01-13 18:24:36 +01:00
Marco Slot 90056f7d3c Remove copy from worker for append-partitioned table 2020-01-13 23:03:40 -08:00
Philip Dubé 62524d152d mitmscripts/fluent.py: use atomic increment 2020-01-13 20:35:08 +00:00
Philip Dubé ccabf19090 Propagate DROP ROUTINE, ALTER ROUTINE 
In two places I've made code more straight forward by using ROUTINE in our own codegen

Two changes which may seem extraneous:

AppendFunctionName was updated to not use pg_get_function_identity_arguments.
This is because that function includes ORDER BY when printing an aggregate like my_rank.
While ALTER AGGREGATE my_rank(x "any" ORDER BY y "any") is accepted by postgres,
ALTER ROUTINE my_rank(x "any" ORDER BY y "any") is not.

Tests were updated to use macaddr over integer. Using integer is flaky, our logic
could sometimes end up on tables like users_table. I originally wanted to use money,
but money isn't hashable.
 2020-01-13 15:37:46 +00:00
Philip Dubé 4b5d6c3ebe Rename RelayFileState to ShardState 
Replace FILE_ prefix with SHARD_STATE_
 2020-01-12 05:57:53 +00:00
Philip Dubé e71386af33 Replace ARRAY_OUT_FUNC_ID with postgres's F_ARRAY_OUT 
Also use stack allocation for walkerContext in multi_logical_optimizer
 2020-01-10 16:54:00 +00:00
Hadi Moshayedi 40ba2cdd6e Test RedistributeTaskListResult 2020-01-09 23:47:25 -08:00
Hadi Moshayedi 527d7d41c1 Implement RedistributeTaskListResult 2020-01-09 23:47:25 -08:00
Philip Dubé 281aacce9b Fix row-gather for subqueries being handled by task-tracker 
task-tracker has specific logic for MultiPartition when GROUP BY is missing

We were ending up in this code path because row-gather removes GROUP BY
 2020-01-10 01:51:37 +00:00
Hadi Moshayedi e1e383cb59 Don't override xact id assigned by coordinator on workers. 
We might need to send commands from workers to other workers. In
these cases we shouldn't override the xact id assigned by coordinator,
or otherwise we won't read the consistent set of result files
accross the nodes.
 2020-01-09 11:09:11 -08:00
Hadi Moshayedi bb65669186 Failure tests for PartitionTasklistResults 2020-01-09 10:55:58 -08:00
Hadi Moshayedi c7c460e843 PartitionTasklistResults: Use different queries per placement 
We need to know which placement succeeded in executing the worker_partition_query_result() call. Otherwise we wouldn't know which node to fetch from. This change allows that by introducing Task::perPlacementQueryStrings.
 2020-01-09 10:55:58 -08:00
Hadi Moshayedi f38d0e5b3f Partitioned task list results. 2020-01-09 10:32:58 -08:00
Philip Dubé 73c06fae3b Introduce GetDistributeObjectOps to organize dispatch of logic dependent on node/object type 2020-01-09 18:24:29 +00:00
Jelte Fennema 9724e25065 Normalize plan numbers in insert_select output 2020-01-07 10:34:08 +01:00
Philip Dubé bf7d86a3e8 Fix typo: aggragate -> aggregate 2020-01-07 01:16:09 +00:00
Philip Dubé 863bf49507 Implement pulling up rows to coordinator when aggregates cannot be pushed down. Enabled by default 2020-01-07 01:16:04 +00:00
Jelte Fennema 5b0baea72c Refactor distributed_planner for better understandability 2020-01-06 14:23:38 +01:00
Onder Kalaci 5a1e752726 Apply feedback - add fastPath field to plan 2020-01-06 12:42:43 +01:00
Onder Kalaci 13a9b55695 Skip expensive checks when fast-path query 
The definition of fast-path query is very strict. So, we don't need
to do some extra checks.
 2020-01-06 12:42:43 +01:00
Onder Kalaci 7f3ab7892d Skip shard pruning when possible 
We're already traversing the queryTree and finding the distribution
key value, so pass it to the later stages of the planning.
 2020-01-06 12:42:43 +01:00
Onder Kalaci ca293116fa Reduce calls to FastPathRouterQuery() 
Before this commit, we called it twice durning planning. Instead,
we save the information and pass it.
 2020-01-06 12:42:43 +01:00
Onder Kalaci c8f14c9f6c Make sure to update shard states of partitions on failures 
Fixes #3331

In #2389, we've implemented support for partitioned tables with rep > 1.
The implementation is limiting the use of modification queries on the
partitions. In fact, we error out when any partition is modified via
EnsurePartitionTableNotReplicated().

However, we seem to forgot an important case, where the parent table's
partition is marked as INVALID. In that case, at least one of the partition
becomes INVALID. However, we do not mark partitions as INVALID ever.

If the user queries the partition table directly, Citus could happily send
the query to INVALID placements -- which are not marked as INVALID.

This PR fixes it by marking the placements of the partitions as INVALID
as well.

The shard placement repair logic already re-creates all the partitions,
so should be fine in that front.
 2020-01-06 12:26:08 +01:00
Jelte Fennema 3c770516eb
Commenting out flaky intermediate data leak test (#3359) 
check-multi apparently has an intermediate data leak, so commenting out
that test for now. This was introduced by #3349 

Examples:
- https://app.circleci.com/jobs/github/citusdata/citus/74675
- https://app.circleci.com/jobs/github/citusdata/citus/74683
- https://app.circleci.com/jobs/github/citusdata/citus/74763
 2020-01-06 11:55:01 +01:00
Jelte Fennema d29ce8965c
Actually check that test output normalization is applied in CI (#3358) 
Fixup of an issue with #3336 that caused CI not to check correctly that
normalized test output was committed.
 2020-01-06 10:37:34 +01:00
Jelte Fennema 4a20ba3bfc Merge remote-tracking branch 'origin/master' into normalized-test-output 2020-01-06 09:36:04 +01:00
Jelte Fennema 2e4e1c030f Make sure the expected .out file always exists when running diff on it 2020-01-06 09:32:03 +01:00
Jelte Fennema 16bcf15e16 Remove unused normalization rule 2020-01-06 09:32:03 +01:00
Jelte Fennema 634ea80009 Add a basic testing README including normalization explanation 2020-01-06 09:32:03 +01:00
Jelte Fennema 7c3e8e150e Normalize tests: s/Subplan [0-9]+\_/Subplan XXX\_/g 2020-01-06 09:32:03 +01:00
Jelte Fennema acd12a6de5 Normalize tests: s/read_intermediate_result\('[0-9]+_/read_intermediate_result('XXX_/g 2020-01-06 09:32:03 +01:00
Jelte Fennema 21dbd4e55d Normalize tests: s/generating subplan [0-9]+\_/generating subplan XXX\_/g 2020-01-06 09:32:03 +01:00
Jelte Fennema 58723dd8b0 Normalize tests: s/DEBUG: Plan [0-9]+/DEBUG: Plan XXX/g 2020-01-06 09:32:03 +01:00
Jelte Fennema 34c5532e9c Add commented out rules to normalize Plan numbers 2020-01-06 09:32:03 +01:00
Jelte Fennema 38ac28b4b8 Normalize tests: intermediate_results 2020-01-06 09:32:03 +01:00
Jelte Fennema 0c6983a80e Normalize tests: pg12 changes 2020-01-06 09:32:03 +01:00
Jelte Fennema 7730bd449c Normalize tests: Remove trailing whitespace 2020-01-06 09:32:03 +01:00
Jelte Fennema 6353c9907f Normalize tests: Line info varies between versions 2020-01-06 09:32:03 +01:00
Jelte Fennema bf2c203908 Normalize tests: solation_ref2ref_foreign_keys 2020-01-06 09:32:03 +01:00
Jelte Fennema 7b2c769a5d Normalize tests: normalize file names for partitioned files 2020-01-06 09:32:03 +01:00
Jelte Fennema 98bab9caab Normalize tests: ignore WAL warnings 2020-01-06 09:32:03 +01:00
Jelte Fennema 5c0f955ab9 Normalize tests: ignore could not consume warnings 2020-01-06 09:32:03 +01:00
Jelte Fennema dc3cff991f Normalize tests: normalize failed task ids 2020-01-06 09:32:03 +01:00
Jelte Fennema d0ade90cd0 Normalize tests: pkey constraints for multi_insert_select 2020-01-06 09:32:03 +01:00
Jelte Fennema 704e1d2bc8 Normalize tests: shard table names for multi_name_lengths 2020-01-06 09:32:03 +01:00
Jelte Fennema 1c4ea6836b Normalize tests: shard table names for multi_insert_select_conflict 2020-01-06 09:32:03 +01:00
Jelte Fennema 27997c054e Normalize tests: shard table names for foreign_key_restrection_enforcement 2020-01-06 09:32:03 +01:00
Jelte Fennema 432b5baac7 Normalize tests: shard table names for custom_aggregate_support 2020-01-06 09:32:03 +01:00
Jelte Fennema 0c23caeb75 Normalize tests: shard table names for multi_subtransactions 2020-01-06 09:32:03 +01:00