We might need to send commands from workers to other workers. In
these cases we shouldn't override the xact id assigned by coordinator,
or otherwise we won't read the consistent set of result files
accross the nodes.
We need to know which placement succeeded in executing the worker_partition_query_result() call. Otherwise we wouldn't know which node to fetch from. This change allows that by introducing Task::perPlacementQueryStrings.
Partitioned task list results.
Implements PartitionTasklistResults(), which partitions results of given SELECT tasks based on shard ranges of a given relation.
Fast-path queries are introduced with #2606. The basic idea is that for very simple queries like SELECT count(*) FROM table WHERE dist_key = X, we can skip some parts of the distributed planning. The most notable thing to skip is standard_planner(), which was already done in #2606.
With this commit, we do some further optimizations. First, we used to call the function which decides whether the query is fast path twice, which can be reduced to one. Second, we used to do shard pruning for every query, now we'll optimize it for some cases. Finally, since the definition of fast-path queries are very strict, we can skip some query traversals.
Fixes#3331
In #2389, we've implemented support for partitioned tables with rep > 1.
The implementation is limiting the use of modification queries on the
partitions. In fact, we error out when any partition is modified via
EnsurePartitionTableNotReplicated().
However, we seem to forgot an important case, where the parent table's
partition is marked as INVALID. In that case, at least one of the partition
becomes INVALID. However, we do not mark partitions as INVALID ever.
If the user queries the partition table directly, Citus could happily send
the query to INVALID placements -- which are not marked as INVALID.
This PR fixes it by marking the placements of the partitions as INVALID
as well.
The shard placement repair logic already re-creates all the partitions,
so should be fine in that front.
We have a `normalize.sed` script that before diffing test output normalizes the
expected file and the actual file. This makes sure that we don't have random
test failures and that we have to update our test output all the time. This PR
takes that one step further and actually commits the normalized files. That way
whenever we DO have to update our test output files only relevant changes will
be visible in the diff.
The other change that this PR does it that it strips trailing whitespace during
normalization. This works well with our editorconfig settings.
As an added benefit of committing these files it's also much more visible what
new normalization rules will result in. The original changes that were proposed
here were a bit to wide and were changing output that was not intentended to
be changed: https://github.com/citusdata/citus/pull/3161#discussion_r360928922
Because these changes are now in the diff of the commit they are much easier to
spot.
Finally the Plan number normalization rules were also added to this PR, because
they are useful even without the CTE inlining PR.
Philip Dubé
Philip Dubé
Hadi Moshayedi
Hadi Moshayedi
Önder Kalacı
Jelte Fennema