Enables an overall plan to be parallel (e.g. over a partition
hierarchy), even though an individual ColumnarScan is not
parallel-aware.
Co-authored-by: Jeff Davis <jefdavi@microsoft.com>
Previously, if columnar.enable_custom_scan was false, parallel paths
could remain, leading to an unexpected error.
Also, ensure that cheapest_parameterized_paths is cleared if a custom
scan is used.
Co-authored-by: Jeff Davis <jefdavi@microsoft.com>
When adding local table to metadata, we are dropping DEFAULT expressions
from shard relation. When finding columns having DEFAULT expressions,
we shouldn't rely on atthasdef since it might be true if column has
GENERATED ALWAYS AS (...) STORED expression.
On the other hand, we should not actually drop such GENERATED expressions from
shard relation since we don't evaluate such columns in coordinator and this would
result in inserting NULL values to such columns.
When finding columns owning sequences, we shouldn't rely on atthasdef
since it might be true when column has GENERATED ALWAYS AS (...)
STORED expression.
Since create_citus_local_table doesn't specify cascadeViaForeignKeys
option, we can't directly call citus_add_local_table_to_metadata
from create_citus_local_table.
Instead, implement an internal method and call it from deprecated udf
too.
fix_partition_constraints() goes over all partitioned distributed tables and renames
constraint names to their original values.
fix_partition_constraints(partitioned_dist_table) goes over all shard placements of a
partitioned distributed table and sends worker_fix_partition_constraints(...) to workers
in a distributed transaction.
worker_fix_partition_constraints(partitioned_dist_table, shardId, constraintName) checks
if a shardId is appended to a constraint, and removes that suffix with an
ALTER TABLE .. RENAME CONSTRAINT command.
* Fix partition column index issue
We send column names to worker_hash/range_partition_table methods, and
in these methods we check the column name index from tuple descriptor.
Then this index is used to decide the bucket that the current row will
be sent for the repartition.
This becomes a problem when there are the same column names in the
tupleDescriptor. Then we can choose the wrong index. Hence the
partitioned data will be put to wrong workers. Then the result could
miss some data because workers might contain different range of data.
An example:
TupleDescriptor contains "trip_id", "car_id", "car_id" for one table.
It contains only "car_id" for the other table. And assuming that the
tables will be partitioned by car_id, it is not certain what should be
used for deciding the bucket number for the first table. Assuming value
2 goes to bucket 2 and value 3 goes to bucket 3, it is not certain which
bucket "1 2 3" (trip_id, car_id, car_id) row will go to.
As a solution we send the index of partition column in targetList
instead of the column name.
The old API is kept so that if workers upgrade work, it still works
(though it will have the same bug)
* Use the same method so that backporting is easier
Fixing a division by zero in the cost calculations for scanning a columnar table.
Due to how the columns in a columnar table are counted an empty table would result in a division by zero. Instead this patch keeps the column selection ratio on zero when this happens, resulting in an accurate cost of zero pages to scan a columnar table.
fixes#4589
In pg11.8 it seemingly tries to parse the full sql file creating the extension,
since we use syntax introduced in postgres 12 this fails.
This patch rewrites the statement not recognized by pg11.8 to be dynamically
executed from a string literal via `EXECUTE`.
* Make undistribute_table() and citus_create_local_table() work with columnar
* Rename and use LocallyExecuteUtilityTask for UDF check
* Remove 'local' references in ExecuteUtilityCommand
jeff-davis
Onur Tirtir
Hanefi Onaldi
Hadi Moshayedi
Jeff Davis
Hadi Moshayedi
Brian Bergeron
SaitTalhaNisanci
Önder Kalacı
Onder Kalaci
Halil Ozan Akgül
Nils Dijk
Ahmet Gedemenli
Naisila Puka