With #4338, the executor is smart enough to failover to
local node if there is not enough space in max_connections
for remote connections.
For COPY, the logic is different. With #4034, we made COPY
work with the adaptive connection management slightly
differently. The cause of the difference is that COPY doesn't
know which placements are going to be accessed hence requires
to get connections up-front.
Similarly, COPY decides to use local execution up-front.
With this commit, we change the logic for COPY on local nodes:
Try to reserve a connection to local host. This logic follows
the same logic (e.g., citus.local_shared_pool_size) as the
executor because COPY also relies on TryToIncrementSharedConnectionCounter().
If reservation to local node fails, switch to local execution
Apart from this, if local execution is disabled, we follow the
exact same logic for multi-node Citus. It means that if we are
out of the connection, we'd give an error.
It seems that we were not considering the case where coordinator was
added to the cluster as a worker in the optimization of intermediate
results.
This could lead to errors when coordinator was added as a worker.
pg_get_tableschemadef_string doesn't know how to deparse identity
columns so we cannot reflect those columns when creating table
from scratch. For this reason, we don't allow using alter_table udfs
with tables having any identity cols.
pg_get_tableschemadef_string doesn't know how to deparse identity
columns so we cannot reflect those columns when creating shell
relation.
For this reason, we don't allow adding local tables -having identity cols-
to metadata.
Postgres doesn't allow inserting into columns having GENERATED ALWAYS
AS (...) STORED expressions.
For this reason, when executing undistribute_table or an alter_* udf,
we should skip copying such columns.
This is not bad since Postgres would already generate such columns.
Enables an overall plan to be parallel (e.g. over a partition
hierarchy), even though an individual ColumnarScan is not
parallel-aware.
Co-authored-by: Jeff Davis <jefdavi@microsoft.com>
Previously, if columnar.enable_custom_scan was false, parallel paths
could remain, leading to an unexpected error.
Also, ensure that cheapest_parameterized_paths is cleared if a custom
scan is used.
Co-authored-by: Jeff Davis <jefdavi@microsoft.com>
When adding local table to metadata, we are dropping DEFAULT expressions
from shard relation. When finding columns having DEFAULT expressions,
we shouldn't rely on atthasdef since it might be true if column has
GENERATED ALWAYS AS (...) STORED expression.
On the other hand, we should not actually drop such GENERATED expressions from
shard relation since we don't evaluate such columns in coordinator and this would
result in inserting NULL values to such columns.
When finding columns owning sequences, we shouldn't rely on atthasdef
since it might be true when column has GENERATED ALWAYS AS (...)
STORED expression.
Since create_citus_local_table doesn't specify cascadeViaForeignKeys
option, we can't directly call citus_add_local_table_to_metadata
from create_citus_local_table.
Instead, implement an internal method and call it from deprecated udf
too.
fix_partition_constraints() goes over all partitioned distributed tables and renames
constraint names to their original values.
fix_partition_constraints(partitioned_dist_table) goes over all shard placements of a
partitioned distributed table and sends worker_fix_partition_constraints(...) to workers
in a distributed transaction.
worker_fix_partition_constraints(partitioned_dist_table, shardId, constraintName) checks
if a shardId is appended to a constraint, and removes that suffix with an
ALTER TABLE .. RENAME CONSTRAINT command.
Hadi Moshayedi
Hadi Moshayedi
Jeff Davis
Halil Ozan Akgül
Ahmet Gedemenli
Önder Kalacı
Onder Kalaci
Sait Talha Nisanci
Onur Tirtir
jeff-davis
Hanefi Onaldi
Brian Bergeron