- Break CheckShardPlacements into multiple functions (The most important
is MarkFailedShardPlacements), so that we can get rid of the global
CoordinatedTransactionUses2PC.
- Call MarkFailedShardPlacements in the router executor, so we mark
shards as invalid and stop using them while inside transaction blocks.
Before this change, when a transaction failed, we update related placements shard states
to FILE_INACTIVE during XACT_EVENT_PRE_COMMIT. However that means if another code block
changed shard state to something else (e.g. FILE_TO_DELETE) before XACT_EVENT_PRE_COMMIT
we overwrite that. To prevent that problem, in case of failure we started to change
shard state, only if its current shard state is FILE_FINALIZED.
Because of foreign keys and similar concerns there should only be a
single modifying/DDL connection for a set of colocated placements to a
node. To enforce placement_connection.c now has an additional
hash-table keeping track of the connections to a set of colocated
placements. In addition to enforcing per placement restrictions on
connections, there's now very similar restrictions for sets of
colocated placements.
Currently there are several places in citus that map placements to
connections and that manage placement health. Centralize this
knowledge. Because of the centralized knowledge about which
connection has previously been used for which shard/placement, this
also provides the basis for relaxing restrictions around combining
various forms of DDL/DML.
Connections for a placement can now be acquired using
GetPlacementConnection(). If the connection is used for DML or DDL the
FOR_DDL/DML flags should be used respectively. If an individual
remote transaction fails (but the transaction on the master succeeds)
and FOR_DDL/DML have been specified, the placement is marked as
invalid, unless that'd mark all placements for a shard as invalid.
Brian Cloutier
Burak Yucesoy
Andres Freund
Onder Kalaci