mirror of https://github.com/citusdata/citus.git
1c5b8588fe
Sometimes in CI our isolation_citus_dist_activity test fails randomly
like this:
```diff
step s2-view-dist:
SELECT query, citus_nodename_for_nodeid(citus_nodeid_for_gpid(global_pid)), citus_nodeport_for_nodeid(citus_nodeid_for_gpid(global_pid)), state, wait_event_type, wait_event, usename, datname FROM citus_dist_stat_activity WHERE query NOT ILIKE ALL(VALUES('%pg_prepared_xacts%'), ('%COMMIT%'), ('%BEGIN%'), ('%pg_catalog.pg_isolation_test_session_is_blocked%'), ('%citus_add_node%')) AND backend_type = 'client backend' ORDER BY query DESC;
query |citus_nodename_for_nodeid|citus_nodeport_for_nodeid|state |wait_event_type|wait_event|usename |datname
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------+-------------------------+-------------------+---------------+----------+--------+----------
INSERT INTO test_table VALUES (100, 100);
|localhost | 57636|idle in transaction|Client |ClientRead|postgres|regression
-(1 row)
+
+ SELECT coalesce(to_jsonb(array_agg(csa_from_one_node.*)), '[{}]'::JSONB)
+ FROM (
+ SELECT global_pid, worker_query AS is_worker_query, pg_stat_activity.* FROM
+ pg_stat_activity LEFT JOIN get_all_active_transactions() ON process_id = pid
+ ) AS csa_from_one_node;
+ |localhost | 57636|active | | |postgres|regression
+(2 rows)
step s3-view-worker:
```
Source: https://app.circleci.com/pipelines/github/citusdata/citus/26692/workflows/3406e4b4-b686-4667-bec6-8253ee0809b1/jobs/765119
I intended to fix this with #6263, but the fix turned out to be
insufficient. This PR tries to address the issue by setting
distributedCommandOriginator correctly in more situations. However, even
with this change it's still possible to reproduce the flaky test in CI.
In any case this should fix at least some instances of this issue.
In passing this changes the isolation_citus_dist_activity test to allow
running it multiple times in a row.
|
||
|---|---|---|
| .. | ||
| README.md | ||
| columnar_index_concurrency.spec | ||
| columnar_temp_tables.spec | ||
| columnar_vacuum_vs_insert.spec | ||
| columnar_write_concurrency.spec | ||
| columnar_write_concurrency_index.spec | ||
| isolation_acquire_distributed_locks.spec | ||
| isolation_add_coordinator.spec | ||
| isolation_add_node_vs_reference_table_operations.spec | ||
| isolation_add_remove_node.spec | ||
| isolation_blocking_move_multi_shard_commands.spec | ||
| isolation_blocking_move_multi_shard_commands_on_mx.spec | ||
| isolation_blocking_move_single_shard_commands.spec | ||
| isolation_blocking_move_single_shard_commands_on_mx.spec | ||
| isolation_blocking_shard_split.spec | ||
| isolation_blocking_shard_split_with_fkey_to_reference.spec | ||
| isolation_cancellation.spec | ||
| isolation_check_mx.spec | ||
| isolation_citus_dist_activity.spec | ||
| isolation_citus_locks.spec | ||
| isolation_cluster_management.spec | ||
| isolation_concurrent_dml.spec | ||
| isolation_concurrent_move_create_table.spec | ||
| isolation_copy_placement_vs_copy_placement.spec | ||
| isolation_copy_placement_vs_modification.spec | ||
| isolation_copy_vs_all_on_mx.spec | ||
| isolation_create_citus_local_table.spec | ||
| isolation_create_distributed_table.spec | ||
| isolation_create_distributed_table_concurrently.spec | ||
| isolation_create_restore_point.spec | ||
| isolation_create_table_vs_add_remove_node.spec | ||
| isolation_data_migration.spec | ||
| isolation_ddl_vs_all.spec | ||
| isolation_delete_vs_all.spec | ||
| isolation_dis2ref_foreign_keys_on_mx.spec | ||
| isolation_distributed_deadlock_detection.spec | ||
| isolation_distributed_transaction_id.spec | ||
| isolation_dml_vs_repair.spec | ||
| isolation_drop_alter_index_select_for_update_on_mx.spec | ||
| isolation_drop_shards.spec | ||
| isolation_drop_vs_all.spec | ||
| isolation_dump_global_wait_edges.spec | ||
| isolation_dump_local_wait_edges.spec | ||
| isolation_ensure_dependency_activate_node.spec | ||
| isolation_extension_commands.spec | ||
| isolation_fix_partition_shard_index_names.spec | ||
| isolation_get_all_active_transactions.spec | ||
| isolation_get_distributed_wait_queries_mx.spec | ||
| isolation_global_pid.spec | ||
| isolation_hash_copy_vs_all.spec | ||
| isolation_insert_select_conflict.spec | ||
| isolation_insert_select_repartition.spec | ||
| isolation_insert_select_vs_all.spec | ||
| isolation_insert_select_vs_all_on_mx.spec | ||
| isolation_insert_vs_all.spec | ||
| isolation_insert_vs_all_on_mx.spec | ||
| isolation_insert_vs_vacuum.spec | ||
| isolation_logical_replication_binaryless.spec | ||
| isolation_logical_replication_multi_shard_commands.spec | ||
| isolation_logical_replication_multi_shard_commands_on_mx.spec | ||
| isolation_logical_replication_single_shard_commands.spec | ||
| isolation_logical_replication_single_shard_commands_on_mx.spec | ||
| isolation_logical_replication_with_partitioning.spec | ||
| isolation_master_update_node.spec | ||
| isolation_max_client_connections.spec | ||
| isolation_metadata_sync_deadlock.spec | ||
| isolation_metadata_sync_vs_all.spec | ||
| isolation_modify_with_subquery_vs_dml.spec | ||
| isolation_move_placement_vs_modification.spec | ||
| isolation_move_placement_vs_modification_fk.spec | ||
| isolation_move_placement_vs_move_placement.spec | ||
| isolation_multi_shard_modify_vs_all.spec | ||
| isolation_multiuser_locking.spec | ||
| isolation_mx_common.include.spec | ||
| isolation_non_blocking_shard_split.spec | ||
| isolation_non_blocking_shard_split_fkey.spec | ||
| isolation_non_blocking_shard_split_with_index_as_replicaIdentity.spec | ||
| isolation_partitioned_copy_vs_all.spec | ||
| isolation_pg_send_cancellation.spec | ||
| isolation_progress_monitoring.spec | ||
| isolation_range_copy_vs_all.spec | ||
| isolation_rebalancer_deferred_drop.spec | ||
| isolation_ref2ref_foreign_keys.spec | ||
| isolation_ref2ref_foreign_keys_enterprise.spec | ||
| isolation_ref2ref_foreign_keys_on_mx.spec | ||
| isolation_ref_select_for_update_vs_all_on_mx.spec | ||
| isolation_ref_update_delete_upsert_vs_all_on_mx.spec | ||
| isolation_reference_copy_vs_all.spec | ||
| isolation_reference_on_mx.spec | ||
| isolation_reference_table.spec | ||
| isolation_remove_coordinator.spec | ||
| isolation_replace_wait_function.spec | ||
| isolation_replicate_reference_tables_to_coordinator.spec | ||
| isolation_replicated_dist_on_mx.spec | ||
| isolation_select_for_update.spec | ||
| isolation_select_vs_all.spec | ||
| isolation_select_vs_all_on_mx.spec | ||
| isolation_setup.spec | ||
| isolation_shard_move_vs_start_metadata_sync.spec | ||
| isolation_shard_rebalancer.spec | ||
| isolation_shard_rebalancer_progress.spec | ||
| isolation_shouldhaveshards.spec | ||
| isolation_tenant_isolation.spec | ||
| isolation_tenant_isolation_nonblocking.spec | ||
| isolation_tenant_isolation_with_fkey_to_reference.spec | ||
| isolation_transaction_recovery.spec | ||
| isolation_truncate_vs_all.spec | ||
| isolation_truncate_vs_all_on_mx.spec | ||
| isolation_undistribute_table.spec | ||
| isolation_update_delete_upsert_vs_all_on_mx.spec | ||
| isolation_update_node.spec | ||
| isolation_update_node_lock_writes.spec | ||
| isolation_update_vs_all.spec | ||
| isolation_upsert_vs_all.spec | ||
| isolation_vacuum_skip_locked.spec | ||
| isolation_validate_vs_insert.spec | ||
| shared_connection_waits.spec | ||
README.md
In this folder, all tests which in the format of '*_add.spec' organized according to specific format.
You should use // in mx files not //. We preprocess mx files with cpp to
include isolation_mx_common.include.spec.
For isolation tests, we selected 'n' representative operations and we aimed to perform all possible pairs of 'n' operations together. So first test just runs first of these 'n' operation with remaining 'n - 1' operation. Similary, second test just runs second operation with remaining 'n - 2' operation. With this logic, we eventually run every selected operation with every other selected operation.