Clean up leftover replication slots in tests (#7338)

This commit fixes the flakiness in `logical_replication` and
`citus_non_blocking_split_shard_cleanup` tests. The flakiness
was related to leftover replication slots.
Below is a flaky example for each test:

logical_replication https://github.com/citusdata/citus/actions/runs/6721324131/attempts/1#summary-18267030604
citus_non_blocking_split_shard_cleanup https://github.com/citusdata/citus/actions/runs/6721324131/attempts/1#summary-18267006967

```diff
 -- Replication slots should be cleaned up
 SELECT slot_name FROM pg_replication_slots;
             slot_name            
 ---------------------------------
-(0 rows)
+ citus_shard_split_slot_19_10_17
+(1 row)
```

The tests by themselves are not flaky: 32 flaky test
schedules each with 20 runs run successfully.
https://github.com/citusdata/citus/actions/runs/6822020127?pr=7338

The conclusion is that:
1. `multi_tenant_isolation_nonblocking` is the problematic test running
before `logical_replication` in the `enterprise_schedule`, so I added a
cleanup at the end of `multi_tenant_isolation_nonblocking`.
https://github.com/citusdata/citus/actions/runs/6824334614/attempts/1#summary-18560127461
2. `citus_split_shard_by_split_points_negative` is the problematic test
running before `citus_non_blocking_split_shards_cleanup` in the split
schedule. Also added cleanup line.

For details on the investigation of leftover replication slots,
please check the PR https://github.com/citusdata/citus/pull/7338
pull/7346/head
Naisila Puka 2023-11-14 18:50:54 +03:00 committed by GitHub
parent cdef2d5224
commit a960799dfb
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 17 additions and 0 deletions

View File

@ -135,4 +135,10 @@ NOTICE: drop cascades to 3 other objects
DETAIL: drop cascades to table citus_split_shard_by_split_points_negative.range_paritioned_table_to_split
drop cascades to table citus_split_shard_by_split_points_negative.table_to_split
drop cascades to table citus_split_shard_by_split_points_negative.table_to_split_replication_factor_2
SELECT public.wait_for_resource_cleanup();
wait_for_resource_cleanup
---------------------------------------------------------------------
(1 row)
--END : Cleanup

View File

@ -1275,3 +1275,10 @@ SELECT count(*) FROM pg_catalog.pg_dist_partition WHERE colocationid > 0;
TRUNCATE TABLE pg_catalog.pg_dist_colocation;
ALTER SEQUENCE pg_catalog.pg_dist_colocationid_seq RESTART 100;
ALTER SEQUENCE pg_catalog.pg_dist_placement_placementid_seq RESTART :last_placement_id;
-- make sure we don't have any replication objects leftover on the nodes
SELECT public.wait_for_resource_cleanup();
wait_for_resource_cleanup
---------------------------------------------------------------------
(1 row)

View File

@ -113,4 +113,5 @@ SELECT citus_split_shard_by_split_points(
--BEGIN : Cleanup
\c - postgres - :master_port
DROP SCHEMA "citus_split_shard_by_split_points_negative" CASCADE;
SELECT public.wait_for_resource_cleanup();
--END : Cleanup

View File

@ -607,3 +607,6 @@ TRUNCATE TABLE pg_catalog.pg_dist_colocation;
ALTER SEQUENCE pg_catalog.pg_dist_colocationid_seq RESTART 100;
ALTER SEQUENCE pg_catalog.pg_dist_placement_placementid_seq RESTART :last_placement_id;
-- make sure we don't have any replication objects leftover on the nodes
SELECT public.wait_for_resource_cleanup();