mirror of https://github.com/citusdata/citus.git
Fix flakyness in ch_benchmarks_1 (#6228)
One of our arbitrary config tests would sometimes fail like this in CI:
```diff
su_nationkey,
cust_nation,
l_year;
- supp_nation | cust_nation | l_year | revenue
----------------------------------------------------------------------
- 9 | C | 2008 | 3.00
-(1 row)
-
+ERROR: cannot connect to localhost:10212 to fetch intermediate results
+CONTEXT: while executing command on localhost:10211
```
When looking at the logs it seems like we were running out of
connections:
```
2022-08-23 14:03:52.856 UTC [28122] FATAL: sorry, too many clients already
2022-08-23 14:03:52.860 UTC [21027] ERROR: cannot connect to localhost:10212 to fetch intermediate results
```
This happened with `CitusThreeWorkersManyShards` config. This test on
purpose tries to push the limits of Citus quite far. And the
`ch_benchmarks_1` test is also run in parallel with a few more ones. So
it's not too weird that it ran out of connections. This doubles the
connection limit in the arbitrary config tests to hopefully not hit this
error again.
Example of failed test: https://app.circleci.com/pipelines/github/citusdata/citus/26365/workflows/7a1b5688-85cc-4bc3-ade5-9bd1d83cd0ed/jobs/747908/parallel-runs/1
(cherry picked from commit 21780b4f65
)
pull/6363/head
parent
5d60bbf7f8
commit
f87940221f
|
@ -105,7 +105,7 @@ class CitusBaseClusterConfig(object, metaclass=NewInitCaller):
|
||||||
"citus.enable_repartition_joins": True,
|
"citus.enable_repartition_joins": True,
|
||||||
"citus.repartition_join_bucket_count_per_node": 2,
|
"citus.repartition_join_bucket_count_per_node": 2,
|
||||||
"citus.log_distributed_deadlock_detection": True,
|
"citus.log_distributed_deadlock_detection": True,
|
||||||
"max_connections": 600,
|
"max_connections": 1200,
|
||||||
}
|
}
|
||||||
self.new_settings = {}
|
self.new_settings = {}
|
||||||
self.add_coordinator_to_metadata = False
|
self.add_coordinator_to_metadata = False
|
||||||
|
|
Loading…
Reference in New Issue