From 532b4575546593ff3649ca14c57cee939ef5b2a4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=96nder=20Kalac=C4=B1?= Date: Mon, 23 Nov 2020 19:20:13 +0100 Subject: [PATCH] Solidify the slow-start algorithm (#4318) The adaptive executor emulates the TCP's slow start algorithm. Whenever the executor needs new connections, it doubles the number of connections established in the previous iteration. This approach is powerful. When the remote queries are very short (like index lookup with < 1ms), even a single connection is sufficent most of the time. When the remote queries are long, the executor can quickly establish necessary number of connections. One missing piece on our implementation seems that the executor keeps doubling the number of connections even if the previous connection attempts have been finalized. Instead, we should wait until all the attempts are finalized. This is how TCP's slow-start works. Plus, it decreases the unnecessary pressure on the remote nodes. --- .../distributed/executor/adaptive_executor.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/src/backend/distributed/executor/adaptive_executor.c b/src/backend/distributed/executor/adaptive_executor.c index 37c7f65e2..fefb2e842 100644 --- a/src/backend/distributed/executor/adaptive_executor.c +++ b/src/backend/distributed/executor/adaptive_executor.c @@ -2470,6 +2470,20 @@ ShouldWaitForSlowStart(WorkerPool *workerPool) { return true; } + + /* + * Refrain from establishing new connections unless we have already + * finalized all the earlier connection attempts. This prevents unnecessary + * load on the remote nodes and emulates the TCP slow-start algorithm. + */ + int initiatedConnectionCount = list_length(workerPool->sessionList); + int finalizedConnectionCount = + workerPool->activeConnectionCount + workerPool->failedConnectionCount; + if (finalizedConnectionCount < initiatedConnectionCount) + { + return true; + } + return false; }