CI checks to check for missing downgrade updates (#6661)

A branch that touches a set of upgrade scripts is also expected to touch
corresponding downgrade scripts as well. To ensure that I introduce a
new CI script. If this script fails, read the output and make sure you
update the downgrade scripts in the printed list.

.circleci/config.yml

@@ -201,6 +201,9 @@ jobs:
       - run:
           name: 'Check if all GUCs are sorted alphabetically'
           command: ci/check_gucs_are_alphabetically_sorted.sh
+      - run:
+          name: 'Check for missing downgrade scripts'
+          command: ci/check_migration_files.sh
 
   check-sql-snapshots:
     docker:

ci/README.md

@@ -283,6 +283,14 @@ actually run in CI. This is most commonly forgotten for newly added CI tests
 that the developer only ran locally. It also checks that all CI scripts have a
 section in this `README.md` file and that they include `ci/ci_helpers.sh`.
 
+## `check_migration_files.sh`
+
+A branch that touches a set of upgrade scripts is also expected to touch
+corresponding downgrade scripts as well. If this script fails, read the output
+and make sure you update the downgrade scripts in the printed list. If you
+really don't need a downgrade to run any SQL. You can write a comment in the
+file explaining why a downgrade step is not necessary.
+
 ## `disallow_c_comments_in_migrations.sh`
 
 We do not use C-style comments in migration files as the stripped

ci/check_migration_files.sh

@@ -0,0 +1,33 @@
+#! /bin/bash
+
+set -euo pipefail
+# shellcheck disable=SC1091
+source ci/ci_helpers.sh
+
+# This file checks for the existence of downgrade scripts for every upgrade script that is changed in the branch.
+
+# create list of migration files for upgrades
+upgrade_files=$(git diff --name-only origin/main | { grep "src/backend/distributed/sql/citus--.*sql" || exit 0 ; })
+downgrade_files=$(git diff --name-only origin/main | { grep "src/backend/distributed/sql/downgrades/citus--.*sql" || exit 0 ; })
+ret_value=0
+
+for file in $upgrade_files
+do
+    # There should always be 2 matches, and no need to avoid splitting here
+    # shellcheck disable=SC2207
+    versions=($(grep --only-matching --extended-regexp "[0-9]+\.[0-9]+[-.][0-9]+" <<< "$file"))
+
+    from_version=${versions[0]};
+    to_version=${versions[1]};
+
+    downgrade_migration_file="src/backend/distributed/sql/downgrades/citus--$to_version--$from_version.sql"
+
+    # check for the existence of migration scripts
+    if [[ $(grep --line-regexp --count downgrade_migration_file <<< "$downgrade_files") == 0 ]]
+    then
+        echo "$file is updated, but $downgrade_migration_file is not updated in branch"
+        ret_value=1
+    fi
+done
+
+exit $ret_value;