Add a job to check if merge to enterprise master would fail (#3777)

* add a job to check if merge to enterprise master would fail

Add a job to check if merge to enterprise master would fail.
The job does the following:
- It checks if there is already a branch with the same name on
enterprise, if so it tries to merge it to enterprise master, if the
merge fails the job fails.
- If the branch doesn't exist on the enterprise, it tries to merge the
current branch to enterprise master, it fails if there is any conflict
while merging.

The motivation is that if a branch on community would create a conflict
on enterprise-master, until we create a PR on enterprise that would
solve this conflict, we won't be able to merge the PR on community. This
way we won't have many conflicts when merging to enterprise master and
the author, who has the most context will be responsible for resolving
the conflict when he has the most context, not after 1 month.

* Improve test suite to be able to easily run locally

* Add documentation on how to resolve conflicts to enterprise master

* Improve enterprise merge script

* Improve merge conflict job README

* Improve merge conflict job README

* Improve merge conflict job README

* Improve merge conflict job README

Co-authored-by: Nils Dijk <nils@citusdata.com>

.circleci/config.yml

@@ -234,7 +234,15 @@ jobs:
            install-and-test-ext --target check-citus-upgrade-mixed --citus-pre-tar /install-pg11-citusv8.3.0.tar
           no_output_timeout: 2m
 
-
+  check-merge-to-enterprise:
+    docker:
+      - image: buildpack-deps:stretch
+    working_directory: /home/circleci/project
+    steps:
+      - checkout
+      - run:
+          command: |
+            ./src/test/scripts/check_enterprise_merge.sh
 
 
 
@@ -242,6 +250,7 @@ workflows:
   version: 2
   build_and_test:
     jobs:
+      - check-merge-to-enterprise
       - build
       - check-style
       - check-sql-snapshots

src/test/scripts/README.md

@@ -0,0 +1,33 @@
+
+# check-merge-to-enterprise Job
+
+When you open a PR on community, if it creates a conflict with enterprise-master, the check-merge-to-enterprise will fail. Say your branch name is `X`, we will refer to `X` on community as `community/X` and on enterprise as `enterprise/X`. If the job already passes, you are done, nothing further required! Otherwise follow the below steps:
+
+- You first need to get approval from your reviewer for `community/X`. Only follow the next steps after you are about to merge the branch to community master. Otherwise, the `check-merge-to-enterprise` check might falsely pass. (For example when you added new commits to `community/X` but forgot to update `enterprise/X`).
+- You need to synchronize the same branch, `X` locally on enterprise (ideally don't push yet). For example if `community` is added as a remote in your enterprise repo, you can do the following:
+
+```bash
+git fetch --all # this will fetch community/X
+git checkout community/X
+git checkout -b X # now you have X in your enterprise repo, which we refer to as enterprise/X
+```
+
+- You need to merge `enterprise-master` to `enterprise/X`. Solve any conflicts( and make sure to remove any parts that should not be in enterprise even though it doesn't have a conflict), on enterprise repository:
+
+```bash
+git checkout enterprise-master
+git pull # make sure enterprise-master in your local is up-to-date
+git checkout X
+git merge origin/enterprise-master # assuming origin is the remote of citus-enterprise
+```
+
+- You should push this to enterprise and open a PR so that the job on community will see this branch. Also this will trigger CI to verify changes are correct.
+- You should get approval for the merge conflict changes on `enterprise/X`, preferably from the same reviewer as they are familiar with the change.
+- You should rerun the `check-merge-to-enterprise` check on `community/X`. You can use re-run from failed option in circle CI.
+- You can now merge `community/X` to `community/master`.
+- You can merge `enterprise/X` to `enterprise-master`.
+- Manually merge `community master` to `enterprise-master`. [https://github.com/citusdata/citus-enterprise/blob/enterprise-master/CONTRIBUTING.md#merging-community-changes-onto-enterprise](https://github.com/citusdata/citus-enterprise/blob/enterprise-master/CONTRIBUTING.md#merging-community-changes-onto-enterprise)
+- You can use `git checkout --ours <conflicting file names>` for all conflicting files. This will use local/our version to resolve the conflicts.
+- Now you can push to `enterprise-master`. Now you can rerun the check on `community/master` and it should pass the check.
+
+The subsequent PRs on community will be able to pass the `check-merge-to-enterprise` check as long as they don't have a conflict with `enterprise-master`.

src/test/scripts/check_enterprise_merge.sh

@@ -0,0 +1,55 @@
+#!/bin/bash
+
+# fail if trying to reference a variable that is not set.
+set -u
+# exit immediately if a command fails
+set -e
+
+# try_merge sees if we can merge "from" branch to "to" branch
+# it will exit with nonzero code if the merge fails because of conflicts.
+try_merge() {
+    to=$1
+    from=$2
+    git checkout "${to}"
+    # this will exit since -e option is set and it will return non-zero code on conflicts.
+    git merge --no-ff --no-commit "${from}"
+
+}
+
+cd /tmp
+if [ ! -d citus-enterprise ]; then
+    git clone https://${GIT_USERNAME}:${GIT_TOKEN}@github.com/citusdata/citus-enterprise
+fi
+
+cd citus-enterprise
+git config user.email "citus-bot@microsoft.com"
+git config user.name "citus bot"
+
+# reset repository into usable state if script ran before
+git fetch origin
+git reset --hard
+git checkout enterprise-master
+git reset --hard origin/enterprise-master
+
+branch_name="${CIRCLE_BRANCH}"
+
+# check if the branch on community exists on enterprise
+# the output will not be empty if it does
+if [ `git branch -r --list origin/$branch_name` ]
+then
+    try_merge enterprise-master origin/$branch_name
+else
+    # add community as a remote if not already added
+    set +e
+    if ! git ls-remote community > /dev/null; then
+        set -e
+        git remote add --no-tags community git@github.com:citusdata/citus.git
+    fi
+    set -e
+
+    # prevent pushes to community and update branch we care about
+    git remote set-url --push community no-pushing
+    git fetch community $branch_name
+
+    try_merge enterprise-master community/$branch_name
+fi