Automatically merge CE master into EE master

This sets up GitLab CI to automatically push CE master changes into EE
master, or revert them if the changes cause merge conflicts. The CI
configuration contains a single job to do this: `merge:master`. This job
is executed for every push to master, and periodically using a CI
schedule.

The periodic job is necessary because incremental jobs may not be able
to revert commits if newly added commits depend on these commits. By
re-running the job periodically (including all changes since a large
enough time frame), we can ensure that such commits are also reverted
(if they still conflict at that time).

The job runs in its own "merge" stage, _after_ the build and prepare
stages, but _before_ running the tests. This ensures that randomly
failing tests won't prevent code from being merged into EE. Running the
stage after the "prepare" stage reduces the chances of the job reverting
CE changes just because it ran before a corresponding EE MR was merged
into EE master.

.gitlab-ci.yml

@@ -48,6 +48,7 @@ after_script:
 stages:
   - build
   - prepare
+  - merge
   - test
   - post-test
   - pages
@@ -1025,3 +1026,26 @@ schedule:review-cleanup:
     - gem install gitlab --no-document
   script:
     - ruby -rrubygems scripts/review_apps/automated_cleanup.rb
+
+merge:master:
+  image: registry.gitlab.com/gitlab-org/merge-train
+  stage: merge
+  # The global before_script/after_script blocks break this job, or aren't
+  # necessary. These two lines result in them being ignored.
+  before_script: []
+  after_script: []
+  only:
+    refs:
+      - master
+      - schedules
+    variables:
+      - $CI_PROJECT_PATH == "gitlab-org/gitlab-ce"
+      - $MERGE_TRAIN_SSH_PUBLIC_KEY
+      - $MERGE_TRAIN_SSH_PRIVATE_KEY
+      - $MERGE_TRAIN_API_TOKEN
+  script:
+    - scripts/merge-train
+  cache:
+    paths:
+      - gitlab-ee
+    key: "merge:master"

scripts/merge-train

+#!/bin/sh
+
+set -e
+
+# The name (including namespace) of the EE repository to merge commits into.
+EE_PROJECT='gitlab-org/gitlab-ee'
+
+# The directory to clone GitLab EE into.
+EE_DIRECTORY="$CI_PROJECT_DIR/gitlab-ee"
+
+# The EE branch to merge the changes into.
+EE_BRANCH='master'
+
+# Runs an incremental or periodic merge of CE to EE. This script should be run
+# from a container built using https://gitlab.com/gitlab-org/merge-train.
+
+# Merges (or reverts) commits in a batch (based on CI_COMMIT_BEFORE_SHA and
+# CI_COMMIT_SHA), or since a specific commit.
+#
+# The optional first argument of this function should be a SHA of a commit. When
+# specified, all commits since this commit will be processed.
+merge() {
+    # We need to source the configure-ssh script instead of running it with
+    # `sh`, since it uses `eval` for SSH agent and we want the result of that to
+    # persist.
+    #
+    # shellcheck disable=SC1091
+    . /app/bin/configure-ssh
+
+    # We can not perform a shallow clone, as this results in Git sometimes
+    # refusing to fetch from CE. To work around this, we perform a full clone
+    # but cache the repository in CI. If a cached repository exists, we simply
+    # just pull from `master`.
+    if [ -d "$EE_DIRECTORY" ]
+    then
+        echo "Updating existing clone of $EE_PROJECT"
+
+        git -C "$EE_DIRECTORY" pull --quiet origin "$EE_BRANCH"
+    else
+        echo "Cloning $EE_PROJECT"
+
+        git clone --quiet --single-branch --branch "$EE_BRANCH" \
+            "git@gitlab.com:$EE_PROJECT.git" "$EE_DIRECTORY"
+    fi
+
+    cd /app
+
+    env GITLAB_TOKEN="$MERGE_TRAIN_API_TOKEN" \
+        bundle exec /app/bin/merge-train "$CI_PROJECT_DIR" "$EE_DIRECTORY" \
+        --source-name "$CI_PROJECT_PATH" \
+        --target-branch "$EE_BRANCH" \
+        ${1:+--before "$1"}
+}
+
+# Merges (or reverts) all commits since a point in time as supported by `git log
+# --since`, such as "12 hours ago".
+merge_since() {
+    commit="$(git log --since="$MERGE_SINCE" --reverse --format=%H | head -n1)"
+
+    if [ "$commit" = '' ]
+    then
+        echo "There are no commits to merge since $MERGE_SINCE"
+    else
+        merge "$commit"
+    fi
+}
+
+if [ "$MERGE_SINCE" ]
+then
+    merge_since
+else
+    merge
+fi