Commit 2973c1b2 authored by Rémy Coutable's avatar Rémy Coutable

Merge branch...

Merge branch 'refine-docs-trigger-script-and-use-a-really-long-branch-name-to-test-it' into 'master'

Refine docs trigger script and use a really long branch name to test it

For branch names more than 63 characters we were hitting the
filesystem limit and rsync created a truncated directory.
This change truncates the branch name beforehand so the Review
App URL is properly shown in the merge request widget.

See merge request gitlab-org/gitlab-ce!14354
parents 706426d5 fb9fec00
...@@ -174,7 +174,8 @@ build-package: ...@@ -174,7 +174,8 @@ build-package:
# Review docs base # Review docs base
.review-docs: &review-docs .review-docs: &review-docs
image: ruby:2.4-alpine image: ruby:2.4-alpine
before_script: [] before_script:
- gem install gitlab --no-doc
services: [] services: []
variables: variables:
SETUP_DB: "false" SETUP_DB: "false"
...@@ -193,10 +194,9 @@ review-docs-deploy: ...@@ -193,10 +194,9 @@ review-docs-deploy:
name: review-docs/$CI_COMMIT_REF_NAME name: review-docs/$CI_COMMIT_REF_NAME
# DOCS_REVIEW_APPS_DOMAIN and DOCS_GITLAB_REPO_SUFFIX are secret variables # DOCS_REVIEW_APPS_DOMAIN and DOCS_GITLAB_REPO_SUFFIX are secret variables
# Discussion: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14236/diffs#note_40140693 # Discussion: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14236/diffs#note_40140693
url: http://$CI_COMMIT_REF_SLUG-built-from-ce-ee.$DOCS_REVIEW_APPS_DOMAIN/$DOCS_GITLAB_REPO_SUFFIX url: http://preview-$CI_COMMIT_REF_SLUG.$DOCS_REVIEW_APPS_DOMAIN/$DOCS_GITLAB_REPO_SUFFIX
on_stop: review-docs-cleanup on_stop: review-docs-cleanup
script: script:
- gem install gitlab --no-doc
- scripts/trigger-build-docs deploy - scripts/trigger-build-docs deploy
# Cleanup remote environment of gitlab-docs # Cleanup remote environment of gitlab-docs
...@@ -207,7 +207,6 @@ review-docs-cleanup: ...@@ -207,7 +207,6 @@ review-docs-cleanup:
name: review-docs/$CI_COMMIT_REF_NAME name: review-docs/$CI_COMMIT_REF_NAME
action: stop action: stop
script: script:
- gem install gitlab --no-doc
- scripts/trigger-build-docs cleanup - scripts/trigger-build-docs cleanup
# Retrieve knapsack and rspec_flaky reports # Retrieve knapsack and rspec_flaky reports
......
...@@ -106,21 +106,84 @@ CE and EE. ...@@ -106,21 +106,84 @@ CE and EE.
## Previewing the changes live ## Previewing the changes live
If you want to preview your changes live, you can use the manual `build-docs` If you want to preview the doc changes of your merge request live, you can use
job in your merge request. the manual `review-docs-deploy` job in your merge request.
TIP: **Tip:**
If your branch contains only documentation changes, you can use
[special branch names](#testing) to avoid long running pipelines.
![Manual trigger a docs build](img/manual_build_docs.png) ![Manual trigger a docs build](img/manual_build_docs.png)
This job will: This job will:
1. Create a new branch in the [gitlab-docs](https://gitlab.com/gitlab-com/gitlab-docs) 1. Create a new branch in the [gitlab-docs](https://gitlab.com/gitlab-com/gitlab-docs)
project named after the scheme: `<CE/EE-branch-slug>-built-from-ce-ee` project named after the scheme: `preview-<branch-slug>`
1. Trigger a pipeline and build the docs site with your changes 1. Trigger a cross project pipeline and build the docs site with your changes
Look for the docs URL at the output of the `build-docs` job. After a few minutes, the Review App will be deployed and you will be able to
preview the changes. The docs URL can be found in two places:
>**Note:**
- In the merge request widget
- In the output of the `review-docs-deploy` job, which also includes the
triggered pipeline so that you can investigate whether something went wrong
In case the Review App URL returns 404, follow these steps to debug:
1. **Did you follow the URL from the merge request widget?** If yes, then check if
the link is the same as the one in the job output. It can happen that if the
branch name slug is longer than 35 characters, it is automatically
truncated. That means that the merge request widget will not show the proper
URL due to a limitation of how `environment: url` works, but you can find the
real URL from the output of the `review-docs-deploy` job.
1. **Did you follow the URL from the job output?** If yes, then it means that
either the site is not yet deployed or something went wrong with the remote
pipeline. Give it a few minutes and it should appear online, otherwise you
can check the status of the remote pipeline from the link in the job output.
If the pipeline failed or got stuck, drop a line in the `#docs` chat channel.
TIP: **Tip:**
Someone that has no merge rights to the CE/EE projects (think of forks from
contributors) will not be able to run the manual job. In that case, you can
ask someone from the GitLab team who has the permissions to do that for you.
NOTE: **Note:**
Make sure that you always delete the branch of the merge request you were Make sure that you always delete the branch of the merge request you were
working on. If you don't, the remote docs branch won't be removed either, working on. If you don't, the remote docs branch won't be removed either,
and the server where the Review Apps are hosted will eventually be out of and the server where the Review Apps are hosted will eventually be out of
disk space. disk space.
### Behind the scenes
If you want to know the hot details, here's what's really happening:
1. You manually run the `review-docs-deploy` job in a CE/EE merge request.
1. The job runs the [`scirpts/trigger-build-docs`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/scripts/trigger-build-docs)
script with the `deploy` flag, which in turn:
1. Takes your branch name and applies the following:
- The slug of the branch name is used to avoid special characters since
ultimately this will be used by NGINX.
- The `preview-` prefix is added to avoid conflicts if there's a remote branch
with the same name that you created in the merge request.
- The final branch name is truncated to 42 characters to avoid filesystem
limitations with long branch names (> 63 chars).
1. The remote branch is then created if it doesn't exist (meaning you can
re-run the manual job as many times as you want and this step will be skipped).
1. A new cross-project pipeline is triggered in the docs project.
1. The preview URL is shown both at the job output and in the merge request
widget. You also get the link to the remote pipeline.
1. In the docs project, the pipeline is created and it
[skips the test jobs](https://gitlab.com/gitlab-com/gitlab-docs/blob/8d5d5c750c602a835614b02f9db42ead1c4b2f5e/.gitlab-ci.yml#L50-55)
to lower the build time.
1. Once the docs site is built, the HTML files are uploaded as artifacts.
1. A specific Runner tied only to the docs project, runs the Review App job
that downloads the artifacts and uses `rsync` to transfer the files over
to a location where NGINX serves them.
The following GitLab features are used among others:
- [Manual actions](../ci/yaml/README.md#manual-actions)
- [Multi project pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html)
- [Review Apps](../ci/review_apps/index.md)
- [Artifacts](../ci/yaml/README.md#artifacts)
- [Specific Runner](../ci/runners/README.md#locking-a-specific-runner-from-being-enabled-for-other-projects)
...@@ -2,13 +2,6 @@ ...@@ -2,13 +2,6 @@
require 'gitlab' require 'gitlab'
#
# Give the remote branch a different name than the current one
# in order to avoid conflicts
#
@docs_branch = "#{ENV["CI_COMMIT_REF_SLUG"]}-built-from-ce-ee"
GITLAB_DOCS_REPO = 'gitlab-com/gitlab-docs'.freeze
# #
# Configure credentials to be used with gitlab gem # Configure credentials to be used with gitlab gem
# #
...@@ -17,6 +10,26 @@ Gitlab.configure do |config| ...@@ -17,6 +10,26 @@ Gitlab.configure do |config|
config.private_token = ENV["DOCS_API_TOKEN"] # GitLab Docs bot access token which has only Developer access to gitlab-docs config.private_token = ENV["DOCS_API_TOKEN"] # GitLab Docs bot access token which has only Developer access to gitlab-docs
end end
#
# The remote docs project
#
GITLAB_DOCS_REPO = 'gitlab-com/gitlab-docs'.freeze
#
# Truncate the remote docs branch name if it's more than 63 characters
# otherwise we hit the filesystem limit and the directory name where
# NGINX serves the site won't match the branch name.
#
def docs_branch
# The maximum string length a file can have on a filesystem (ext4)
# is 63 characters. Let's use something smaller to be 100% sure.
max = 42
# Prefix the remote branch with 'preview-' in order to avoid
# name conflicts in the rare case the branch name already
# exists in the docs repo and truncate to max length.
"preview-#{ENV["CI_COMMIT_REF_SLUG"]}"[0...max]
end
# #
# Dummy way to find out in which repo we are, CE or EE # Dummy way to find out in which repo we are, CE or EE
# #
...@@ -28,18 +41,18 @@ end ...@@ -28,18 +41,18 @@ end
# Create a remote branch in gitlab-docs # Create a remote branch in gitlab-docs
# #
def create_remote_branch def create_remote_branch
Gitlab.create_branch(GITLAB_DOCS_REPO, @docs_branch, 'master') Gitlab.create_branch(GITLAB_DOCS_REPO, docs_branch, 'master')
puts "Remote branch '#{@docs_branch}' created" puts "Remote branch '#{docs_branch}' created"
rescue Gitlab::Error::BadRequest rescue Gitlab::Error::BadRequest
puts "Remote branch '#{@docs_branch}' already exists" puts "Remote branch '#{docs_branch}' already exists"
end end
# #
# Remove a remote branch in gitlab-docs # Remove a remote branch in gitlab-docs
# #
def remove_remote_branch def remove_remote_branch
Gitlab.delete_branch(GITLAB_DOCS_REPO, @docs_branch) Gitlab.delete_branch(GITLAB_DOCS_REPO, docs_branch)
puts "Remote branch '#{@docs_branch}' deleted" puts "Remote branch '#{docs_branch}' deleted"
end end
# #
...@@ -50,11 +63,11 @@ def trigger_pipeline ...@@ -50,11 +63,11 @@ def trigger_pipeline
param_name = ee? ? 'BRANCH_EE' : 'BRANCH_CE' param_name = ee? ? 'BRANCH_EE' : 'BRANCH_CE'
# The review app URL # The review app URL
app_url = "http://#{@docs_branch}.#{ENV["DOCS_REVIEW_APPS_DOMAIN"]}/#{ee? ? 'ee' : 'ce'}" app_url = "http://#{docs_branch}.#{ENV["DOCS_REVIEW_APPS_DOMAIN"]}/#{ee? ? 'ee' : 'ce'}"
# Create the pipeline # Create the pipeline
puts "=> Triggering a pipeline..." puts "=> Triggering a pipeline..."
pipeline = Gitlab.run_trigger(GITLAB_DOCS_REPO, ENV["DOCS_TRIGGER_TOKEN"], @docs_branch, { param_name => ENV["CI_COMMIT_REF_NAME"] }) pipeline = Gitlab.run_trigger(GITLAB_DOCS_REPO, ENV["CI_JOB_TOKEN"], docs_branch, { param_name => ENV["CI_COMMIT_REF_NAME"] })
puts "=> Pipeline created:" puts "=> Pipeline created:"
puts "" puts ""
...@@ -77,4 +90,8 @@ when 'deploy' ...@@ -77,4 +90,8 @@ when 'deploy'
trigger_pipeline trigger_pipeline
when 'cleanup' when 'cleanup'
remove_remote_branch remove_remote_branch
else
puts "Please provide a valid option:
deploy - Creates the remote branch and triggers a pipeline
cleanup - Deletes the remote branch and stops the Review App"
end end
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment