Merge branch 'jyavorska-master-patch-24279' into 'master'

Add templates for use with rules keyword

Closes #217732

See merge request gitlab-org/gitlab!32070

doc/ci/merge_request_pipelines/index.md

@@ -37,79 +37,23 @@ To enable pipelines for merge requests:
 
 ## Configuring pipelines for merge requests
 
-To configure pipelines for merge requests, configure your [CI/CD configuration file](../yaml/README.md).
-There are a few different ways to do this.
+To configure pipelines for merge requests you need to configure your [CI/CD configuration file](../yaml/README.md).
+There are a few different ways to do this:
 
-### Enable pipelines for merge requests for all jobs
+### Use `rules` to run pipelines for merge requests
 
-The recommended method for enabling pipelines for merge requests for all jobs in
-a pipeline is to use [`workflow:rules`](../yaml/README.md#workflowrules).
-
-In this example, the pipeline always runs for all merge requests, as well as for all changes
-to the master branch:
-
-```yaml
-workflow:
-  rules:
-    - if: $CI_MERGE_REQUEST_ID               # Execute jobs in merge request context
-    - if: $CI_COMMIT_BRANCH == 'master'      # Execute jobs when a new commit is pushed to master branch
-
-build:
-  stage: build
-  script: ./build
-
-test:
-  stage: test
-  script: ./test
-
-deploy:
-  stage: deploy
-  script: ./deploy
-```
-
-### Enable pipelines for merge requests for specific jobs
-
-To enable pipelines for merge requests for specific jobs, you can use
-[`rules`](../yaml/README.md#rules).
-
-In the following example:
-
-- The `build` job runs for all changes to the `master` branch, as well as for all merge requests.
-- The `test` job runs for all merge requests.
-- The `deploy` job runs for all changes to the `master` branch, but does *not* run
-  for merge requests.
-
-```yaml
-build:
-  stage: build
-  script: ./build
-  rules:
-    - if: $CI_COMMIT_BRANCH == 'master'   # Execute jobs when a new commit is pushed to master branch
-    - if: $CI_MERGE_REQUEST_ID            # Execute jobs in merge request context
-
-test:
-  stage: test
-  script: ./test
-  rules:
-    - if: $CI_MERGE_REQUEST_ID             # Execute jobs in merge request context
-
-deploy:
-  stage: deploy
-  script: ./deploy
-  rules:
-    - if: $CI_COMMIT_BRANCH == 'master'    # Execute jobs when a new commit is pushed to master branch
-```
+When using `rules`, which is the preferred method, we recommend starting with one
+of the [`workflow:rules` templates](../yaml/README.md#workflowrules-templates) to ensure
+your basic configuration is correct. Instructions on how to do this, as well as how
+to customize, are available at that link.
 
 ### Use `only` or `except` to run pipelines for merge requests
 
-NOTE: **Note**:
-The [`only` / `except`](../yaml/README.md#onlyexcept-basic) keywords are going to be deprecated
-and you should not use them.
-
-To enable pipelines for merge requests, you can use `only / except`. When you use this method,
-you have to specify `only: - merge_requests` for each job.
+If you want to continue using `only/except`, this is possible but please review the drawbacks
+below.
 
-In this example, the pipeline contains a `test` job that is configured to run on merge requests.
+When you use this method, you have to specify `only: - merge_requests` for each job. In this
+example, the pipeline contains a `test` job that is configured to run on merge requests.
 
 The `build` and `deploy` jobs don't have the `only: - merge_requests` parameter,
 so they will not run on merge requests.
@@ -259,60 +203,16 @@ The variable names begin with the `CI_MERGE_REQUEST_` prefix.
 
 ### Two pipelines created when pushing to a merge request
 
-If two pipelines are created when you push a new change to a merge request,
-check your CI configuration file.
-
-For example, with this `.gitlab-ci.yml` configuration:
-
-```yaml
-test:
-  script: ./test
-  rules:
-    - if: $CI_MERGE_REQUEST_ID                      # Include this job in pipelines for merge request
-    - if: $CI_COMMIT_BRANCH                         # Include this job in all branch pipelines
-  # Or, if you are using the `only:` keyword:
-  # only:
-  #  - merge_requests
-  #  - branches
-```
-
-Two pipelines are created when you push a commit to a branch that also has a pending
-merge request:
-
-- A merge request pipeline that runs for the changes in the merge request. In
-  **CI/CD > Pipelines**, the merge request icon (**{merge-request}**)
-  and the merge request ID are displayed. If you hover over the ID, the merge request name is displayed.
-
-  ![MR pipeline icon example](img/merge_request_pipelines_doubled_MR_v12_09.png)
-
-- A "branch" pipeline that runs for the commit pushed to the branch. In **CI/CD > Pipelines**,
-  the branch icon (**{branch}**) and branch name are displayed. This pipeline is
-  created even if no merge request exists.
-
-  ![branch pipeline icon example](img/merge_request_pipelines_doubled_branch_v12_09.png)
-
-With the example configuration above, there is overlap between these two events.
-When you push a commit to a branch that also has an open merge request pending,
-both types of pipelines are created.
-
-To fix this overlap, you must explicitly define which job should run for which
-purpose, for example:
-
-```yaml
-test:
-  script: ./test
-  rules:
-    - if: $CI_MERGE_REQUEST_ID                      # Include this job in pipelines for merge request
-    - if: $CI_COMMIT_BRANCH == 'master'             # Include this job in master branch pipelines
-```
+If you are experiencing duplicated pipelines when using `rules`, take a look at
+the [key details when using `rules`](../yaml/README.md#key-details-when-using-rules),
+which will help you get your starting configuration correct.
 
-Similar `rules:` should be added to all jobs to avoid any overlapping pipelines. Alternatively,
-you can use the [`workflow:`](../yaml/README.md#exclude-jobs-with-rules-from-certain-pipelines)
-parameter to add the same rules to all jobs globally.
+If you are seeing two pipelines when using `only/except`, please see the caveats
+related to using `only/except` above (or, consider moving to `rules`).
 
 ### Two pipelines created when pushing an invalid CI configuration file
 
-Similar to above, pushing to a branch with an invalid CI configuration file can trigger
+Pushing to a branch with an invalid CI configuration file can trigger
 the creation of two types of failed pipelines. One pipeline is a failed merge request
 pipeline, and the other is a failed branch pipeline, but both are caused by the same
 invalid configuration.

doc/ci/yaml/README.md

@@ -286,7 +286,45 @@ determine whether or not a pipeline is created. It currently accepts a single
 `rules:` key that operates similarly to [`rules:` defined within jobs](#rules),
 enabling dynamic configuration of the pipeline.
 
-The configuration options currently available for `workflow:rules` are:​
+#### `workflow:rules` templates
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217732) in GitLab 13.0.
+
+We provide pre-made templates for use with your pipelines that set up `workflow: rules`
+for common scenarios. Usage of these will make things easier and prevent duplicate pipelines from running.
+
+The [`Branch-Pipelines` template](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Workflows/Branch-Pipelines.gitlab-ci.yml)
+makes your pipelines run for branches and tags.
+
+Branch pipeline status will be displayed within merge requests that use that branch
+as a source, but this pipeline type does not support any features offered by
+[Merge Request Pipelines](../merge_request_pipelines/) like
+[Pipelines for Merge Results](../merge_request_pipelines/#pipelines-for-merged-results-premium)
+or [Merge Trains](../merge_request_pipelines/pipelines_for_merged_results/merge_trains/).
+Use this template if you are intentionally avoiding those features.
+
+It is [included](#include) as follows:
+
+```yaml
+include:
+  - template: 'Workflows/Branch-Pipelines.gitlab-ci.yml'
+```
+
+The [`MergeRequest-Pipelines` include](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Workflows/MergeRequest-Pipelines.gitlab-ci.yml) sets your pipelines to run for the default branch (usually `master`), tags, and
+The [`MergeRequest-Pipelines` template](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Workflows/MergeRequest-Pipelines.gitlab-ci.yml)
+makes your pipelines run for the default branch (usually `master`), tags, and
+all types of merge request pipelines. Use this template if you use any of the
+the [Pipelines for Merge Requests features](../merge_request_pipelines/), as mentioned
+above.
+
+It is [included](#include) as follows:
+
+```yaml
+include:
+  - template: 'Workflows/MergeRequest-Pipelines.gitlab-ci.yml'
+```
+
+If you prefer to define your own rules, the configuration options currently available are:​
 
 - [`if`](#rulesif): Define a rule.
 - [`when`](#when): May be set to `always` or `never` only. If not provided, the default value is `always`​.
@@ -858,10 +896,26 @@ the `.template` job, and uses the `alpine` Docker image as defined in the local
 
 `rules` allows for a list of individual rule objects to be evaluated
 *in order*, until one matches and dynamically provides attributes to the job.
-Note that `rules` can't be used in combination with `only/except` since it's intended
-to replace that functionality. If you attempt to do this the linter will return a
+
+CAUTION: **Caution:**
+`rules` can't be used in combination with `only/except` as it is a replacement for that functionality. If you attempt to do this, the linter will return a
 `key may not be used with rules` error.
 
+#### Key details when using `rules`
+
+A very important difference between `rules` and `only/except`, is that jobs defined
+with `rules` trigger merge request pipelines by default, but `only/except` jobs do not.
+This may be surprising if migrating from `only` and `except`, so new users of `rules`
+can use one of the [`workflow: rules` templates](#workflowrules-templates) to get started.
+This will ensure that the behavior is more stable as you start adding additional `rules`
+blocks, and will avoid issues like creating a duplicate, merge request (detached) pipeline.
+
+We don't recomment mixing `only/except` jobs with `rules` jobs in the same pipeline.
+It may not cause YAML errors, but debugging the exact execution behavior can be complex
+due to the different default behaviors of `only/except` and `rules`.
+
+### Rules clauses
+
 Available rule clauses include:
 
 - [`if`](#rulesif) (similar to [`only:variables`](#onlyvariablesexceptvariables))
@@ -995,47 +1049,6 @@ job:
 
 In this example, if the first rule matches, then the job will have `when: manual` and `allow_failure: true`.
 
-#### Exclude jobs with `rules:` from certain pipelines
-
-Jobs with `rules:` can cause two pipelines to be created unexpectedly:
-
-- One pipeline from pushing a commit to a branch.
-- A second ["detached" pipeline for a merge request](../merge_request_pipelines/index.md).
-
-`only` and `except` jobs don't trigger merge request pipelines by default, but this
-is not the case for jobs with `rules:`, which may be surprising if migrating from `only`
-and `except` to `rules:`.
-
-If you're using `rules:` and you see two pipelines for commits to branches that have
-a merge request, you have two options:
-
-- Individually exclude each job that uses `rules:` from merge request pipelines. The
-  example below will cause the job to **not** run in *pipelines for merge requests*,
-  but it **will** run in pipelines for *new tags and pipelines running on branch refs*:
-
-  ```yaml
-  job:
-    rules:
-      - if: $CI_MERGE_REQUEST_ID
-        when: never
-      - when: manual
-    script:
-      - echo hello
-  ```
-
-- Add a global [`workflow: rules`](#workflowrules) to allow pipelines in only certain
-  situations. The example below will only run pipelines for merge requests, new tags and
-  changes to master. It will **not** run any pipelines *on any branch except master*, but
-  it will run **detached merge request pipelines** for any merge request, targeting any branch:
-
-  ```yaml
-  workflow:
-    rules:
-      - if: $CI_MERGE_REQUEST_ID
-      - if: $CI_COMMIT_TAG
-      - if: $CI_COMMIT_BRANCH == "master"
-  ```
-
 #### Complex rule clauses
 
 To conjoin `if`, `changes`, and `exists` clauses with an AND, use them in the

lib/gitlab/ci/templates/Workflows/Branch-Pipelines.gitlab-ci.yml

+# Read more on when to use this template at
+# https://docs.gitlab.com/ee/ci/yaml/#workflowrules
+
+workflow:
+  rules:
+    - if: $CI_COMMIT_TAG
+    - if: $CI_COMMIT_BRANCH

lib/gitlab/ci/templates/Workflows/MergeRequest-Pipelines.gitlab-ci.yml

+# Read more on when to use this template at
+# https://docs.gitlab.com/ee/ci/yaml/#workflowrules
+
+workflow:
+  rules:
+    - if: $CI_MERGE_REQUEST_IID
+    - if: $CI_COMMIT_TAG
+    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH