@@ -116,7 +116,7 @@ unexpected timing. For example, when a source or target branch is advanced.
In this case, the pipeline fails because of `fatal: reference is not a tree:` error,
which indicates that the checkout-SHA is not found in the merge ref.
This behavior was improved at GitLab 12.4 by introducing [Persistent pipeline refs](../../pipelines/index.md#persistent-pipeline-refs).
This behavior was improved at GitLab 12.4 by introducing [Persistent pipeline refs](../../pipelines/index.md#troubleshooting-fatal-reference-is-not-a-tree).
You should be able to create pipelines at any timings without concerning the error.
webcast to see a comprehensive demo of GitLab CI/CD pipeline.
webcast to see a comprehensive demo of a GitLab CI/CD pipeline.
Pipelines are the top-level component of continuous integration, delivery, and deployment.
Pipelines comprise:
- Jobs that define what to run. For example, code compilation or test runs.
- Stages that define when and how to run. For example, that tests run only after code compilation.
- Jobs, which define *what* to do. For example, jobs that compile or test code.
- Stages, which define *when* to run the jobs. For example, stages that run tests after stages that compile the code.
Multiple jobs in the same stage are executed by [Runners](../runners/README.md) in parallel, if there are enough concurrent [Runners](../runners/README.md).
Jobs are executed by [Runners](../runners/README.md). Multiple jobs in the same stage are executed in parallel,
if there are enough concurrent runners.
If all the jobs in a stage:
- Succeed, the pipeline moves on to the next stage.
- Fail, the next stage is not (usually) executed and the pipeline ends early.
In general, pipelines are executed automatically and require no intervention once created. However, there are
also times when you can manually interact with a pipeline.
A typical pipeline might consist of four stages, executed in the following order:
- A `build` stage, with a job called `compile`.
- A `test` stage, with two jobs called `test1` and `test2`.
- A `staging` stage, with a job called `deploy-to-stage`.
- A `production` stage, with a job called `deploy-to-prod`.
NOTE: **Note:**
If you have a [mirrored repository that GitLab pulls from](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository-starter),
you may need to enable pipeline triggering in your project's
**Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**.
## Simple pipeline example
## Types of pipelines
As an example, imagine a pipeline consisting of four stages, executed in the following order:
Pipelines can be configured in many different ways:
-`build`, with a job called `compile`.
-`test`, with two jobs called `test` and `test2`.
-`staging`, with a job called `deploy-to-stage`.
-`production`, with a job called `deploy-to-prod`.
-[Multi-project pipelines](../multi_project_pipelines.md) combine pipelines for different projects together.
-[Parent-Child pipelines](../parent_child_pipelines.md) break down complex pipelines
into one parent pipeline that can trigger multiple child sub-pipelines, which all
run in the same project and with the same SHA.
-[Pipelines for Merge Requests](../merge_request_pipelines/index.md) run for merge
requests only (rather than for every commit).
-[Pipelines for Merged Results](../merge_request_pipelines/pipelines_for_merged_results/index.md)
are merge request pipelines that act as though the changes from the source branch have
For example, your pipeline might start automatically, but it requires manual action to
[deploy to production](../environments.md#configuring-manual-deployments). In the example below, the `production`
stage has a job with a manual action.
In the example above:
-`date +%s`: The Unix timestamp (for example `1560896352`).
-`my_first_section`: The name given to the section.
-`\r\e[0K`: Prevents the section markers from displaying in the rendered (colored)
job log, but they are displayed in the raw job log. To see them, in the top right
of the job log, click **{doc-text}** (**Show complete raw**).
-`\r`: carriage return.
-`\e[0K`: clear line ANSI escape code.
#### Start multiple manual actions in a stage
Sample raw job log:
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/27188) in GitLab 11.11.
```plaintext
section_start:1560896352:my_first_section\r\e[0KHeader of the 1st collapsible section
this line should be hidden when collapsed
section_end:1560896353:my_first_section\r\e[0K
```
Multiple manual actions in a single stage can be started at the same time using the "Play all manual" button.
Once the user clicks this button, each individual manual action will be triggered and refreshed
to an updated status.
### Pipeline success and duration charts
This functionality is only available:
> - Introduced in GitLab 3.1.1 as Commit Stats, and later renamed to Pipeline Charts.
> - [Renamed](https://gitlab.com/gitlab-org/gitlab/issues/38318) to CI / CD Analytics in GitLab 12.8.
- For users with at least Developer access.
- If the stage contains [manual actions](#add-manual-interaction-to-your-pipeline).
GitLab tracks the history of your pipeline successes and failures, as well as how long each pipeline ran. To view this information, go to **Analytics > CI / CD Analytics**.
### Delete a pipeline
View successful pipelines:
> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/24851) in GitLab 12.7.
Deleting a pipeline will expire all pipeline caches, and delete all related objects,
such as builds, logs, artifacts, and triggers. **This action cannot be undone.**
## Pipeline quotas
### Pipeline quotas
Each user has a personal pipeline quota that tracks the usage of shared runners in all personal projects.
Each group has a [usage quota](../../subscriptions/index.md#ci-pipeline-minutes) that tracks the usage of shared runners for all projects created within the group.
When a pipeline is triggered, regardless of who triggered it, the pipeline quota for the project owner's [namespace](../../user/group/index.md#namespaces) is used. In this case, the namespace can be the user or group that owns the project.
### How pipeline duration is calculated
#### How pipeline duration is calculated
Total running time for a given pipeline excludes retries and pending
(queued) time.
...
...
@@ -216,245 +226,285 @@ The union of A, B, and C is (1, 4) and (6, 7). Therefore, the total running time
(4 - 1) + (7 - 6) => 4
```
## Configuring pipelines
### Pipeline security on protected branches
Pipelines, and their component jobs and stages, are defined in the [`.gitlab-ci.yml`](../yaml/README.md) file for each project.
A strict security model is enforced when pipelines are executed on
The following actions are allowed on protected branches only if the user is
[allowed to merge or push](../../user/project/protected_branches.md#using-the-allowed-to-merge-and-allowed-to-push-settings)
on that specific branch:
- Jobs are the [basic configuration](../yaml/README.md#introduction) component.
- Stages are defined using the [`stages`](../yaml/README.md#stages) keyword.
- Run manual pipelines (using the [Web UI](#run-a-pipeline-manually) or [pipelines API](#pipelines-api)).
- Run scheduled pipelines.
- Run pipelines using triggers.
- Trigger manual actions on existing pipelines.
- Retry or cancel existing jobs (using the Web UI or pipelines API).
For all available configuration options, see the [GitLab CI/CD Pipeline Configuration Reference](../yaml/README.md).
**Variables** marked as **protected** are accessible only to jobs that
run on protected branches, preventing untrusted users getting unintended access to
sensitive information like deployment credentials and tokens.
### Settings and schedules
**Runners** marked as **protected** can run jobs only on protected
branches, preventing untrusted code from executing on the protected runner and
preserving deployment keys and other credentials from being unintentionally
accessed. In order to ensure that jobs intended to be executed on protected
runners will not use regular runners, they must be tagged accordingly.
In addition to configuring jobs through `.gitlab-ci.yml`, additional configuration options are available
through the GitLab UI:
## View jobs in a pipeline
- Pipeline settings for each project. For more information, see [Pipeline settings](settings.md).
- Schedules for pipelines. For more information, see [Pipeline schedules](schedules.md).
When you access a pipeline, you can see the related jobs for that pipeline.
### Grouping jobs
Clicking an individual job will show you its job log, and allow you to:
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/6242) in GitLab 8.12.
- Cancel the job.
- Retry the job.
- Erase the job log.
If you have many similar jobs, your [pipeline graph](#visualizing-pipelines) becomes long and hard
to read.
### See why a job failed
For that reason, similar jobs can automatically be grouped together.
If the job names are formatted in certain ways, they will be collapsed into
a single group in regular pipeline graphs (not the mini graphs).
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/17782) in GitLab 10.7.
You'll know when a pipeline has grouped jobs if you don't see the retry or
cancel button inside them. Hovering over them will show the number of grouped
jobs. Click to expand them.
When a pipeline fails or is allowed to fail, there are several places where you
can find the reason:
- In the [pipeline graph](#visualize-pipelines), on the pipeline detail view.
- In the pipeline widgets, in the merge requests and commit pages.
- In the job views, in the global and detailed views of a job.
#### Configuring grouping
In each place, if you hover over the failed job you can see the reason it failed.
In the pipeline [configuration file](../yaml/README.md), job names must include two numbers separated with one of
the following (you can even use them interchangeably):
- A space.
- A slash (`/`).
- A colon (`:`).
In [GitLab 10.8](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/17814) and later,
you can also see the reason it failed on the Job detail page.
NOTE: **Note:**
More specifically, it uses [this](https://gitlab.com/gitlab-org/gitlab/blob/2f3dc314f42dbd79813e6251792853bc231e69dd/app/models/commit_status.rb#L99) regular expression: `\d+[\s:\/\\]+\d+\s*`.
### The order of jobs in a pipeline
#### How grouping works
The order of jobs in a pipeline depends on the type of pipeline graph.
The jobs will be ordered by comparing those two numbers from left to right. You
usually want the first to be the index and the second the total.
- For [regular pipeline graphs](#regular-pipeline-graphs), jobs are sorted by name.
- For [pipeline mini graphs](#pipeline-mini-graphs), jobs are sorted by severity and then by name.
For example, the following jobs will be grouped under a job named `test`:
The order of severity is:
-`test 0 3`
-`test 1 3`
-`test 2 3`
- failed
- warning
- pending
- running
- manual
- scheduled
- canceled
- success
- skipped
- created
The following jobs will be grouped under a job named `test ruby`:
For example:
-`test 1:2 ruby`
-`test 2:2 ruby`
The following jobs will be grouped under a job named `test ruby` as well:
### Group jobs in a pipeline
-`1/3 test ruby`
-`2/3 test ruby`
-`3/3 test ruby`
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/6242) in GitLab 8.12.
### Pipelines for merge requests
If you have many similar jobs, your [pipeline graph](#visualize-pipelines) becomes long and hard
to read.
GitLab supports configuring pipelines that run only for merge requests. For more information, see
[Pipelines for merge requests](../merge_request_pipelines/index.md).
You can automatically group similar jobs together. If the job names are formatted in a certain way,
they will be collapsed into a single group in regular pipeline graphs (not the mini graphs).
### Badges
You'll know when a pipeline has grouped jobs if you don't see the retry or
cancel button inside them. Hovering over them will show the number of grouped
jobs. Click to expand them.
Pipeline status and test coverage report badges are available and configurable for each project.
For information on adding pipeline badges to projects, see [Pipeline badges](settings.md#pipeline-badges).
To create a group of jobs, in the [CI/CD pipeline configuration file](../yaml/README.md),
separate each job name with a number and one of the following:
## Multi-project pipelines
- A slash (`/`), for example, `test 1/3`, `test 2/3`, `test 3/3`.
- A colon (`:`), for example, `test 1:3`, `test 2:3`, `test 3:3`.
- A space, for example `test 0 3`, `test 1 3`, `test 2 3`.
Pipelines for different projects can be combined together into [Multi-project pipelines](../multi_project_pipelines.md).
You can use these symbols interchangeably.
[Multi-project pipeline graphs](../multi_project_pipelines.md#multi-project-pipeline-visualization-premium) help
you visualize the entire pipeline, including all cross-project inter-dependencies. **(PREMIUM)**
For example, these three jobs will be in a group named `build ruby`:
## Parent-child pipelines
```yaml
build ruby 1/3:
stage:build
script:
-echo "ruby1"
Complex pipelines can be broken down into one parent pipeline that can trigger
multiple child sub-pipelines, which all run in the same project and with the same SHA.
build ruby 2/3:
stage:build
script:
-echo "ruby2"
For more information, see [Parent-Child pipelines](../parent_child_pipelines.md).
build ruby 3/3:
stage:build
script:
-echo "ruby3"
```
## Working with pipelines
In the pipeline, the result is a group named `build ruby` with three jobs:
In general, pipelines are executed automatically and require no intervention once created.
However, there are instances where you'll need to interact with pipelines. These are documented below.
The jobs will be ordered by comparing the numbers from left to right. You
usually want the first number to be the index and the second number to be the total.
|  |  |
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/21767) in GitLab 11.4.
### Pipeline success and duration charts
When you do not want to run a job immediately, you can use the [`when:delayed`](../yaml/README.md#whendelayed) parameter to
delay a job's execution for a certain period.
> - Introduced in GitLab 3.1.1 as Commit Stats, and later renamed to Pipeline Charts.
> - [Renamed](https://gitlab.com/gitlab-org/gitlab/issues/38318) to CI / CD Analytics in GitLab 12.8.
This is especially useful for timed incremental rollout where new code is rolled out gradually.
GitLab tracks the history of your pipeline successes and failures, as well as how long each pipeline ran. To view this information, go to **Analytics > CI / CD Analytics**.
For example, if you start rolling out new code and:
View successful pipelines:
- Users do not experience trouble, GitLab can automatically complete the deployment from 0% to 100%.
- Users experience trouble with the new code, you can stop the timed incremental rollout by canceling the pipeline
and [rolling](../environments.md#retrying-and-rolling-back) back to the last stable version.
Deleting a pipeline will expire all pipeline caches, and delete all related objects,
such as builds, logs, artifacts, and triggers. **This action cannot be undone.**
## Most Recent Pipeline
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/50499) in GitLab 12.3.
There's a link to the latest pipeline for the last commit of a given branch at `/project/pipelines/[branch]/latest`. Also, `/project/pipelines/latest` will redirect you to the latest pipeline for the last commit on the project's default branch.
## Security on protected branches
A strict security model is enforced when pipelines are executed on
@@ -101,7 +101,7 @@ or link to useful information directly in the merge request page:
| [Metrics Reports](../../../ci/metrics_reports.md)**(PREMIUM)** | Display the Metrics Report on the merge request so that it's fast and easy to identify changes to important metrics. |
| [Multi-Project pipelines](../../../ci/multi_project_pipelines.md)**(PREMIUM)** | When you set up GitLab CI/CD across multiple projects, you can visualize the entire pipeline, including all cross-project interdependencies. |
| [Pipelines for merge requests](../../../ci/merge_request_pipelines/index.md) | Customize a specific pipeline structure for merge requests in order to speed the cycle up by running only important jobs. |
| [Pipeline Graphs](../../../ci/pipelines/index.md#visualizing-pipelines) | View the status of pipelines within the merge request, including the deployment process. |
| [Pipeline Graphs](../../../ci/pipelines/index.md#visualize-pipelines) | View the status of pipelines within the merge request, including the deployment process. |
| [Test Coverage visualization](test_coverage_visualization.md) | See test coverage results for merge requests, within the file diff. |
