| [Introduction to pipelines and jobs](pipelines.md) | Provides an overview of GitLab CI/CD and jobs. |
| [Creating and using CI/CD pipelines](pipelines.md) | Understand, visualize, create, and use CI/CD pipelines. |
| [CI/CD Variables](variables/README.md) | How environment variables can be configured and made available in pipelines. |
| [CI/CD Variables](variables/README.md) | How environment variables can be configured and made available in pipelines. |
| [Where variables can be used](variables/where_variables_can_be_used.md) | A deeper look into where and how CI/CD variables can be used. |
| [Where variables can be used](variables/where_variables_can_be_used.md) | A deeper look into where and how CI/CD variables can be used. |
| [User](../user/permissions.md#gitlab-cicd-permissions) and [job](../user/permissions.md#job-permissions) permissions | Learn about the access levels a user can have for performing certain CI actions. |
| [User](../user/permissions.md#gitlab-cicd-permissions) and [job](../user/permissions.md#job-permissions) permissions | Learn about the access levels a user can have for performing certain CI actions. |
Pipelines are the top-level component of continuous integration, deployment, and delivery.
Pipelines comprise:
- Jobs that define what to run. For example, code compilation or test runs.
- Stages that define when and how to run. For example, that test run after code compilation.
Jobs in a stage are executed by [Runners](runners/README.md) in parallel, if there are enough concurrent [Runners](runners/README.md).
If the jobs in a stage:
- Succeed, the pipeline moves on to the next stage.
- Fail, the next stage is not (usually) executed.
NOTE: **Note:**
NOTE: **Note:**
If you have a [mirrored repository where GitLab pulls from](https://docs.gitlab.com/ee/workflow/repository_mirroring.html#pulling-from-a-remote-repository-starter),
If you have a [mirrored repository where GitLab pulls from](https://docs.gitlab.com/ee/workflow/repository_mirroring.html#pulling-from-a-remote-repository-starter),
you may need to enable pipeline triggering in your project's
you may need to enable pipeline triggering in your project's
**Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**.
**Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**.
## Pipelines
### Simple example
A pipeline is a group of [jobs] that get executed in [stages].
As an example, imagine a pipeline consisting of four stages, executed in the following order:
All of the jobs in a stage are executed in parallel (if there are enough
concurrent [Runners]), and if they all succeed, the pipeline moves on to the
next stage. If one of the jobs fails, the next stage is not (usually)
executed. You can access the pipelines page in your project's **Pipelines** tab.
In the following image you can see that the pipeline consists of four stages
-`build`, with a job called `compile`.
(`build`, `test`, `staging`, `production`) each one having one or more jobs.
-`test`, with two jobs called `test` and `test2`.
-`staging`, with a job called `deploy-to-stage`.
-`production`, with a job called `deploy-to-prod`.
>**Note:**
## Visualizing pipelines
GitLab capitalizes the stages' names when shown in the [pipeline graphs](#pipeline-graphs).
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5742) in GitLab 8.11.
## Types of pipelines
Pipelines can be complex structures with many sequential and parallel jobs.
There are three types of pipelines that often use the single shorthand of "pipeline". People often talk about them as if each one is "the" pipeline, but really, they're just pieces of a single, comprehensive pipeline.
To make it easier to understand the flow of a pipeline, GitLab has pipeline graphs for viewing pipeline
and their statuses.
1.**CI Pipeline**: Build and test stages defined in `.gitlab-ci.yml`.
Pipeline graphs can be displayed in two different ways, depending on what page you
1.**Deploy Pipeline**: Deploy stage(s) defined in `.gitlab-ci.yml` The flow of deploying code to servers through various stages: e.g. development to staging to production.
access the graph.
1.**Project Pipeline**: Cross-project CI dependencies [triggered via API][triggers], particularly for micro-services, but also for complicated build dependencies: e.g. api -> front-end, ce/ee -> omnibus.
## Development workflows
NOTE: **Note:**
GitLab capitalizes the stages' names when shown in the [pipeline graphs](#pipeline-graphs).
Pipelines accommodate several development workflows:
### Regular pipeline graphs
1.**Branch Flow** (e.g. different branch for dev, qa, staging, production).
Regular pipeline graphs that show the names of the jobs of each stage. Regular pipeline graphs can
1.**Trunk-based Flow** (e.g. feature branches and single master branch, possibly with tags for releases).
be found when you are on a [single pipeline page](#seeing-pipeline-status). For example:
1.**Fork-based Flow** (e.g. merge requests come from forks).
Example continuous delivery flow:
### Pipeline mini graphs
## Jobs
Pipeline mini graphs takes less space and can give you a
quick glance if all jobs pass or something failed. The pipeline mini graph can
be found when you navigate to:
Jobs can be defined in the [`.gitlab-ci.yml`][jobs-yaml] file. Not to be
- The pipelines index page.
confused with a `build` job or `build` stage.
- A single commit page.
- A merge request page.
## Defining pipelines
Pipeline mini graphs allow you to see all related jobs for a single commit and the net result
of each stage of your pipeline. This allows you to quickly see what failed and
fix it.
Pipelines are defined in `.gitlab-ci.yml` by specifying [jobs] that run in
Stages in pipeline mini graphs are collapsible. Hover your mouse over them and click to expand their jobs.
[stages].
See the reference [documentation for jobs](yaml/README.md#jobs).
|  |  |
## Manually executing pipelines
### Job ordering in pipeline graphs
Pipelines can be manually executed, with predefined or manually-specified [variables](variables/README.md).
Job ordering depends on the type of pipeline graph. For [regular pipeline graphs](#regular-pipeline-graphs), jobs are sorted by name.
To execute a pipeline manually:
For [pipeline mini graphs](#pipeline-mini-graphs)([introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9760)
in GitLab 9.0), jobs are sorted by severity and then by name.
1. Navigate to your project's **CI/CD > Pipelines**.
The order of severity is:
1. Click on the **Run Pipeline** button.
1. Select the branch to run the pipeline for and enter any environment variables required for the pipeline run.
## Seeing pipeline status
- failed
- warning
- pending
- running
- manual
- scheduled
- canceled
- success
- skipped
- created
You can find the current and historical pipeline runs under your project's
For example:
**Pipelines** tab. Clicking on a pipeline will show the jobs that were run for
that pipeline.
## Seeing job status
### How pipeline duration is calculated
When you visit a single pipeline you can see the related jobs for that pipeline.
Total running time for a given pipeline excludes retries and pending
Clicking on an individual job will show you its job trace, and allow you to
(queue) time.
cancel the job, retry it, or erase the job trace.
Each job is represented as a `Period`, which consists of:
## Seeing the failure reason for jobs
-`Period#first` (when the job started).
-`Period#last` (when the job finished).
> [Introduced][ce-17782] in GitLab 10.7.
A simple example is:
When a pipeline fails or is allowed to fail, there are several places where you
- A (1, 3)
can quickly check the reason it failed:
- B (2, 4)
- C (6, 7)
-**In the pipeline graph** present on the pipeline detail view.
In the example:
-**In the pipeline widgets** present in the merge requests and commit pages.
-**In the job views** present in the global and detailed views of a job.
In any case, if you hover over the failed job you can see the reason it failed.
- A begins at 1 and ends at 3.
- B begins at 2 and ends at 4.
- C begins at 6 and ends at 7.
Visually it can be viewed as:
From [GitLab 10.8][ce-17814] you can also see the reason it failed on the Job detail page.
```text
0 1 2 3 4 5 6 7
AAAAAAA
BBBBBBB
CCCC
```
## Pipeline graphs
The union of A, B, and C is (1, 4) and (6, 7). Therefore, the total running time is:
> [Introduced][ce-5742] in GitLab 8.11.
```text
(4 - 1) + (7 - 6) => 4
```
Pipelines can be complex structures with many sequential and parallel jobs.
## Configuring pipelines
To make it a little easier to see what is going on, you can view a graph
of a single pipeline and its status.
A pipeline graph can be shown in two different ways depending on what page you
Pipelines, and their component jobs and stages, are defined in the [`.gitlab-ci.yml`](yaml/README.md) file for each project.
are on.
---
In particular:
The regular pipeline graph that shows the names of the jobs of each stage can
- Jobs are the [basic configuration](yaml/README.html#introduction) component.
be found when you are on a [single pipeline page](#seeing-pipeline-status).
- Stages are defined using the [`stages`](yaml/README.html#stages) keyword.
For all available configuration options, see [GitLab CI/CD Pipeline Configuration Reference](yaml/README.md).
Then, there is the pipeline mini graph which takes less space and can give you a
### Settings and schedules
quick glance if all jobs pass or something failed. The pipeline mini graph can
be found when you visit:
- The pipelines index page.
In addition to configuring jobs through `.gitlab-ci.yml`, additional configuration options are available
- A single commit page.
through the GitLab UI:
- A merge request page.
That way, you can see all related jobs for a single commit and the net result
- Pipeline settings for each project. For more information, see [Pipeline settings](../user/project/pipelines/settings.md).
of each stage of your pipeline. This allows you to quickly see what failed and
- Schedules for pipelines. For more information, see [Pipeline schedules](../user/project/pipelines/schedules.md).
fix it. Stages in pipeline mini graphs are collapsible. Hover your mouse over
them and click to expand their jobs.
| **Mini graph** | **Mini graph expanded** |
### Grouping jobs
| :------------: | :---------------------: |
|  |  |
### Grouping similar jobs in the pipeline graph
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6242) in GitLab 8.12.
> [Introduced][ce-6242] in GitLab 8.12.
If you have many similar jobs, your [pipeline graph](#visualizing-pipelines) becomes long and hard
to read.
If you have many similar jobs, your pipeline graph becomes very long and hard
For that reason, similar jobs can automatically be grouped together.
to read. For that reason, similar jobs can automatically be grouped together.
If the job names are formatted in certain ways, they will be collapsed into
If the job names are formatted in certain ways, they will be collapsed into
a single group in regular pipeline graphs (not the mini graphs).
a single group in regular pipeline graphs (not the mini graphs).
You'll know when a pipeline has grouped jobs if you don't see the retry or
You'll know when a pipeline has grouped jobs if you don't see the retry or
cancel button inside them. Hovering over them will show the number of grouped
cancel button inside them. Hovering over them will show the number of grouped
jobs. Click to expand them.
jobs. Click to expand them.
The basic requirements is that there are two numbers separated with one of
#### Configuring grouping
In the pipeline [configuration file](yaml/README.md), job names must include two numbers separated with one of
the following (you can even use them interchangeably):
the following (you can even use them interchangeably):
- A space (` `)
- A space.
- A slash (`/`)
- A slash (`/`).
- A colon (`:`)
- A colon (`:`).
NOTE: **Note:**
More specifically, it uses [this](https://gitlab.com/gitlab-org/gitlab-ce/blob/2f3dc314f42dbd79813e6251792853bc231e69dd/app/models/commit_status.rb#L99) regular expression: `\d+[\s:\/\\]+\d+\s*`.
>**Note:**
#### How grouping works
More specifically, [it uses][regexp] this regular expression: `\d+[\s:\/\\]+\d+\s*`.
The jobs will be ordered by comparing those two numbers from left to right. You
The jobs will be ordered by comparing those two numbers from left to right. You
usually want the first to be the index and the second the total.
usually want the first to be the index and the second the total.
For example, the following jobs will be grouped under a job named `test`:
For example, the following jobs will be grouped under a job named `test`:
-`test 0 3` => `test`
-`test 0 3`
-`test 1 3` => `test`
-`test 1 3`
-`test 2 3` => `test`
-`test 2 3`
The following jobs will be grouped under a job named `test ruby`:
The following jobs will be grouped under a job named `test ruby`:
-`test 1:2 ruby` => `test ruby`
-`test 1:2 ruby`
-`test 2:2 ruby` => `test ruby`
-`test 2:2 ruby`
The following jobs will be grouped under a job named `test ruby` as well:
The following jobs will be grouped under a job named `test ruby` as well:
-`1/3 test ruby` => `test ruby`
-`1/3 test ruby`
-`2/3 test ruby` => `test ruby`
-`2/3 test ruby`
-`3/3 test ruby` => `test ruby`
-`3/3 test ruby`
### Manual actions from the pipeline graph
### Pipelines for merge requests
> [Introduced][ce-7931] in GitLab 8.15.
GitLab supports configuring pipelines that run only for merge requests. For more information, see
[Pipelines for merge requests](merge_request_pipelines/index.md).
[Manual actions][manual] allow you to require manual interaction before moving
### Badges
forward with a particular job in CI. Your entire pipeline can run automatically,
but the actual [deploy to production][env-manual] will require a click.
You can do this straight from the pipeline graph. Just click on the play button
Pipeline status and test coverage report badges are available and configurable for each project.
to execute that particular job. For example, in the image below, the `production`
stage has a job with a manual action.
For information on adding pipeline badges to project, see [Pipeline badges](../user/project/pipelines/settings.md#pipeline-badges).
### Delay a particular job in the pipeline graph
## Multi-project pipelines **[PREMIUM]**
> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21767) in GitLab 11.4.
Pipelines for different projects can be combined and visualized together.
When you do not want to run a job immediately, you can [delay the job to run after a certain period](yaml/README.md#whendelayed).
For more information, see [Multi-project pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipelines.html).
This is especially useful for timed incremental rollout that new code is rolled out gradually.
For example, if you start rolling out new code and users do not experience trouble, GitLab automatically completes the deployment from 0% to 100%.
Alternatively, if you start rolling out and you noticed that a few users experience trouble with the version,
you can stop the timed incremental rollout by canceling the pipeline, and [rolling](environments.md#rolling-back-changes) it back to the stable version.
