Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
1
Merge Requests
1
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
nexedi
gitlab-ce
Commits
04afd852
Commit
04afd852
authored
Nov 05, 2021
by
Suzanne Selhorn
Committed by
Marcel Amirault
Nov 05, 2021
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Updating parallel keyword to match standards
parent
7ada6239
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
130 additions
and
103 deletions
+130
-103
doc/ci/jobs/job_control.md
doc/ci/jobs/job_control.md
+100
-0
doc/ci/yaml/index.md
doc/ci/yaml/index.md
+29
-102
doc/development/ruby_upgrade.md
doc/development/ruby_upgrade.md
+1
-1
No files found.
doc/ci/jobs/job_control.md
View file @
04afd852
...
...
@@ -640,6 +640,106 @@ This job can no longer be scheduled to run automatically. You can, however, exec
To start a delayed job immediately, select
**Play**
(
**{play}**
).
Soon GitLab Runner starts the job.
## Parallelize large jobs
To split a large job into multiple smaller jobs that run in parallel, use the
[
`parallel`
](
../yaml/index.md#parallel
)
keyword in your
`.gitlab-ci.yml`
file.
Different languages and test suites have different methods to enable parallelization.
For example, use
[
Semaphore Test Boosters
](
https://github.com/renderedtext/test-boosters
)
and RSpec to run Ruby tests in parallel:
```
ruby
# Gemfile
source
'https://rubygems.org'
gem
'rspec'
gem
'semaphore_test_boosters'
```
```
yaml
test
:
parallel
:
3
script
:
-
bundle
-
bundle exec rspec_booster --job $CI_NODE_INDEX/$CI_NODE_TOTAL
```
You can then navigate to the
**Jobs**
tab of a new pipeline build and see your RSpec
job split into three separate jobs.
WARNING:
Test Boosters reports usage statistics to the author.
### Run a one-dimensional matrix of parallel jobs
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26362) in GitLab 13.5.
You can create a one-dimensional matrix of parallel jobs:
```
yaml
deploystacks
:
stage
:
deploy
script
:
-
bin/deploy
parallel
:
matrix
:
-
PROVIDER
:
[
aws
,
ovh
,
gcp
,
vultr
]
```
You can also
[
create a multi-dimensional matrix
](
../yaml/index.md#parallelmatrix
)
.
### Run a matrix of parallel trigger jobs
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/270957) in GitLab 13.10.
You can run a
[
trigger
](
../yaml/index.md#trigger
)
job multiple times in parallel in a single pipeline,
but with different variable values for each instance of the job.
```
yaml
deploystacks
:
stage
:
deploy
trigger
:
include
:
path/to/child-pipeline.yml
parallel
:
matrix
:
-
PROVIDER
:
aws
STACK
:
[
monitoring
,
app1
]
-
PROVIDER
:
ovh
STACK
:
[
monitoring
,
backup
]
-
PROVIDER
:
[
gcp
,
vultr
]
STACK
:
[
data
]
```
This example generates 6 parallel
`deploystacks`
trigger jobs, each with different values
for
`PROVIDER`
and
`STACK`
, and they create 6 different child pipelines with those variables.
```
plaintext
deploystacks: [aws, monitoring]
deploystacks: [aws, app1]
deploystacks: [ovh, monitoring]
deploystacks: [ovh, backup]
deploystacks: [gcp, data]
deploystacks: [vultr, data]
```
In
[
GitLab 14.1 and later
](
https://gitlab.com/gitlab-org/gitlab/-/issues/239737
)
, you can
use the variables defined in
`parallel: matrix`
with the
[
`tags`
](
../yaml/index.md#tags
)
keyword for
dynamic runner selection.
```
yaml
deploystacks
:
stage
:
deploy
parallel
:
matrix
:
-
PROVIDER
:
aws
STACK
:
[
monitoring
,
app1
]
-
PROVIDER
:
gcp
STACK
:
[
data
]
tags
:
-
${PROVIDER}-${STACK}
```
## Use predefined CI/CD variables to run jobs only in specific pipeline types
You can use
[
predefined CI/CD variables
](
../variables/predefined_variables.md
)
to choose
...
...
doc/ci/yaml/index.md
View file @
04afd852
...
...
@@ -3679,11 +3679,17 @@ test:
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/21480) in GitLab 11.5.
Use
`parallel`
to configure how many instances of a job to run in parallel.
The value can be from 2 to 50.
Use
`parallel`
to run a job multiple times in parallel in a single pipeline.
The
`parallel`
keyword creates N instances of the same job that run in parallel.
They are named sequentially from
`job_name 1/N`
to
`job_name N/N`
:
Multiple runners must exist, or a single runner must be configured to run multiple jobs concurrently.
Parallel jobs are named sequentially from
`job_name 1/N`
to
`job_name N/N`
.
**Keyword type**
: Job keyword. You can use it only as part of a job.
**Possible inputs**
: A numeric value from
`2`
to
`50`
.
**Example of `parallel`**
:
```
yaml
test
:
...
...
@@ -3691,47 +3697,32 @@ test:
parallel
:
5
```
Every parallel job has a
`CI_NODE_INDEX`
and
`CI_NODE_TOTAL`
[
predefined CI/CD variable
](
../variables/index.md#predefined-cicd-variables
)
set.
This example creates 5 jobs that run in parallel, named
`test 1/5`
to
`test 5/5`
.
Different languages and test suites have different methods to enable parallelization.
For example, use
[
Semaphore Test Boosters
](
https://github.com/renderedtext/test-boosters
)
and RSpec to run Ruby tests in parallel:
```
ruby
# Gemfile
source
'https://rubygems.org'
gem
'rspec'
gem
'semaphore_test_boosters'
```
**Additional details**
:
```
yaml
test
:
parallel
:
3
script
:
-
bundle
-
bundle exec rspec_booster --job $CI_NODE_INDEX/$CI_NODE_TOTAL
```
-
Every parallel job has a
`CI_NODE_INDEX`
and
`CI_NODE_TOTAL`
[
predefined CI/CD variable
](
../variables/index.md#predefined-cicd-variables
)
set.
WARNING:
Test Boosters reports usage statistics to the author.
**Related topics**
:
You can then navigate to the
**Jobs**
tab of a new pipeline build and see your RSpec
job split into three separate jobs.
-
[
Parallelize large jobs
](
../jobs/job_control.md#parallelize-large-jobs
)
.
####
Parallel `matrix` jobs
####
`parallel:matrix`
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15356) in GitLab 13.3.
> - The job naming style was [improved in GitLab 13.4](https://gitlab.com/gitlab-org/gitlab/-/issues/230452).
Use
`
matrix:
`
to run a job multiple times in parallel in a single pipeline,
Use
`
parallel:matrix
`
to run a job multiple times in parallel in a single pipeline,
but with different variable values for each instance of the job.
There can be from 2 to 50 jobs.
Jobs can only run in parallel if there are multiple runners, or a single runner is
configured to run multiple jobs concurrently.
Multiple runners must exist, or a single runner must be configured to run multiple jobs concurrently.
**Keyword type**
: Job keyword. You can use it only as part of a job.
**Possible inputs**
: A numeric value from
`2`
to
`50`
.
Every job gets the same
`CI_NODE_TOTAL`
[
CI/CD variable
](
../variables/index.md#predefined-cicd-variables
)
value, and a unique
`CI_NODE_INDEX`
value.
**Example of `parallel:matrix`**
:
```
yaml
deploystacks
:
...
...
@@ -3751,7 +3742,7 @@ deploystacks:
STACK
:
[
data
,
processing
]
```
The
following
example generates 10 parallel
`deploystacks`
jobs, each with different values
The example generates 10 parallel
`deploystacks`
jobs, each with different values
for
`PROVIDER`
and
`STACK`
:
```
plaintext
...
...
@@ -3767,74 +3758,10 @@ deploystacks: [vultr, data]
deploystacks: [vultr, processing]
```
The job naming style was
[
improved in GitLab 13.4
](
https://gitlab.com/gitlab-org/gitlab/-/issues/230452
)
.
##### One-dimensional `matrix` jobs
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26362) in GitLab 13.5.
You can also have one-dimensional matrices with a single job:
```
yaml
deploystacks
:
stage
:
deploy
script
:
-
bin/deploy
parallel
:
matrix
:
-
PROVIDER
:
[
aws
,
ovh
,
gcp
,
vultr
]
```
##### Parallel `matrix` trigger jobs
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/270957) in GitLab 13.10.
Use
`matrix:`
to run a
[
trigger
](
#trigger
)
job multiple times in parallel in a single pipeline,
but with different variable values for each instance of the job.
```
yaml
deploystacks
:
stage
:
deploy
trigger
:
include
:
path/to/child-pipeline.yml
parallel
:
matrix
:
-
PROVIDER
:
aws
STACK
:
[
monitoring
,
app1
]
-
PROVIDER
:
ovh
STACK
:
[
monitoring
,
backup
]
-
PROVIDER
:
[
gcp
,
vultr
]
STACK
:
[
data
]
```
This example generates 6 parallel
`deploystacks`
trigger jobs, each with different values
for
`PROVIDER`
and
`STACK`
, and they create 6 different child pipelines with those variables.
```
plaintext
deploystacks: [aws, monitoring]
deploystacks: [aws, app1]
deploystacks: [ovh, monitoring]
deploystacks: [ovh, backup]
deploystacks: [gcp, data]
deploystacks: [vultr, data]
```
In
[
GitLab 14.1 and later
](
https://gitlab.com/gitlab-org/gitlab/-/issues/239737
)
, you can
use the variables defined in
`parallel: matrix`
with the
[
`tags`
](
#tags
)
keyword for
dynamic runner selection.
**Related topics**
:
```
yaml
deploystacks
:
stage
:
deploy
parallel
:
matrix
:
-
PROVIDER
:
aws
STACK
:
[
monitoring
,
app1
]
-
PROVIDER
:
gcp
STACK
:
[
data
]
tags
:
-
${PROVIDER}-${STACK}
```
-
[
Run a one-dimensional matrix of parallel jobs
](
../jobs/job_control.md#run-a-one-dimensional-matrix-of-parallel-jobs
)
.
-
[
Run a matrix of triggered parallel jobs
](
../jobs/job_control.md#run-a-matrix-of-parallel-trigger-jobs
)
.
### `trigger`
...
...
doc/development/ruby_upgrade.md
View file @
04afd852
...
...
@@ -137,7 +137,7 @@ This is typically necessary, since gems or Ruby applications that we maintain ou
update these repositories for the GitLab Rails application to work with a new Ruby,
it is good practice to keep Ruby versions in lock-step across all our repositories. For minor and major
upgrades, add new CI/CD jobs to these repositories using the new Ruby.
A
[
build matrix definition
](
../ci/yaml/index.md#parallel
-matrix-jobs
)
can do this efficiently.
A
[
build matrix definition
](
../ci/yaml/index.md#parallel
matrix
)
can do this efficiently.
#### Decide which repositories to update
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment