Skip to content

G
gitlab-ce
Commit 770504ba authored by Tomasz Maczukin's avatar Tomasz Maczukin Committed by Achilleas Pipinellis
Browse files
Options

Add documentation about variables usage in GitLab CI

parent a30e6b81
Hide whitespace changes
Inline Side-by-side
Showing with 135 additions and 52 deletions
doc/ci/README.md
View file @ 770504ba
......@@ -19,7 +19,7 @@ Here's some info we've gathered to get you started.
The first steps towards your GitLab CI/CD journey.
- [Getting started with GitLab CI/CD](quick_start/README.md): understand how GitLab CI/CD works.
- GitLab CI/CD configuration file: [`.gitlab-ci.yml`](yaml/README.md) - Learn all about the ins and outs of `.gitlab-ci.yml`.
- [GitLab CI/CD configuration file: `.gitlab-ci.yml`](yaml/README.md) - Learn all about the ins and outs of `.gitlab-ci.yml`.
- [Pipelines and jobs](pipelines.md): configure your GitLab CI/CD pipelines to build, test, and deploy your application.
- Runners: The [GitLab Runner](https://docs.gitlab.com/runner/) is responsible by running the jobs in your CI/CD pipeline. On GitLab.com, Shared Runners are enabled by default, so
you don't need to set up anything to start to use them with GitLab CI/CD.
......@@ -46,7 +46,9 @@ you don't need to set up anything to start to use them with GitLab CI/CD.
## Exploring GitLab CI/CD
- [CI/CD Variables](variables/README.md) - Learn how to use variables defined in
your `.gitlab-ci.yml` or secured ones defined in your project's settings
your `.gitlab-ci.yml` or the ones defined in your project's settings
- [Where variables can be used](variables/where_variables_can_be_used.md) - A
deeper look on where and how the CI/CD variables can be used
- **The permissions model** - Learn about the access levels a user can have for
performing certain CI actions
- [User permissions](../user/permissions.md#gitlab-ci)
......
doc/ci/environments.md
View file @ 770504ba
......@@ -246,23 +246,14 @@ As the name suggests, it is possible to create environments on the fly by just
declaring their names dynamically in `.gitlab-ci.yml`. Dynamic environments is
the basis of [Review apps](review_apps/index.md).
>**Note:**
The `name` and `url` parameters can use most of the defined CI variables,
including predefined, secure variables and `.gitlab-ci.yml`
[`variables`](yaml/README.md#variables). You however cannot use variables
defined under `script` or on the Runner's side. There are other variables that
are unsupported in environment name context:
- `CI_PIPELINE_ID`
- `CI_JOB_ID`
- `CI_JOB_TOKEN`
- `CI_BUILD_ID`
- `CI_BUILD_TOKEN`
- `CI_REGISTRY_USER`
- `CI_REGISTRY_PASSWORD`
- `CI_REPOSITORY_URL`
- `CI_ENVIRONMENT_URL`
- `CI_DEPLOY_USER`
- `CI_DEPLOY_PASSWORD`
NOTE: **Note:**
The `name` and `url` parameters can use most of the CI/CD variables,
including [predefined](variables/README.md#predefined-variables-environment-variables),
[secret](variables/README.md#secret-variables) and
[`.gitlab-ci.yml` variables](yaml/README.md#variables). You however cannot use variables
defined under `script` or on the Runner's side. There are also other variables that
are unsupported in the context of `environment:name`. You can read more about
[where variables can be used](variables/where_variables_can_be_used.md).
GitLab Runner exposes various [environment variables][variables] when a job runs,
and as such, you can use them as environment names. Let's add another job in
......
doc/ci/variables/README.md
View file @ 770504ba
......@@ -22,6 +22,12 @@ For example, if you define `API_TOKEN=secure` as a secret variable and
`API_TOKEN=yaml` in your `.gitlab-ci.yml`, the `API_TOKEN` will take the value
`secure` as the secret variables are higher in the chain.
## Unsupported variables
There are cases where some variables cannot be used in the context of a
`.gitlab-ci.yml` definition (for example under `script`). Read more
about which variables are [not supported](where_variables_can_be_used.md).
## Predefined variables (Environment variables)
Some of the predefined environment variables are available only if a minimum
......@@ -36,6 +42,7 @@ future GitLab releases.**
| Variable | GitLab | Runner | Description |
|-------------------------------- |--------|--------|-------------|
| **ARTIFACT_DOWNLOAD_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to download artifacts running a job |
| **CI** | all | 0.4 | Mark that job is executed in CI environment |
| **CI_COMMIT_REF_NAME** | 9.0 | all | The branch or tag name for which project is built |
| **CI_COMMIT_REF_SLUG** | 9.0 | all | `$CI_COMMIT_REF_NAME` lowercased, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. No leading / trailing `-`. Use in URLs, host names and domain names. |
......@@ -46,6 +53,8 @@ future GitLab releases.**
| **CI_COMMIT_DESCRIPTION** | 10.8 | all | The description of the commit: the message without first line, if the title is shorter than 100 characters; full message in other case. |
| **CI_CONFIG_PATH** | 9.4 | 0.5 | The path to CI config file. Defaults to `.gitlab-ci.yml` |
| **CI_DEBUG_TRACE** | all | 1.7 | Whether [debug tracing](#debug-tracing) is enabled |
| **CI_DEPLOY_USER** | 10.8 | all | Authentication username of the [GitLab Deploy Token][gitlab-deploy-token], only present if the Project has one related.|
| **CI_DEPLOY_PASSWORD** | 10.8 | all | Authentication password of the [GitLab Deploy Token][gitlab-deploy-token], only present if the Project has one related.|
| **CI_DISPOSABLE_ENVIRONMENT** | all | 10.1 | Marks that the job is executed in a disposable environment (something that is created only for this job and disposed of/destroyed after the execution - all executors except `shell` and `ssh`). If the environment is disposable, it is set to true, otherwise it is not defined at all. |
| **CI_ENVIRONMENT_NAME** | 8.15 | all | The name of the environment for this job |
| **CI_ENVIRONMENT_SLUG** | 8.15 | all | A simplified version of the environment name, suitable for inclusion in DNS, URLs, Kubernetes labels, etc. |
......@@ -82,16 +91,13 @@ future GitLab releases.**
| **CI_SERVER_REVISION** | all | all | GitLab revision that is used to schedule jobs |
| **CI_SERVER_VERSION** | all | all | GitLab version that is used to schedule jobs |
| **CI_SHARED_ENVIRONMENT** | all | 10.1 | Marks that the job is executed in a shared environment (something that is persisted across CI invocations like `shell` or `ssh` executor). If the environment is shared, it is set to true, otherwise it is not defined at all. |
| **ARTIFACT_DOWNLOAD_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to download artifacts running a job |
| **GET_SOURCES_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to fetch sources running a job |
| **GITLAB_CI** | all | all | Mark that job is executed in GitLab CI environment |
| **GITLAB_USER_ID** | 8.12 | all | The id of the user who started the job |
| **GITLAB_USER_EMAIL** | 8.12 | all | The email of the user who started the job |
| **GITLAB_USER_ID** | 8.12 | all | The id of the user who started the job |
| **GITLAB_USER_LOGIN** | 10.0 | all | The login username of the user who started the job |
| **GITLAB_USER_NAME** | 10.0 | all | The real name of the user who started the job |
| **RESTORE_CACHE_ATTEMPTS** | 8.15 | 1.9 | Number of attempts to restore the cache running a job |
| **CI_DEPLOY_USER** | 10.8 | all | Authentication username of the [GitLab Deploy Token][gitlab-deploy-token], only present if the Project has one related.|
| **CI_DEPLOY_PASSWORD** | 10.8 | all | Authentication password of the [GitLab Deploy Token][gitlab-deploy-token], only present if the Project has one related.|
## 9.0 Renaming
......@@ -540,34 +546,6 @@ Below you can find supported syntax reference:
Pattern matching is case-sensitive by default. Use `i` flag modifier, like
`/pattern/i` to make a pattern case-insensitive.
### Unsupported predefined variables
Because GitLab evaluates variables before creating jobs, we do not support a
few variables that depend on persistence layer, like `$CI_JOB_ID`.
Environments (like `production` or `staging`) are also being created based on
what jobs pipeline consists of, thus some environment-specific variables are
not supported as well.
We do not support variables containing tokens because of security reasons.
You can find a full list of unsupported variables below:
- `CI_PIPELINE_ID`
- `CI_JOB_ID`
- `CI_JOB_TOKEN`
- `CI_BUILD_ID`
- `CI_BUILD_TOKEN`
- `CI_REGISTRY_USER`
- `CI_REGISTRY_PASSWORD`
- `CI_REPOSITORY_URL`
- `CI_ENVIRONMENT_URL`
- `CI_DEPLOY_USER`
- `CI_DEPLOY_PASSWORD`
These variables are also not supported in a context of a
[dynamic environment name][dynamic-environments].
[ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 "Simple protection of CI secret variables"
[eep]: https://about.gitlab.com/products/ "Available only in GitLab Premium"
[envs]: ../environments.md
......@@ -579,5 +557,4 @@ These variables are also not supported in a context of a
[triggers]: ../triggers/README.md#pass-job-variables-to-a-trigger
[subgroups]: ../../user/group/subgroups/index.md
[builds-policies]: ../yaml/README.md#only-and-except-complex
[dynamic-environments]: ../environments.md#dynamic-environments
[gitlab-deploy-token]: ../../user/project/deploy_tokens/index.md#gitlab-deploy-token
doc/ci/variables/where_variables_can_be_used.md 0 → 100644
View file @ 770504ba
# Where variables can be used
As it's described in the [CI/CD variables](README.md) docs, you can
define many different variables. Some of them can be used for all GitLab CI/CD
features, but some of them are more or less limited.
This document describes where and how the different types of variables can be used.
## Variables usage
There are basically two places where you can use any defined variables:
1. On GitLab's side there's `.gitlab-ci.yml`
1. On the Runner's side there's `config.toml`
### `.gitlab-ci.yml` file
| Definition | Can be expanded? | Expansion place | Description |
|--------------------------------------|-------------------|-----------------|--------------|
| `environment:url` | yes | GitLab | The variable expansion is made by GitLab's [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism).<ul><li>**Supported:** all variables defined for a job (secret variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules)</li><li>**Not suported:** variables defined in Runner's `config.toml` and variables created in job's `script`</li></ul> |
| `environment:name` | yes | GitLab | Similar to `environment:url`, but the variables expansion **doesn't support**: <ul><li>variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`)</li><li>any other variables related to environment (currently only `CI_ENVIRONMENT_URL`)</li><li>[persisted variables](#persisted-variables)</li></ul> |
| `variables` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
| `image` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
| `services:[]` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
| `services:[]:name` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
| `cache:key` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
| `artifacts:name` | yes | Runner | The variable expansion is made by GitLab Runner's shell environment |
| `script`, `before_script`, `after_script` | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment) |
| `only:variables:[]`, `except:variables:[]` | no | n/a | The variable must be in the form of `$variable`.<br/>**Not supported:**<ul><li>variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`)</li><li>any other variables related to environment (currently only `CI_ENVIRONMENT_URL`)</li><li>[persisted variables](#persisted-variables)</li></ul> |
### `config.toml` file
NOTE: **Note:**
You can read more about `config.toml` in the [Runner's docs](https://docs.gitlab.com/runner/configuration/advanced-configuration.html).
| Definition | Can be expanded? | Description |
|--------------------------------------|------------------|-------------|
| `runners.environment` | yes | The variable expansion is made by the Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
| `runners.kubernetes.pod_labels` | yes | The Variable expansion is made by the Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
| `runners.kubernetes.pod_annotations` | yes | The Variable expansion is made by the Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism) |
## Expansion mechanisms
There are three expansion mechanisms:
- GitLab
- GitLab Runner
- Execution shell environment
### GitLab internal variable expansion mechanism
The expanded part needs to be in a form of `$variable`, or `${variable}` or `%variable%`.
Each form is handled in the same way, no matter which OS/shell will finally handle the job,
since the expansion is done in GitLab before any Runner will get the job.
### GitLab Runner internal variable expansion mechanism
- **Supported:** secret variables, `.gitlab-ci.yml` variables, `config.toml` variables, and
variables from triggers and pipeline schedules
- **Not supported:** variables defined inside of scripts (e.g., `export MY_VARIABLE="test"`)
The Runner uses Go's `os.Expand()` method for variable expansion. It means that it will handle
only variables defined as `$variable` and `${variable}`. What's also important, is that
the expansion is done only once, so nested variables may or may not work, depending on the
ordering of variables definitions.
### Execution shell environment
This is an expansion that takes place during the `script` execution.
How it works depends on the used shell (bash/sh/cmd/PowerShell). For example, if the job's
`script` contains a line `echo $MY_VARIABLE-${MY_VARIABLE_2}`, it should be properly handled
by bash/sh (leaving empty strings or some values depending whether the variables were
defined or not), but will not work with Windows' cmd/PowerShell, since these shells
are using a different variables syntax.
**Supported:**
- The `script` may use all available variables that are default for the shell (e.g., `$PATH` which
should be present in all bash/sh shells) and all variables defined by GitLab CI/CD (secret variables,
`.gitlab-ci.yml` variables, `config.toml` variables, and variables from triggers and pipeline schedules).
- The `script` may also use all variables defined in the lines before. So, for example, if you define
a variable `export MY_VARIABLE="test"`:
- in `before_script`, it will work in the following lines of `before_script` and
all lines of the related `script`
- in `script`, it will work in the following lines of `script`
- in `after_script`, it will work in following lines of `after_script`
## Persisted variables
NOTE: **Note:**
Some of the persisted variables contain tokens and cannot be used by some definitions
due to security reasons.
The following variables are known as "persisted":
- `CI_PIPELINE_ID`
- `CI_JOB_ID`
- `CI_JOB_TOKEN`
- `CI_BUILD_ID`
- `CI_BUILD_TOKEN`
- `CI_REGISTRY_USER`
- `CI_REGISTRY_PASSWORD`
- `CI_REPOSITORY_URL`
- `CI_DEPLOY_USER`
- `CI_DEPLOY_PASSWORD`
They are:
- **supported** for all definitions as [described in the table](#gitlab-ci-yml-file) where the "Expansion place" is "Runner"
- **not supported:**
- by the definitions [described in the table](#gitlab-ci-yml-file) where the "Expansion place" is "GitLab"
- in the `only` and `except` [variables expressions](README.md#variables-expressions)
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7