Merge branch 'ci-secrets-docs' into 'master'

CI Secrets documentation

See merge request gitlab-org/gitlab!38552
parents c203f8df c931c16c
......@@ -9,6 +9,12 @@ type: tutorial
This tutorial demonstrates how to authenticate, configure, and read secrets with HashiCorp's Vault from GitLab CI/CD.
NOTE: **Note:**
[GitLab Premium](https://about.gitlab.com/pricing/) supports read access to a
Hashicorp Vault, and enables you to
[use Vault secrets in a CI job](../../secrets/index.md#use-vault-secrets-in-a-ci-job-premium).
To learn more, read [Using external secrets in CI](../../secrets/index.md).
## Requirements
This tutorial assumes you are familiar with GitLab CI/CD and Vault.
......
---
stage: Release
group: Release Management
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
type: concepts, howto
---
# Using external secrets in CI
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218746) in GitLab 13.4 and GitLab Runner 13.4.
Secrets represent sensitive information your CI job needs to complete work. This
sensitive information can be items like API tokens, database credentials, or private keys.
Secrets are sourced from your secrets provider.
Unlike CI variables, which are always presented to a job, secrets must be explicitly
required by a job. Read [GitLab CI/CD pipeline configuration reference](../yaml/README.md#secrets)
for more information about the syntax.
GitLab has selected [Vault by Hashicorp](https://www.vaultproject.io) as the
first supported provider, and [KV-V2](https://www.vaultproject.io/docs/secrets/kv/kv-v2)
as the first supported secrets engine.
GitLab authenticates using Vault's
[JWT Auth method](https://www.vaultproject.io/docs/auth/jwt#jwt-authentication), using
the [JSON Web Token](https://gitlab.com/gitlab-org/gitlab/-/issues/207125) (`CI_JOB_JWT`)
introduced in GitLab 12.10.
You must [configure your Vault server](#configure-your-vault-server) before you
can use [use Vault secrets in a CI job](#use-vault-secrets-in-a-ci-job-premium).
NOTE: **Note:**
Read the [Authenticating and Reading Secrets With Hashicorp Vault](../examples/authenticating-with-hashicorp-vault/index.md)
tutorial for a version of this feature that is available to all
subscription levels, supports writing secrets to and deleting secrets from Vault,
and multiple secrets engines.
## Configure your Vault server
To configure your Vault server:
1. Enable the authentication method by running these commands. They provide your Vault
server the [JSON Web Key Set](https://tools.ietf.org/html/rfc7517) (JWKS) endpoint for your GitLab instance, so Vault
can fetch the public signing key and verify the JSON Web Token (JWT) when authenticating:
```shell
$ vault auth enable jwt
$ vault write auth/jwt/config \
jwks_url="https://gitlab.example.com/-/jwks" \
bound_issuer="gitlab.example.com"
```
1. Configure policies on your Vault server to grant or forbid access to certain
paths and operations. This example grants read access to the set of secrets
required by your production environment:
```shell
vault policy write myproject-production - <<EOF
# Read-only permission on 'ops/data/production/*' path
path "ops/data/production/*" {
capabilities = [ "read" ]
}
EOF
```
1. Configure roles on your Vault server, restricting roles to a project or namespace,
as described in [Configure Vault server roles](#configure-vault-server-roles) on this page.
1. [Create the following CI variables](../variables/README.md#custom-environment-variables)
to provide details about your Vault server:
- `VAULT_SERVER_URL` - The URL of your Vault server, such as `https://vault.example.com:8200`.
Required.
- `VAULT_AUTH_ROLE` - (Optional) The role to use when attempting to authenticate.
If no role is specified, Vault uses the [default role](https://www.vaultproject.io/api/auth/jwt#default_role)
specified when the authentication method was configured.
- `VAULT_AUTH_PATH` - (Optional) The path where the authentication method is mounted, default is `jwt`.
NOTE: **Note:**
Support for [providing these values in the user interface](https://gitlab.com/gitlab-org/gitlab/-/issues/218677)
is planned but not yet implemented.
## Use Vault secrets in a CI job **(PREMIUM)**
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/28321) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4 and GitLab Runner 13.4.
After [configuring your Vault server](#configure-your-vault-server), you can use
the secrets stored in Vault by defining them with the `vault` keyword:
```yaml
secrets:
DATABASE_PASSWORD:
vault: production/db/password@ops # translates to secret `ops/data/production/db`, field `password`
```
In this example:
- `production/db` - The secret.
- `password` The field.
- `ops` - The path where the secrets engine is mounted.
After GitLab fetches the secret from Vault, the value is saved in a temporary file.
The path to this file is stored in environment variable named `DATABASE_PASSWORD`,
similar to [CI variables of type `file`](../variables/README.md#custom-environment-variables-of-type-file).
For more information about the supported syntax, read the
[`.gitlab-ci.yml` reference](../yaml/README.md#secretsvault-premium).
## Configure Vault server roles
When a CI job attempts to authenticate, it specifies a role. You can use roles to group
different policies together. If authentication is successful, these policies are
attached to the resulting Vault token.
[Bound claims](https://www.vaultproject.io/docs/auth/jwt#bound-claims) are predefined
values that are matched to the JWT's claims. With bounded claims, you can restrict access
to specific GitLab users, specific projects, or even jobs running for specific Git
references. You can have as many bounded claims you need, but they must *all* match
for authentication to be successful.
Combining bounded claims with GitLab features like [user roles](../../user/permissions.md)
and [protected branches](../../user/project/protected_branches.md), you can tailor
these rules to fit your specific use case. In this example, authentication is allowed
only for jobs running for protected tags with names matching the pattern used for
production releases:
```shell
$ vault write auth/jwt/role/myproject-production - <<EOF
{
"role_type": "jwt",
"policies": ["myproject-production"],
"token_explicit_max_ttl": 60,
"user_claim": "user_email",
"bound_claims_type": "glob",
"bound_claims": {
"project_id": "42",
"ref_protected": "true",
"ref_type": "tag",
"ref": "auto-deploy-*"
}
}
EOF
```
CAUTION: **Caution:**
Always restrict your roles to a project or namespace by using one of the provided
claims like `project_id` or `namespace_id`. Without these restrictions, any JWT
generated by this GitLab instance may be allowed to authenticate using this role.
For a full list of `CI_JOB_JWT` claims, read the
[How it works](../examples/authenticating-with-hashicorp-vault/index.md#how-it-works) section of the
[Authenticating and Reading Secrets With Hashicorp Vault](../examples/authenticating-with-hashicorp-vault/index.md) tutorial.
You can also specify some attributes for the resulting Vault tokens, such as time-to-live,
IP address range, and number of uses. The full list of options is available in
[Vault's documentation on creating roles](https://www.vaultproject.io/api/auth/jwt#create-role)
for the JSON web token method.
......@@ -431,7 +431,7 @@ script:
You can define per-project or per-group variables
that are set in the pipeline environment. Group-level variables are stored out of
the repository (not in `.gitlab-ci.yml`) and are securely passed to GitLab Runner,
which makes them available during a pipeline run. For Premium users who do **not** use an external key store or who use GitLab's [integration with HashiCorp Vault](../examples/authenticating-with-hashicorp-vault/index.md), we recommend using group environment variables to store secrets like passwords, SSH keys, and credentials.
which makes them available during a pipeline run. For Premium users who do **not** use an external key store or who use GitLab's [integration with HashiCorp Vault](../secrets/index.md), we recommend using group environment variables to store secrets like passwords, SSH keys, and credentials.
Group-level variables can be added by:
......
......@@ -70,7 +70,7 @@ Kubernetes-specific environment variables are detailed in the
| `CI_JOB_NAME` | 9.0 | 0.5 | The name of the job as defined in `.gitlab-ci.yml` |
| `CI_JOB_STAGE` | 9.0 | 0.5 | The name of the stage as defined in `.gitlab-ci.yml` |
| `CI_JOB_TOKEN` | 9.0 | 1.2 | Token used for authenticating with the [GitLab Container Registry](../../user/packages/container_registry/index.md), downloading [dependent repositories](../../user/project/new_ci_build_permissions_model.md#dependent-repositories), and accessing [GitLab-managed Terraform state](../../user/infrastructure/index.md#gitlab-managed-terraform-state). |
| `CI_JOB_JWT` | 12.10 | all | RS256 JSON web token that can be used for authenticating with third party systems that support JWT authentication, for example [HashiCorp's Vault](../examples/authenticating-with-hashicorp-vault). |
| `CI_JOB_JWT` | 12.10 | all | RS256 JSON web token that can be used for authenticating with third party systems that support JWT authentication, for example [HashiCorp's Vault](../secrets/index.md). |
| `CI_JOB_URL` | 11.1 | 0.5 | Job details URL |
| `CI_KUBERNETES_ACTIVE` | 13.0 | all | Included with the value `true` only if the pipeline has a Kubernetes cluster available for deployments. Not included if no cluster is available. Can be used as an alternative to [`only:kubernetes`/`except:kubernetes`](../yaml/README.md#onlykubernetesexceptkubernetes) with [`rules:if`](../yaml/README.md#rulesif) |
| `CI_MERGE_REQUEST_ASSIGNEES` | 11.9 | all | Comma-separated list of username(s) of assignee(s) for the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. |
......
......@@ -4042,6 +4042,53 @@ The YAML described above would be translated into a CLI command like this:
release-cli create --name "Release $CI_COMMIT_SHA" --description "Created using the release-cli $EXTRA_DESCRIPTION" --tag-name "v${MAJOR}.${MINOR}.${REVISION}" --ref "$CI_COMMIT_SHA" --released-at "2020-07-15T08:00:00Z" --milestone "m1" --milestone "m2" --milestone "m3"
```
### `secrets`
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/33014) in GitLab 13.4.
`secrets` indicates the [CI Secrets](../secrets/index.md) this job needs. It should be a hash,
and the keys should be the names of the environment variables the job needs to access the secrets.
#### `secrets:vault` **(PREMIUM)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/28321) in GitLab 13.4.
`vault` keyword specifies secrets provided by [Hashicorp's Vault](https://www.vaultproject.io/).
This syntax has multiple forms. The shortest form asssumes the use of the
[KV-V2](https://www.vaultproject.io/docs/secrets/kv/kv-v2) secrets engine,
mounted at the default path `kv-v2`. The last part of the secret's path is the
field to fetch the value for:
```yaml
job:
secrets:
DATABASE_PASSWORD:
vault: production/db/password # translates to secret `kv-v2/data/production/db`, field `password`
```
You can specify a custom secrets engine path by adding a suffix starting with `@`:
```yaml
job:
secrets:
DATABASE_PASSWORD:
vault: production/db/password@ops # translates to secret `ops/data/production/db`, field `password`
```
In the detailed form of the syntax, you can specify all details explicitly:
```yaml
job:
secrets:
DATABASE_PASSWORD: # translates to secret `ops/data/production/db`, field `password`
vault:
engine:
name: kv-v2
path: ops
path: production/db
field: password
```
### `pages`
`pages` is a special job that is used to upload static content to GitLab that
......
......@@ -172,4 +172,4 @@ CAUTION: **Important:**
Make sure to never commit the `auth.json` file to your repository. To install packages from a CI job,
consider using the [`composer config`](https://getcomposer.org/doc/articles/handling-private-packages-with-satis.md#authentication) tool with your personal access token
stored in a [GitLab CI/CD environment variable](../../../ci/variables/README.md) or in
[Hashicorp Vault](../../../ci/examples/authenticating-with-hashicorp-vault/index.md).
[Hashicorp Vault](../../../ci/secrets/index.md).
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment