| [Compliance Dashboard](user/compliance/compliance_dashboard/index.md)**(ULTIMATE)** | View the most recent Merge Request activity in a group. |
| [Container Scanning](user/application_security/container_scanning/index.md)**(ULTIMATE)** | Use Clair to scan docker images for known vulnerabilities. |
| [Container Scanning](user/application_security/container_scanning/index.md)**(ULTIMATE)** | Use Clair to scan Docker images for known vulnerabilities. |
| [Dependency List](user/application_security/dependency_list/index.md)**(ULTIMATE)** | View your project's dependencies and their known vulnerabilities. |
| [Dependency Scanning](user/application_security/dependency_scanning/index.md)**(ULTIMATE)** | Analyze your dependencies for known vulnerabilities. |
| [Dynamic Application Security Testing (DAST)](user/application_security/dast/index.md)**(ULTIMATE)** | Analyze running web applications for known vulnerabilities. |
It's possible to preconfigure the GitLab docker image by adding the environment
It's possible to preconfigure the GitLab Docker image by adding the environment
variable `GITLAB_OMNIBUS_CONFIG` to the `docker run` command.
For more information see the ['preconfigure-docker-container' section in the Omnibus documentation](https://docs.gitlab.com/omnibus/docker/#preconfigure-docker-container).
For more information see the [Pre-configure Docker container](https://docs.gitlab.com/omnibus/docker/#pre-configure-docker-container)
GitLab CI/CD allows you to use Docker Engine to build and test docker-based projects.
GitLab CI/CD allows you to use Docker Engine to build and test Docker-based projects.
One of the new trends in Continuous Integration/Deployment is to:
...
...
@@ -91,15 +91,15 @@ NOTE: **Note:**
By adding `gitlab-runner` to the `docker` group you are effectively granting `gitlab-runner` full root permissions.
For more information please read [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful).
### Use docker-in-docker workflow with Docker executor
### Use Docker-in-Docker workflow with Docker executor
The second approach is to use the special docker-in-docker (dind)
The second approach is to use the special Docker-in-Docker (dind)
[Docker image](https://hub.docker.com/_/docker/) with all tools installed
(`docker`) and run the job script in context of that
image in privileged mode.
NOTE: **Note:**
`docker-compose` is not part of docker-in-docker (dind). To use `docker-compose` in your
`docker-compose` is not part of Docker-in-Docker (dind). To use `docker-compose` in your
@@ -74,7 +74,7 @@ And this is basically all you need in the `before_script` section.
## How to deploy
As we stated above, we need to deploy the `build` folder from the docker image to our server. To do so, we create a new job:
As we stated above, we need to deploy the `build` folder from the Docker image to our server. To do so, we create a new job:
```yaml
stage_deploy:
...
...
@@ -94,7 +94,7 @@ stage_deploy:
Here's the breakdown:
1.`only:dev` means that this build will run only when something is pushed to the `dev` branch. You can remove this block completely and have everything be ran on every push (but probably this is something you don't want)
1.`ssh-add ...` we will add that private key you added on the web UI to the docker container
1.`ssh-add ...` we will add that private key you added on the web UI to the Docker container
1. We will connect via `ssh` and create a new `_tmp` folder
1. We will connect via `scp` and upload the `build` folder (which was generated by a `npm` script) to our previously created `_tmp` folder
1. We will connect again via `ssh` and move the `live` folder to an `_old` folder, then move `_tmp` to `live`.
| [`script`](#script) | Shell script which is executed by Runner. |
| [`image`](#image) | Use docker images. Also available: `image:name` and `image:entrypoint`. |
| [`services`](#services) | Use docker services images. Also available: `services:name`, `services:alias`, `services:entrypoint`, and `services:command`. |
| [`image`](#image) | Use Docker images. Also available: `image:name` and `image:entrypoint`. |
| [`services`](#services) | Use Docker services images. Also available: `services:name`, `services:alias`, `services:entrypoint`, and `services:command`. |
| [`before_script`](#before_script-and-after_script) | Override a set of commands that are executed before job. |
| [`after_script`](#before_script-and-after_script) | Override a set of commands that are executed after job. |
| [`stage`](#stage) | Defines a job stage (default: `test`). |
...
...
@@ -521,13 +521,13 @@ For:
#### `image:name`
An [extended docker configuration option](../docker/using_docker_images.md#extended-docker-configuration-options).
An [extended Docker configuration option](../docker/using_docker_images.md#extended-docker-configuration-options).
For more information, see [Available settings for `image`](../docker/using_docker_images.md#available-settings-for-image).
#### `image:entrypoint`
An [extended docker configuration option](../docker/using_docker_images.md#extended-docker-configuration-options).
An [extended Docker configuration option](../docker/using_docker_images.md#extended-docker-configuration-options).
For more information, see [Available settings for `image`](../docker/using_docker_images.md#available-settings-for-image).
...
...
@@ -543,25 +543,25 @@ For:
##### `services:name`
An [extended docker configuration option](../docker/using_docker_images.md#extended-docker-configuration-options).
An [extended Docker configuration option](../docker/using_docker_images.md#extended-docker-configuration-options).
For more information, see [Available settings for `services`](../docker/using_docker_images.md#available-settings-for-services).
##### `services:alias`
An [extended docker configuration option](../docker/using_docker_images.md#extended-docker-configuration-options).
An [extended Docker configuration option](../docker/using_docker_images.md#extended-docker-configuration-options).
For more information, see [Available settings for `services`](../docker/using_docker_images.md#available-settings-for-services).
##### `services:entrypoint`
An [extended docker configuration option](../docker/using_docker_images.md#extended-docker-configuration-options).
An [extended Docker configuration option](../docker/using_docker_images.md#extended-docker-configuration-options).
For more information, see [Available settings for `services`](../docker/using_docker_images.md#available-settings-for-services).
##### `services:command`
An [extended docker configuration option](../docker/using_docker_images.md#extended-docker-configuration-options).
An [extended Docker configuration option](../docker/using_docker_images.md#extended-docker-configuration-options).
For more information, see [Available settings for `services`](../docker/using_docker_images.md#available-settings-for-services).
@@ -139,7 +139,7 @@ might also help with keeping the image small.
As documented in the [Docker Official Images](https://github.com/docker-library/official-images#tags-and-aliases) project,
it is strongly encouraged that version number tags be given aliases which allows the user to easily refer to the "most recent" release of a particular series.
See also [Docker Tagging: Best practices for tagging and versioning docker images](https://docs.microsoft.com/en-us/archive/blogs/stevelasker/docker-tagging-best-practices-for-tagging-and-versioning-docker-images).
See also [Docker Tagging: Best practices for tagging and versioning Docker images](https://docs.microsoft.com/en-us/archive/blogs/stevelasker/docker-tagging-best-practices-for-tagging-and-versioning-docker-images).
@@ -336,7 +336,7 @@ Snowplow Inspector Chrome Extension is a browser extension for testing frontend
Snowplow Micro is a very small version of a full Snowplow data collection pipeline: small enough that it can be launched by a test suite. Events can be recorded into Snowplow Micro just as they can a full Snowplow pipeline. Micro then exposes an API that can be queried.
Snowplow Micro is a docker-based solution for testing frontend and backend events in a local development environment. You need to modify GDK using the instructions below to set this up.
Snowplow Micro is a Docker-based solution for testing frontend and backend events in a local development environment. You need to modify GDK using the instructions below to set this up.
@@ -9,11 +9,11 @@ This is a partial list of the [RSpec metadata](https://relishapp.com/rspec/rspec
|-----|-------------|
| `:elasticsearch` | The test requires an Elasticsearch service. It is used by the [instance-level scenario](https://gitlab.com/gitlab-org/gitlab-qa#definitions)[`Test::Integration::Elasticsearch`](https://gitlab.com/gitlab-org/gitlab/-/blob/72b62b51bdf513e2936301cb6c7c91ec27c35b4d/qa/qa/ee/scenario/test/integration/elasticsearch.rb) to include only tests that require Elasticsearch. |
| `:kubernetes` | The test includes a GitLab instance that is configured to be run behind an SSH tunnel, allowing a TLS-accessible GitLab. This test will also include provisioning of at least one Kubernetes cluster to test against. *This tag is often be paired with `:orchestrated`.* |
| `:orchestrated` | The GitLab instance under test may be [configured by `gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/master/docs/what_tests_can_be_run.md#orchestrated-tests) to be different to the default GitLab configuration, or `gitlab-qa` may launch additional services in separate docker containers, or both. Tests tagged with `:orchestrated` are excluded when testing environments where we can't dynamically modify GitLab's configuration (for example, Staging). |
| `:orchestrated` | The GitLab instance under test may be [configured by `gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/master/docs/what_tests_can_be_run.md#orchestrated-tests) to be different to the default GitLab configuration, or `gitlab-qa` may launch additional services in separate Docker containers, or both. Tests tagged with `:orchestrated` are excluded when testing environments where we can't dynamically modify GitLab's configuration (for example, Staging). |
| `:quarantine` | The test has been [quarantined](https://about.gitlab.com/handbook/engineering/quality/guidelines/debugging-qa-test-failures/#quarantining-tests), will run in a separate job that only includes quarantined tests, and is allowed to fail. The test will be skipped in its regular job so that if it fails it will not hold up the pipeline. |
| `:reliable` | The test has been [promoted to a reliable test](https://about.gitlab.com/handbook/engineering/quality/guidelines/reliable-tests/#promoting-an-existing-test-to-reliable) meaning it passes consistently in all pipelines, including merge requests. |
| `:requires_admin` | The test requires an admin account. Tests with the tag are excluded when run against Canary and Production environments. |
| `:runner` | The test depends on and will set up a GitLab Runner instance, typically to run a pipeline. |
| `:gitaly_ha` | The test will run against a GitLab instance where repositories are stored on redundant Gitaly nodes behind a Praefect node. All nodes are [separate containers](../../../administration/gitaly/praefect.md#requirements-for-configuring-a-gitaly-cluster). Tests that use this tag have a longer setup time since there are three additional containers that need to be started. |
| `:skip_live_env` | The test will be excluded when run against live deployed environments such as Staging, Canary, and Production. |
| `:jira` | The test requires a Jira Server. [GitLab-QA](https://gitlab.com/gitlab-org/gitlab-qa) will provision the Jira Server in a docker container when the `Test::Integration::Jira` test scenario is run.
| `:jira` | The test requires a Jira Server. [GitLab-QA](https://gitlab.com/gitlab-org/gitlab-qa) will provision the Jira Server in a Docker container when the `Test::Integration::Jira` test scenario is run.
The [`jenkins_build_status_spec`](https://gitlab.com/gitlab-org/gitlab/blob/163c8a8c814db26d11e104d1cb2dcf02eb567dbe/qa/qa/specs/features/ee/browser_ui/3_create/jenkins/jenkins_build_status_spec.rb) spins up a Jenkins instance in a docker container based on an image stored in the [GitLab-QA container registry](https://gitlab.com/gitlab-org/gitlab-qa/container_registry).
The docker image it uses is preconfigured with some base data and plugins.
The [`jenkins_build_status_spec`](https://gitlab.com/gitlab-org/gitlab/blob/163c8a8c814db26d11e104d1cb2dcf02eb567dbe/qa/qa/specs/features/ee/browser_ui/3_create/jenkins/jenkins_build_status_spec.rb) spins up a Jenkins instance in a Docker container based on an image stored in the [GitLab-QA container registry](https://gitlab.com/gitlab-org/gitlab-qa/container_registry).
The Docker image it uses is preconfigured with some base data and plugins.
The test then configures the GitLab plugin in Jenkins with a URL of the GitLab instance that will be used
to run the tests. Unfortunately, the GitLab Jenkins plugin does not accept ports so `http://localhost:3000` would
not be accepted. Therefore, this requires us to run GitLab on port 80 or inside a docker container.
not be accepted. Therefore, this requires us to run GitLab on port 80 or inside a Docker container.
To start a docker container for GitLab based on the nightly image:
To start a Docker container for GitLab based on the nightly image:
```shell
docker run \
...
...
@@ -24,7 +24,7 @@ To run the tests from the `/qa` directory:
@@ -74,7 +74,7 @@ Follow these steps to enable the container registry. Note that these steps refle
sudo gitlab-ctl reconfigure
```
## Allow the docker daemon to trust the registry and GitLab Runner
## Allow the Docker daemon to trust the registry and GitLab Runner
Provide your Docker daemon with your certs by
[following the steps for using trusted certificates with your registry](../../administration/packages/container_registry.md#using-self-signed-certificates-with-container-registry):
...
...
@@ -125,7 +125,7 @@ Now we must add some additional configuration to our runner:
Make the following changes to `/etc/gitlab-runner/config.toml`:
- Add docker socket to volumes `volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]`
- Add Docker socket to volumes `volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]`
- Add `pull_policy = "if-not-present"` to the executor configuration
| [Container Scanning](container_scanning/index.md) | Uses `clair`. The latest `clair-db` version is used for each job by running the [`latest` docker image tag](https://gitlab.com/gitlab-org/gitlab/blob/438a0a56dc0882f22bdd82e700554525f552d91b/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L37). The `clair-db` database [is updated daily according to the author](https://github.com/arminc/clair-local-scan#clair-server-or-local). |
| [Container Scanning](container_scanning/index.md) | Uses `clair`. The latest `clair-db` version is used for each job by running the [`latest` Docker image tag](https://gitlab.com/gitlab-org/gitlab/blob/438a0a56dc0882f22bdd82e700554525f552d91b/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L37). The `clair-db` database [is updated daily according to the author](https://github.com/arminc/clair-local-scan#clair-server-or-local). |
| [Dependency Scanning](dependency_scanning/index.md) | Relies on `bundler-audit` (for Rubygems), `retire.js` (for NPM packages), and `gemnasium` (GitLab's own tool for all libraries). Both `bundler-audit` and `retire.js` fetch their vulnerabilities data from GitHub repositories, so vulnerabilities added to `ruby-advisory-db` and `retire.js` are immediately available. The tools themselves are updated once per month if there's a new version. The [Gemnasium DB](https://gitlab.com/gitlab-org/security-products/gemnasium-db) is updated at least once a week. See our [current measurement of time from CVE being issued to our product being updated](https://about.gitlab.com/handbook/engineering/development/performance-indicators/#cve-issue-to-update). |
| [Dynamic Application Security Testing (DAST)](dast/index.md) | The scanning engine is updated on a periodic basis. See the [version of the underlying tool `zaproxy`](https://gitlab.com/gitlab-org/security-products/dast/blob/master/Dockerfile#L1). The scanning rules are downloaded at scan runtime. |
| [Static Application Security Testing (SAST)](sast/index.md) | Relies exclusively on [the tools GitLab wraps](sast/index.md#supported-languages-and-frameworks). The underlying analyzers are updated at least once per month if a relevant update is available. The vulnerabilities database is updated by the upstream tools. |
@@ -67,7 +67,7 @@ This example shows how to run Code Quality on your code by using GitLab CI/CD an
First, you need GitLab Runner configured:
- For the [docker-in-docker workflow](../../../ci/docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor).
- For the [Docker-in-Docker workflow](../../../ci/docker/using_docker_build.md#use-docker-in-docker-workflow-with-docker-executor).
- With enough disk space to handle generated Code Quality files. For example on the [GitLab project](https://gitlab.com/gitlab-org/gitlab) the files are approximately 7 GB.
Once you set up the Runner, include the CodeQuality template in your CI config:
...
...
@@ -120,7 +120,7 @@ This information will be automatically extracted and shown right in the merge re
CAUTION: **Caution:**
On self-managed instances, if a malicious actor compromises the Code Quality job
definition they will be able to execute privileged docker commands on the Runner
definition they will be able to execute privileged Docker commands on the Runner
host. Having proper access control policies mitigates this attack vector by