| AFL (any language that works on top of AFL) | [AFL](https://lcamtuf.coredump.cx/afl/)| [afl-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/afl-fuzzing-example) |
| AFL (any language that works on top of AFL) | [AFL](https://lcamtuf.coredump.cx/afl/) | [afl-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/afl-fuzzing-example) |
the [`Coverage-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Coverage-Fuzzing.gitlab-ci.yml)
provided as part of your GitLab installation.
To enable coverage-guided fuzz testing, edit the `.gitlab-ci.yml` file:
To do so, add the following to your `.gitlab-ci.yml` file:
1. Add the `fuzz` stage to the list of stages.
1. If your application is not written in Go, [provide a Docker image](../../../ci/yaml/index.md#image) using the matching fuzzing
engine. For example:
```yaml
image:python:latest
```
1.[Include](../../../ci/yaml/index.md#includetemplate) the
| `COVFUZZ_ADDITIONAL_ARGS` | Arguments passed to `gitlab-cov-fuzz`. Used to customize the behavior of the underlying fuzzing engine. Read the fuzzing engine's documentation for a complete list of arguments. |
| `COVFUZZ_BRANCH` | The branch on which long-running fuzzing jobs are to be run. On all other branches, only fuzzing regression tests are run. Default: Repository's default branch. |
| `COVFUZZ_SEED_CORPUS` | Path to a seed corpus directory. Default: empty. |
| `COVFUZZ_URL_PREFIX` | Path to the `gitlab-cov-fuzz` repository cloned for use with an offline environment. You should only change this value when using an offline environment. Default: `https://gitlab.com/gitlab-org/security-products/analyzers/gitlab-cov-fuzz/-/raw`. |
The `gitlab-cov-fuzz` is a command-line tool that runs the instrumented application. It parses and
analyzes the exception information that the fuzzer outputs. It also downloads the [corpus](../terminology/index.md#corpus)
and crash events from previous pipelines automatically. This helps your fuzz targets build on the
progress of previous fuzzing jobs. The parsed crash events and data are written to
`gl-coverage-fuzzing-report.json`.
#### Seed corpus
### Artifacts
Files in the [seed corpus](../terminology/index.md#seed-corpus) must be updated manually. They are
not updated or overwritten by the coverage-guide fuzz testing job.
## Output
Each fuzzing step outputs these artifacts:
-`gl-coverage-fuzzing-report.json`: This file's format may change in future releases.
-`gl-coverage-fuzzing-report.json`: A report containing details of the coverage-guided fuzz testing
and its results.
-`artifacts.zip`: This file contains two directories:
-`corpus`: Holds all test cases generated by the current and all previous jobs.
-`crashes`: Holds all crash events the current job encountered as well as those not fixed in
-`corpus`: Contains all test cases generated by the current and all previous jobs.
-`crashes`: Contains all crash events the current job found and those not fixed in
previous jobs.
### Types of fuzzing jobs
There are two types of jobs:
- Fuzzing: Standard fuzzing session. You can configure a long session through a user defined
timeout.
- Regression: Run the fuzz targets through the accumulated test cases generated by previous fuzzing
sessions plus fixed crashes from previous sessions. This is usually very quick.
Here's our current suggestion for configuring your fuzz target's timeout:
- Set `COVFUZZ_BRANCH` to the branch where you want to run long-running (asynchronous) fuzzing
jobs. This is normally the default branch.
- Use regression or short-running fuzzing jobs for other branches or merge requests.
This suggestion helps find new bugs on the development branch and catch old bugs in merge requests
(like unit tests).
You can configure this by passing `--regression=false/true` to `gitlab-cov-fuzz` as the [Go example](https://gitlab.com/gitlab-org/security-products/demos/go-fuzzing-example/-/blob/master/.gitlab-ci.yml)
shows. Also note that `gitlab-cov-fuzz` is a wrapper, so you can pass those arguments to configure
any option available in the underlying fuzzing engine.
| `COVFUZZ_BRANCH` | The branch for long-running fuzzing jobs. This is normally the default branch. |
| `COVFUZZ_SEED_CORPUS` | Path to a seed corpus directory. The default is empty. |
| `COVFUZZ_URL_PREFIX` | Path to the `gitlab-cov-fuzz` repository cloned for use with an offline environment. You should only change this when using an offline environment. The default value is `https://gitlab.com/gitlab-org/security-products/analyzers/gitlab-cov-fuzz/-/raw`. |
The files in the [seed corpus](../terminology/index.md#seed-corpus)(`COVFUZZ_SEED_CORPUS`), if provided, aren't updated unless you commit new
files to your Git repository. There's usually no need to frequently update the seed corpus. As part
of the GitLab artifacts system, GitLab saves in a corpus directory the new test cases that every run
generates. In any subsequent runs, GitLab also reuses the generated corpus together with the seed
corpus.
You can download the JSON report file from the CI/CD pipelines page. For more information, see
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/220062) in GitLab 13.3 as an [Alpha feature](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha).
The `gitlab-cov-fuzz` tool emits a JSON report file. For more information, see the
[schema for this report](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/coverage-fuzzing-report-format.json).
You can download the JSON report file from the CI pipelines page. For more information, see
to a private repository that your offline GitLab instance can access.
- 10-minute duration: Recommended for the default branch.
- 60-minute duration: Recommended for the development branch and merge requests. The longer duration provides greater coverage.
In the `COVFUZZ_ADDITIONAL_ARGS` variable set the value `--regression=true`.
1. For each fuzzing step, set `COVFUZZ_URL_PREFIX` to `${NEW_URL_GITLAB_COV_FUZ}/-/raw`, where
`NEW_URL_GITLAB_COV_FUZ` is the URL of the private `gitlab-cov-fuzz` clone that you set up in the
first step.
For a complete example, read the [Go coverage-guided fuzzing example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example/-/blob/master/.gitlab-ci.yml).
It's also possible to run the fuzzing jobs longer and without blocking your main pipeline. This
configuration uses the GitLab [parent-child pipelines](../../../ci/pipelines/parent_child_pipelines.md).
The full example is available in the [repository](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example/-/tree/continuous_fuzzing#running-go-fuzz-from-ci).
This example uses Go, but is applicable for any other supported languages.
It's also possible to run the coverage-guided fuzzing jobs longer and without blocking your main
The suggested workflow in this scenario is to have long-running, asynchronous fuzzing jobs on a
main/development branch, and short, blocking sync fuzzing jobs on all other branches and MRs. This
is a good way to balance the needs of letting a developer's per-commit pipeline complete quickly,
and also giving the fuzzer a large amount of time to fully explore and test the app.
The suggested workflow in this scenario is to have long-running, asynchronous fuzzing jobs on the
main or development branch, and short synchronous fuzzing jobs on all other branches and MRs. This
balances the needs of completing the per-commit pipeline complete quickly, while also giving the
fuzzer a large amount of time to fully explore and test the app. Long-running fuzzing jobs are
usually necessary for the coverage-guided fuzzer to find deeper bugs in your codebase.
Long-running fuzzing jobs are usually necessary for the coverage guided fuzzer to find deeper bugs
in your latest codebase. The following is an example of what `.gitlab-ci.yml` looks like in this
workflow (for the full example, see the [repository](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example/-/tree/continuous_fuzzing)):
The following is an extract of the `.gitlab-ci.yml` file for this
workflow. For the full example, see the [Go fuzzing example's repository](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example/-/tree/continuous_fuzzing):
1.`sync_fuzzing`: Runs all your fuzz targets for a short period of time in a blocking
configuration. This finds simple bugs and allows you to be confident that your MRs aren't
...
...
@@ -246,6 +214,17 @@ This essentially creates two steps:
The `covfuzz-ci.yml` is the same as that in the [original synchronous example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example#running-go-fuzz-from-ci).
## Offline environment
To use coverage fuzzing in an offline environment:
