| AFL (any language that works on top of AFL) | [AFL](https://lcamtuf.coredump.cx/afl/)| [afl-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/afl-fuzzing-example) |
| AFL (any language that works on top of AFL) | [AFL](https://lcamtuf.coredump.cx/afl/) | [afl-fuzzing-example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/afl-fuzzing-example) |
## Configuration
## Configuration
To enable fuzzing, you must
To enable coverage-guided fuzz testing, edit the `.gitlab-ci.yml` file:
the [`Coverage-Fuzzing.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Coverage-Fuzzing.gitlab-ci.yml)
provided as part of your GitLab installation.
To do so, add the following to your `.gitlab-ci.yml` file:
1. Add the `fuzz` stage to the list of stages.
1. If your application is not written in Go, [provide a Docker image](../../../ci/yaml/index.md#image) using the matching fuzzing
engine. For example:
```yaml
image:python:latest
```
1.[Include](../../../ci/yaml/index.md#includetemplate) the
- Runs the target with the `gitlab-cov-fuzz` command, which is available to each job that extends
| `COVFUZZ_ADDITIONAL_ARGS` | Arguments passed to `gitlab-cov-fuzz`. Used to customize the behavior of the underlying fuzzing engine. Read the fuzzing engine's documentation for a complete list of arguments. |
`.fuzz_base`.
| `COVFUZZ_BRANCH` | The branch on which long-running fuzzing jobs are to be run. On all other branches, only fuzzing regression tests are run. Default: Repository's default branch. |
- Runs on a fuzz stage that usually comes after a test stage.
| `COVFUZZ_SEED_CORPUS` | Path to a seed corpus directory. Default: empty. |
| `COVFUZZ_URL_PREFIX` | Path to the `gitlab-cov-fuzz` repository cloned for use with an offline environment. You should only change this value when using an offline environment. Default: `https://gitlab.com/gitlab-org/security-products/analyzers/gitlab-cov-fuzz/-/raw`. |
The `gitlab-cov-fuzz` is a command-line tool that runs the instrumented application. It parses and
#### Seed corpus
analyzes the exception information that the fuzzer outputs. It also downloads the [corpus](../terminology/index.md#corpus)
and crash events from previous pipelines automatically. This helps your fuzz targets build on the
progress of previous fuzzing jobs. The parsed crash events and data are written to
`gl-coverage-fuzzing-report.json`.
### Artifacts
Files in the [seed corpus](../terminology/index.md#seed-corpus) must be updated manually. They are
not updated or overwritten by the coverage-guide fuzz testing job.
## Output
Each fuzzing step outputs these artifacts:
Each fuzzing step outputs these artifacts:
-`gl-coverage-fuzzing-report.json`: This file's format may change in future releases.
-`gl-coverage-fuzzing-report.json`: A report containing details of the coverage-guided fuzz testing
and its results.
-`artifacts.zip`: This file contains two directories:
-`artifacts.zip`: This file contains two directories:
-`corpus`: Holds all test cases generated by the current and all previous jobs.
-`corpus`: Contains all test cases generated by the current and all previous jobs.
-`crashes`: Holds all crash events the current job encountered as well as those not fixed in
-`crashes`: Contains all crash events the current job found and those not fixed in
previous jobs.
previous jobs.
### Types of fuzzing jobs
You can download the JSON report file from the CI/CD pipelines page. For more information, see
- Fuzzing: Standard fuzzing session. You can configure a long session through a user defined
timeout.
- Regression: Run the fuzz targets through the accumulated test cases generated by previous fuzzing
sessions plus fixed crashes from previous sessions. This is usually very quick.
Here's our current suggestion for configuring your fuzz target's timeout:
- Set `COVFUZZ_BRANCH` to the branch where you want to run long-running (asynchronous) fuzzing
jobs. This is normally the default branch.
- Use regression or short-running fuzzing jobs for other branches or merge requests.
This suggestion helps find new bugs on the development branch and catch old bugs in merge requests
(like unit tests).
You can configure this by passing `--regression=false/true` to `gitlab-cov-fuzz` as the [Go example](https://gitlab.com/gitlab-org/security-products/demos/go-fuzzing-example/-/blob/master/.gitlab-ci.yml)
shows. Also note that `gitlab-cov-fuzz` is a wrapper, so you can pass those arguments to configure
any option available in the underlying fuzzing engine.
| `COVFUZZ_BRANCH` | The branch for long-running fuzzing jobs. This is normally the default branch. |
| `COVFUZZ_SEED_CORPUS` | Path to a seed corpus directory. The default is empty. |
| `COVFUZZ_URL_PREFIX` | Path to the `gitlab-cov-fuzz` repository cloned for use with an offline environment. You should only change this when using an offline environment. The default value is `https://gitlab.com/gitlab-org/security-products/analyzers/gitlab-cov-fuzz/-/raw`. |
The files in the [seed corpus](../terminology/index.md#seed-corpus)(`COVFUZZ_SEED_CORPUS`), if provided, aren't updated unless you commit new
files to your Git repository. There's usually no need to frequently update the seed corpus. As part
of the GitLab artifacts system, GitLab saves in a corpus directory the new test cases that every run
generates. In any subsequent runs, GitLab also reuses the generated corpus together with the seed
corpus.
### Reports JSON format
### Coverage-guided fuzz testing report
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/220062) in GitLab 13.3 as an [Alpha feature](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha).
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/220062) in GitLab 13.3 as an [Alpha feature](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha).
The `gitlab-cov-fuzz` tool emits a JSON report file. For more information, see the
For detailed information about the `gl-coverage-fuzzing-report.json` file's format, read the
[schema for this report](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/blob/master/dist/coverage-fuzzing-report-format.json).
- 10-minute duration: Recommended for the default branch.
to a private repository that your offline GitLab instance can access.
- 60-minute duration: Recommended for the development branch and merge requests. The longer duration provides greater coverage.
In the `COVFUZZ_ADDITIONAL_ARGS` variable set the value `--regression=true`.
1. For each fuzzing step, set `COVFUZZ_URL_PREFIX` to `${NEW_URL_GITLAB_COV_FUZ}/-/raw`, where
For a complete example, read the [Go coverage-guided fuzzing example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example/-/blob/master/.gitlab-ci.yml).
`NEW_URL_GITLAB_COV_FUZ` is the URL of the private `gitlab-cov-fuzz` clone that you set up in the
It's also possible to run the fuzzing jobs longer and without blocking your main pipeline. This
It's also possible to run the coverage-guided fuzzing jobs longer and without blocking your main
configuration uses the GitLab [parent-child pipelines](../../../ci/pipelines/parent_child_pipelines.md).
pipeline. This configuration uses the GitLab
The full example is available in the [repository](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example/-/tree/continuous_fuzzing#running-go-fuzz-from-ci).
This example uses Go, but is applicable for any other supported languages.
The suggested workflow in this scenario is to have long-running, asynchronous fuzzing jobs on a
The suggested workflow in this scenario is to have long-running, asynchronous fuzzing jobs on the
main/development branch, and short, blocking sync fuzzing jobs on all other branches and MRs. This
main or development branch, and short synchronous fuzzing jobs on all other branches and MRs. This
is a good way to balance the needs of letting a developer's per-commit pipeline complete quickly,
balances the needs of completing the per-commit pipeline complete quickly, while also giving the
and also giving the fuzzer a large amount of time to fully explore and test the app.
fuzzer a large amount of time to fully explore and test the app. Long-running fuzzing jobs are
usually necessary for the coverage-guided fuzzer to find deeper bugs in your codebase.
Long-running fuzzing jobs are usually necessary for the coverage guided fuzzer to find deeper bugs
The following is an extract of the `.gitlab-ci.yml` file for this
in your latest codebase. The following is an example of what `.gitlab-ci.yml` looks like in this
workflow. For the full example, see the [Go fuzzing example's repository](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example/-/tree/continuous_fuzzing):
workflow (for the full example, see the [repository](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example/-/tree/continuous_fuzzing)):
1.`sync_fuzzing`: Runs all your fuzz targets for a short period of time in a blocking
1.`sync_fuzzing`: Runs all your fuzz targets for a short period of time in a blocking
configuration. This finds simple bugs and allows you to be confident that your MRs aren't
configuration. This finds simple bugs and allows you to be confident that your MRs aren't
...
@@ -246,6 +214,17 @@ This essentially creates two steps:
...
@@ -246,6 +214,17 @@ This essentially creates two steps:
The `covfuzz-ci.yml` is the same as that in the [original synchronous example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example#running-go-fuzz-from-ci).
The `covfuzz-ci.yml` is the same as that in the [original synchronous example](https://gitlab.com/gitlab-org/security-products/demos/coverage-fuzzing/go-fuzzing-example#running-go-fuzz-from-ci).
## Offline environment
To use coverage fuzzing in an offline environment:
