Skip to content

G
gitlab-ce
Commit bacd8c9d authored by Russell Dickenson's avatar Russell Dickenson
Browse files
Options

Merge branch 'docs-api-fuzz-edit' into 'master' 

Follow up edits to API fuzzing doc

See merge request gitlab-org/gitlab!52634
parents 49de8479 774dd4f1
Hide whitespace changes
Inline Side-by-side
Showing with 76 additions and 90 deletions
doc/user/application_security/api_fuzzing/index.md
View file @ bacd8c9d
...@@ -335,16 +335,18 @@ provide a script that performs an authentication flow or calculates the token. ...@@ -335,16 +335,18 @@ provide a script that performs an authentication flow or calculates the token.
#### HTTP Basic Authentication #### HTTP Basic Authentication
[HTTP basic authentication](https://en.wikipedia.org/wiki/Basic_access_authentication) [HTTP basic authentication](https://en.wikipedia.org/wiki/Basic_access_authentication)
is an authentication method built into the HTTP protocol and used in-conjunction with is an authentication method built in to the HTTP protocol and used in conjunction with
[transport layer security (TLS)](https://en.wikipedia.org/wiki/Transport_Layer_Security). [transport layer security (TLS)](https://en.wikipedia.org/wiki/Transport_Layer_Security).
To use HTTP basic authentication, two variables are added to your `.gitlab-ci.yml` file:
To use HTTP basic authentication, add these two variables to your `.gitlab-ci.yml` file:
- `FUZZAPI_HTTP_USERNAME`: The username for authentication. - `FUZZAPI_HTTP_USERNAME`: The username for authentication.
- `FUZZAPI_HTTP_PASSWORD`: The password for authentication. - `FUZZAPI_HTTP_PASSWORD`: The password for authentication.
For the password, we recommended that you [create a CI/CD variable](../../../ci/variables/README.md#create-a-custom-variable-in-the-ui) For the password, we recommended that you [create a CI/CD variable](../../../ci/variables/README.md#create-a-custom-variable-in-the-ui)
(for example, `TEST_API_PASSWORD`) set to the password. You can create CI/CD variables from the (for example, `TEST_API_PASSWORD`) set to the password. You can create CI/CD variables from the
GitLab projects page at **Settings > CI/CD**, in the **Variables** section. GitLab projects page at **Settings > CI/CD**, in the **Variables** section. Use that variable
as the value for `FUZZAPI_HTTP_PASSWORD`:
```yaml ```yaml
include: include:
...@@ -371,36 +373,39 @@ tokens with API fuzzing, you need one of the following: ...@@ -371,36 +373,39 @@ tokens with API fuzzing, you need one of the following:
##### Token doesn't expire ##### Token doesn't expire
If the bearer token doesn't expire, you can provide it using the `FUZZAPI_OVERRIDES_ENV` variable. If the bearer token doesn't expire, use the `FUZZAPI_OVERRIDES_ENV` variable to provide it. This
The `FUZZAPI_OVERRIDES_ENV` content is a JSON snippet that provides headers and cookies that should variable's content is a JSON snippet that provides headers and cookies to add to API fuzzing's
be added to outgoing HTTP requests made by API fuzzing. outgoing HTTP requests.
Create a CI/CD variable, for example `TEST_API_BEARERAUTH`, with the value Follow these steps to provide the bearer token with `FUZZAPI_OVERRIDES_ENV`:
`{"headers":{"Authorization":"Bearer dXNlcm5hbWU6cGFzc3dvcmQ="}}` (substitute your token). You can
create CI/CD variables from the GitLab projects page at **Settings > CI/CD** in the **Variables**
section.
Set `FUZZAPI_OVERRIDES_ENV` in your `.gitlab-ci.yml` file: 1. [Create a CI/CD variable](../../../ci/variables/README.md#create-a-custom-variable-in-the-ui),
for example `TEST_API_BEARERAUTH`, with the value
`{"headers":{"Authorization":"Bearer dXNlcm5hbWU6cGFzc3dvcmQ="}}` (substitute your token). You
can create CI/CD variables from the GitLab projects page at **Settings > CI/CD**, in the
**Variables** section.
```yaml 1. In your `.gitlab-ci.yml` file, set `FUZZAPI_OVERRIDES_ENV` to the variable you just created:
include:
- template: API-Fuzzing.gitlab-ci.yml
variables: ```yaml
FUZZAPI_PROFILE: Quick-10 include:
FUZZAPI_OPENAPI: test-api-specification.json - template: API-Fuzzing.gitlab-ci.yml
FUZZAPI_TARGET_URL: http://test-deployment/
FUZZAPI_OVERRIDES_ENV: $TEST_API_BEARERAUTH
```
To validate that authentication is working, run an API fuzzing test and review the fuzzing logs and variables:
the test API's application logs. FUZZAPI_PROFILE: Quick-10
FUZZAPI_OPENAPI: test-api-specification.json
FUZZAPI_TARGET_URL: http://test-deployment/
FUZZAPI_OVERRIDES_ENV: $TEST_API_BEARERAUTH
```
##### Token generated at test-runtime 1. To validate that authentication is working, run an API fuzzing test and review the fuzzing logs
and the test API's application logs.
If the bearer token must be generated, and the resulting token doesn't expire during testing, you ##### Token generated at test runtime
can provide to API fuzzing a file containing the token. This file can be generated by a prior stage
and job, or as part of the API fuzzing job. If the bearer token must be generated and doesn't expire during testing, you can provide to API
fuzzing a file containing the token. A prior stage and job, or part of the API fuzzing job, can
generate this file.
API fuzzing expects to receive a JSON file with the following structure: API fuzzing expects to receive a JSON file with the following structure:
...@@ -412,10 +417,8 @@ API fuzzing expects to receive a JSON file with the following structure: ...@@ -412,10 +417,8 @@ API fuzzing expects to receive a JSON file with the following structure:
} }
``` ```
This file can be generated by a prior stage and provided to API fuzzing through the You can provide this file to API fuzzing through the `FUZZAPI_OVERRIDES_FILE` variable, in your
`FUZZAPI_OVERRIDES_FILE` variable. `.gitlab-ci.yml` file:
Set `FUZZAPI_OVERRIDES_FILE` in your `.gitlab-ci.yml` file:
```yaml ```yaml
include: include:
...@@ -450,9 +453,11 @@ The script must create a JSON file containing the bearer token in a specific for ...@@ -450,9 +453,11 @@ The script must create a JSON file containing the bearer token in a specific for
You must provide three variables, each set for correct operation: You must provide three variables, each set for correct operation:
- `FUZZAPI_OVERRIDES_FILE`: File generated by the provided command. - `FUZZAPI_OVERRIDES_FILE`: JSON file the provided command generates.
- `FUZZAPI_OVERRIDES_CMD`: Command to generate JSON file. - `FUZZAPI_OVERRIDES_CMD`: Command that generates the JSON file.
- `FUZZAPI_OVERRIDES_INTERVAL`: Interval in seconds to run command. - `FUZZAPI_OVERRIDES_INTERVAL`: Interval (in seconds) to run command.
For example:
```yaml ```yaml
include: include:
...@@ -472,18 +477,18 @@ the test API's application logs. ...@@ -472,18 +477,18 @@ the test API's application logs.
### Configuration files ### Configuration files
To get started quickly, GitLab provides you with the configuration file To get you started quickly, GitLab provides the configuration file
[`gitlab-api-fuzzing-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing/-/blob/master/gitlab-api-fuzzing-config.yml). [`gitlab-api-fuzzing-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing/-/blob/master/gitlab-api-fuzzing-config.yml).
This file has several testing profiles that perform various amounts of testing. The run time of each This file has several testing profiles that perform various numbers of tests. The run time of each
increases as the numbers go up. To use a configuration file, add it to your repository's root as profile increases as the test numbers go up. To use a configuration file, add it to your
`.gitlab-api-fuzzing.yml`. repository's root as `.gitlab-api-fuzzing.yml`.
| Profile | Scan Type | | Profile | Fuzz Tests (per parameter) |
|:---------|:-----------| |:---------|:-----------|
|Quick-10 |Fuzzing 10 times per parameter | | Quick-10 | 10 |
|Medium-20 |Fuzzing 20 times per parameter | | Medium-20 | 20 |
|Medium-50 |Fuzzing 50 times per parameter | | Medium-50 | 50 |
|Long-100 |Fuzzing 100 times per parameter | | Long-100 | 100 |
### Available variables ### Available variables
...@@ -491,18 +496,18 @@ increases as the numbers go up. To use a configuration file, add it to your repo ...@@ -491,18 +496,18 @@ increases as the numbers go up. To use a configuration file, add it to your repo
|-----------------------------|--------------------| |-----------------------------|--------------------|
| `FUZZAPI_VERSION` | Specify API Fuzzing container version. Defaults to `latest`. | | `FUZZAPI_VERSION` | Specify API Fuzzing container version. Defaults to `latest`. |
| `FUZZAPI_TARGET_URL` | Base URL of API testing target. | | `FUZZAPI_TARGET_URL` | Base URL of API testing target. |
|[`FUZZAPI_CONFIG`](#configuration-files) | API Fuzzing configuration file. Defaults to `.gitlab-apifuzzer.yml`. | | [`FUZZAPI_CONFIG`](#configuration-files) | API Fuzzing configuration file. Defaults to `.gitlab-apifuzzer.yml`. |
|[`FUZZAPI_PROFILE`](#configuration-files) | Configuration profile to use during testing. Defaults to `Quick`. | | [`FUZZAPI_PROFILE`](#configuration-files) | Configuration profile to use during testing. Defaults to `Quick`. |
| `FUZZAPI_REPORT` | Scan report filename. Defaults to `gl-api_fuzzing-report.xml`. | | `FUZZAPI_REPORT` | Scan report filename. Defaults to `gl-api_fuzzing-report.xml`. |
|[`FUZZAPI_OPENAPI`](#openapi-specification) | OpenAPI specification file or URL. | | [`FUZZAPI_OPENAPI`](#openapi-specification) | OpenAPI specification file or URL. |
|[`FUZZAPI_HAR`](#http-archive-har) | HTTP Archive (HAR) file. | | [`FUZZAPI_HAR`](#http-archive-har) | HTTP Archive (HAR) file. |
|[`FUZZAPI_POSTMAN_COLLECTION`](#postman-collection) | Postman Collection file. | | [`FUZZAPI_POSTMAN_COLLECTION`](#postman-collection) | Postman Collection file. |
|[`FUZZAPI_OVERRIDES_FILE`](#overrides) | Path to a JSON file containing overrides. | | [`FUZZAPI_OVERRIDES_FILE`](#overrides) | Path to a JSON file containing overrides. |
|[`FUZZAPI_OVERRIDES_ENV`](#overrides) | JSON string containing headers to override. | | [`FUZZAPI_OVERRIDES_ENV`](#overrides) | JSON string containing headers to override. |
|[`FUZZAPI_OVERRIDES_CMD`](#overrides) | Overrides command. | | [`FUZZAPI_OVERRIDES_CMD`](#overrides) | Overrides command. |
|[`FUZZAPI_OVERRIDES_INTERVAL`](#overrides) | How often to run overrides command in seconds. Defaults to `0` (once). | | [`FUZZAPI_OVERRIDES_INTERVAL`](#overrides) | How often to run overrides command in seconds. Defaults to `0` (once). |
|[`FUZZAPI_HTTP_USERNAME`](#http-basic-authentication) | Username for HTTP authentication. | | [`FUZZAPI_HTTP_USERNAME`](#http-basic-authentication) | Username for HTTP authentication. |
|[`FUZZAPI_HTTP_PASSWORD`](#http-basic-authentication) | Password for HTTP authentication. | | [`FUZZAPI_HTTP_PASSWORD`](#http-basic-authentication) | Password for HTTP authentication. |
<!--|[`FUZZAPI_D_TARGET_IMAGE`](#target-container) |API target docker image | <!--|[`FUZZAPI_D_TARGET_IMAGE`](#target-container) |API target docker image |
|[`FUZZAPI_D_TARGET_ENV`](#target-container) |Docker environment options | |[`FUZZAPI_D_TARGET_ENV`](#target-container) |Docker environment options |
...@@ -518,11 +523,11 @@ increases as the numbers go up. To use a configuration file, add it to your repo ...@@ -518,11 +523,11 @@ increases as the numbers go up. To use a configuration file, add it to your repo
### Overrides ### Overrides
API Fuzzing provides a method to add or override headers and cookies for all outbound HTTP requests API Fuzzing provides a method to add or override headers and cookies for all outbound HTTP requests.
made. You can use this to inject semantic version headers, authentication, and so on. The You can use this to inject semantic version headers, authentication, and so on. The
[authentication section](#authentication) includes examples of using overrides for that purpose. [authentication section](#authentication) includes examples of using overrides for that purpose.
Overrides uses a JSON document to define the headers and cookies: Overrides use a JSON document to define the headers and cookies:
```json ```json
{ {
...@@ -537,7 +542,7 @@ Overrides uses a JSON document to define the headers and cookies: ...@@ -537,7 +542,7 @@ Overrides uses a JSON document to define the headers and cookies:
} }
``` ```
Example usage for setting a single header: Example of setting a single header:
```json ```json
{ {
...@@ -547,7 +552,7 @@ Example usage for setting a single header: ...@@ -547,7 +552,7 @@ Example usage for setting a single header:
} }
``` ```
Example usage for setting both a header and cookie: Example of setting both a header and cookie:
```json ```json
{ {
...@@ -565,9 +570,10 @@ to generate the JSON document. The command can run at intervals to support value ...@@ -565,9 +570,10 @@ to generate the JSON document. The command can run at intervals to support value
#### Using a file #### Using a file
To provide the overrides JSON as a file, the `FUZZAPI_OVERRIDES_FILE` environment variable is set. The path is relative to the job current working directory. To provide the override JSON as a file, set the `FUZZAPI_OVERRIDES_FILE` environment variable to the
file. The file path is relative to the job's current working directory.
Example `.gitlab-ci.yml`: Here's an example `.gitlab-ci.yml`:
```yaml ```yaml
include: include:
...@@ -582,10 +588,10 @@ variables: ...@@ -582,10 +588,10 @@ variables:
#### Using an environment variable #### Using an environment variable
To provide the overrides JSON as an environment variable, use the `FUZZAPI_OVERRIDES_ENV` variable. To provide the override JSON as an environment variable, set the `FUZZAPI_OVERRIDES_ENV` variable to
This allows you to place the JSON as CI/CD variables that can be masked and protected. the JSON. You can also place the JSON as CI/CD variables that can be masked and protected.
In this example `.gitlab-ci.yml`, the JSON is provided directly: In this example `.gitlab-ci.yml`, the `FUZZAPI_OVERRIDES_ENV` variable is set directly to the JSON:
```yaml ```yaml
include: include:
...@@ -599,7 +605,7 @@ variables: ...@@ -599,7 +605,7 @@ variables:
``` ```
In this example `.gitlab-ci.yml`, the CI/CD variable `SECRET_OVERRIDES` provides the JSON. This is a In this example `.gitlab-ci.yml`, the CI/CD variable `SECRET_OVERRIDES` provides the JSON. This is a
[group or instance level environment variable defined in the UI](../../../ci/variables/README.md#instance-level-cicd-variables): [group or instance-level environment variable defined in the UI](../../../ci/variables/README.md#instance-level-cicd-variables):
```yaml ```yaml
include: include:
...@@ -615,29 +621,9 @@ variables: ...@@ -615,29 +621,9 @@ variables:
#### Using a command #### Using a command
If the value must be generated or regenerated on expiration, you can provide a program or script for If the value must be generated or regenerated on expiration, you can provide a program or script for
the API fuzzer to execute on a specified interval. The provided script runs in an Alpine Linux the API fuzzer to execute on a specified interval. To do this, follow the instructions in the
container that has Python 3 and Bash installed. If the Python script requires additional packages, section [Token has short expiration](#token-has-short-expiration),
it must detect this and install the packages at runtime. The script creates the overrides JSON file which uses the same three variables. The script creates the overrides JSON file as defined.
as defined above.
You must provide three variables, each set for correct operation:
- `FUZZAPI_OVERRIDES_FILE`: File generated by the provided command.
- `FUZZAPI_OVERRIDES_CMD`: Command to generate JSON file.
- `FUZZAPI_OVERRIDES_INTERVAL`: Interval in seconds to run command.
```yaml
include:
- template: API-Fuzzing.gitlab-ci.yml
variables:
FUZZAPI_PROFILE: Quick
FUZZAPI_OPENAPI: test-api-specification.json
FUZZAPI_TARGET_URL: http://test-deployment/
FUZZAPI_OVERRIDES_FILE: output/api-fuzzing-overrides.json
FUZZAPI_OVERRIDES_CMD: renew_token.py
FUZZAPI_OVERRIDES_INTERVAL: 300
```
### Header Fuzzing ### Header Fuzzing
...@@ -689,9 +675,9 @@ for off. To turn header fuzzing on, change this setting to `true`: ...@@ -689,9 +675,9 @@ for off. To turn header fuzzing on, change this setting to `true`:
Headers: Headers:
``` ```
`Headers` is a list of headers to fuzz. Only headers listed are fuzzed. For example, to fuzz a `Headers` is a list of headers to fuzz. Only headers listed are fuzzed. To fuzz a header used by
custom header `X-Custom` used by your APIs, add an entry for it using the syntax your APIs, add an entry for it using the syntax `- Name: HeaderName`. For example, to fuzz a
`- Name: HeaderName`, substituting `HeaderName` with the header to fuzz: custom header `X-Custom`, add `- Name: X-Custom`:
```yaml ```yaml
- Name: GeneralFuzzingCheck - Name: GeneralFuzzingCheck
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7