Skip to content

G
gitlab-ce
Commit 47bc5528 authored by Russell Dickenson's avatar Russell Dickenson Committed by Craig Norris
Browse files
Options

Improve DAST API scanning page

parent 898599bd
Hide whitespace changes
Inline Side-by-side
Showing with 124 additions and 241 deletions
doc/user/application_security/dast_api/index.md
View file @ 47bc5528
...@@ -7,65 +7,56 @@ type: reference, howto ...@@ -7,65 +7,56 @@ type: reference, howto
# DAST API **(ULTIMATE)** # DAST API **(ULTIMATE)**
You can add dynamic application security testing of web APIs to your [GitLab CI/CD](../../../ci/index.md) pipelines. You can add dynamic application security testing (DAST) of web APIs to your
This helps you discover bugs and potential security issues that other QA processes may miss. [GitLab CI/CD](../../../ci/index.md) pipelines. This helps you discover bugs and potential security
issues that other QA processes may miss.
We recommend that you use DAST API testing in addition to [GitLab Secure](../index.md)'s We recommend that you use DAST API testing in addition to [GitLab Secure](../index.md)'s
other security scanners and your own test processes. If you're using [GitLab CI/CD](../../../ci/index.md), other security scanners and your own test processes. If you're using [GitLab CI/CD](../../../ci/index.md),
you can run DAST API tests as part your CI/CD workflow. you can run DAST API tests as part your CI/CD workflow.
## Requirements WARNING:
Do not run DAST API testing against a production server. Not only can it perform *any* function that
- One of the following web API types: the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting
- REST API data. Only run DAST API against a test server.
- SOAP
- GraphQL
- Form bodies, JSON, or XML
- One of the following assets to provide APIs to test:
- OpenAPI v2 or v3 API definition
- Postman Collection v2.0 or v2.1
- HTTP Archive (HAR) of API requests to test
## When DAST API scans run You can run DAST API scanning against the following web API types:
When using the `DAST-API.gitlab-ci.yml` template, the defined jobs use the `dast` stage by default. To enable your `.gitlab-ci.yml` file must include the `dast` stage in your `stages` definition. To ensure DAST API scans the latest code, your CI pipeline should deploy changes to a test environment in a stage before the `dast` stage: - REST API
- SOAP
- GraphQL
- Form bodies, JSON, or XML
```yaml ## When DAST API scans run
stages:
- build
- test
- deploy
- dast
```
Note that if your pipeline is configured to deploy to the same web server on each run, running a
pipeline while another is still running could cause a race condition in which one pipeline
overwrites the code from another. The API to scan should be excluded from changes for the duration
of a DAST API scan. The only changes to the API should be from the DAST API scanner. Be aware that
any changes made to the API (for example, by users, scheduled tasks, database changes, code
changes, other pipelines, or other scanners) during a scan could cause inaccurate results.
## Enable DAST API scanning DAST API scanning runs in the `dast` stage by default. To ensure DAST API scanning examines the latest
code, ensure your CI/CD pipeline deploys changes to a test environment in a stage before the `dast`
stage.
There are three ways to perform scans. See the configuration section for the one you wish to use: If your pipeline is configured to deploy to the same web server on each run, running a pipeline
while another is still running could cause a race condition in which one pipeline overwrites the
code from another. The API to be scanned should be excluded from changes for the duration of a
DAST API scan. The only changes to the API should be from the DAST API scanner. Changes made to the
API (for example, by users, scheduled tasks, database changes, code changes, other pipelines, or
other scanners) during a scan could cause inaccurate results.
- [OpenAPI v2 or v3 specification](#openapi-specification) ## Example DAST API scanning configurations
- [HTTP Archive (HAR)](#http-archive-har)
- [Postman Collection v2.0 or v2.1](#postman-collection)
Examples of various configurations can be found here: The following projects demonstrate DAST API scanning:
- [Example OpenAPI v2 specification project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/openapi-example) - [Example OpenAPI v2 Specification project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/openapi-example)
- [Example HTTP Archive (HAR) project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/har-example) - [Example HTTP Archive (HAR) project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/har-example)
- [Example Postman Collection project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/postman-example) - [Example Postman Collection project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/postman-example)
- [Example GraphQL project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/graphql-example) - [Example GraphQL project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/graphql-example)
- [Example SOAP project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/soap-example) - [Example SOAP project](https://gitlab.com/gitlab-org/security-products/demos/api-dast/soap-example)
WARNING: ## Targeting API for DAST scanning
GitLab 14.0 will require that you place DAST API configuration files (for example,
`gitlab-dast-api-config.yml`) in your repository's `.gitlab` directory instead of your You can specify the API you want to scan by using:
repository's root. You can continue using your existing configuration files as they are, but
starting in GitLab 14.0, GitLab will not check your repository's root for configuration files. - [OpenAPI v2 or v3 Specification](#openapi-specification)
- [HTTP Archive (HAR)](#http-archive-har)
- [Postman Collection v2.0 or v2.1](#postman-collection)
### OpenAPI Specification ### OpenAPI Specification
...@@ -84,52 +75,19 @@ the body generation is limited to these body types: ...@@ -84,52 +75,19 @@ the body generation is limited to these body types:
- `application/json` - `application/json`
- `application/xml` - `application/xml`
To configure DAST API scanning with an OpenAPI specification: #### DAST API scanning with an OpenAPI Specification
1. To use DAST API scanning, [include](../../../ci/yaml/index.md#includetemplate)
the [`DAST-API.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml)
that's provided as part of your GitLab installation. Add the following to your
`.gitlab-ci.yml` file:
```yaml To configure DAST API scanning with an OpenAPI Specification:
stages:
- dast
include: 1. [Include](../../../ci/yaml/index.md#includetemplate)
- template: DAST-API.gitlab-ci.yml the [`DAST-API.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml) in your `.gitlab-ci.yml` file.
```
1. The [configuration file](#configuration-files) has several testing profiles defined with different checks enabled. We recommend that you start with the `Quick` profile. 1. The [configuration file](#configuration-files) has several testing profiles defined with different checks enabled. We recommend that you start with the `Quick` profile.
Testing with this profile completes faster, allowing for easier configuration validation. Testing with this profile completes faster, allowing for easier configuration validation.
Provide the profile by adding the `DAST_API_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file.
Provide the profile by adding the `DAST_API_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file, 1. Provide the location of the OpenAPI Specification as either a file or URL.
substituting `Quick` for the profile you choose: Specify the location by adding the `DAST_API_OPENAPI` variable.
```yaml
stages:
- dast
include:
- template: DAST-API.gitlab-ci.yml
variables:
DAST_API_PROFILE: Quick
```
1. Provide the location of the OpenAPI specification. You can provide the specification as a file
or URL. Specify the location by adding the `DAST_API_OPENAPI` variable:
```yaml
stages:
- dast
include:
- template: DAST-API.gitlab-ci.yml
variables:
DAST_API_PROFILE: Quick
DAST_API_OPENAPI: test-api-specification.json
```
1. The target API instance's base URL is also required. Provide it by using the `DAST_API_TARGET_URL` 1. The target API instance's base URL is also required. Provide it by using the `DAST_API_TARGET_URL`
variable or an `environment_url.txt` file. variable or an `environment_url.txt` file.
...@@ -140,20 +98,20 @@ To configure DAST API scanning with an OpenAPI specification: ...@@ -140,20 +98,20 @@ To configure DAST API scanning with an OpenAPI specification:
automatically parses that file to find its scan target. You can see an automatically parses that file to find its scan target. You can see an
[example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). [example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml).
Here's an example of using `DAST_API_TARGET_URL`: Complete example configuration of using an OpenAPI Specification:
```yaml ```yaml
stages: stages:
- dast - dast
include: include:
- template: DAST-API.gitlab-ci.yml - template: DAST-API.gitlab-ci.yml
variables: variables:
DAST_API_PROFILE: Quick DAST_API_PROFILE: Quick
DAST_API_OPENAPI: test-api-specification.json DAST_API_OPENAPI: test-api-specification.json
DAST_API_TARGET_URL: http://test-deployment/ DAST_API_TARGET_URL: http://test-deployment/
``` ```
This is a minimal configuration for DAST API. From here you can: This is a minimal configuration for DAST API. From here you can:
...@@ -161,14 +119,12 @@ This is a minimal configuration for DAST API. From here you can: ...@@ -161,14 +119,12 @@ This is a minimal configuration for DAST API. From here you can:
- [Add authentication](#authentication). - [Add authentication](#authentication).
- Learn how to [handle false positives](#handling-false-positives). - Learn how to [handle false positives](#handling-false-positives).
WARNING:
**NEVER** run DAST API testing against a production server. Not only can it perform *any* function that the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting data. Only run DAST API scanning against a test server.
### HTTP Archive (HAR) ### HTTP Archive (HAR)
The [HTTP Archive format (HAR)](http://www.softwareishard.com/blog/har-12-spec/) The [HTTP Archive format (HAR)](../api_fuzzing/create_har_files.md) is an archive file format for
is an archive file format for logging HTTP transactions. When used with the GitLab DAST API scanner, HAR must contain records of calling the web API to test. The DAST API scanner extracts all the requests and logging HTTP transactions. When used with the GitLab DAST API scanner, the HAR file must contain
uses them to perform testing. records of calling the web API to test. The DAST API scanner extracts all of the requests and uses them
to perform testing.
You can use various tools to generate HAR files: You can use various tools to generate HAR files:
...@@ -182,52 +138,20 @@ WARNING: ...@@ -182,52 +138,20 @@ WARNING:
HAR files may contain sensitive information such as authentication tokens, API keys, and session HAR files may contain sensitive information such as authentication tokens, API keys, and session
cookies. We recommend that you review the HAR file contents before adding them to a repository. cookies. We recommend that you review the HAR file contents before adding them to a repository.
To configure DAST API scanning to use a HAR file: #### DAST API scanning with a HAR file
1. To use DAST API, you must [include](../../../ci/yaml/index.md#includetemplate) To configure DAST API to use a HAR file that provides information about the target API to test:
the [`DAST-API.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml)
that's provided as part of your GitLab installation. To do so, add the following to your
`.gitlab-ci.yml` file:
```yaml 1. [Include](../../../ci/yaml/index.md#includetemplate)
stages: the [`DAST-API.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml) in your `.gitlab-ci.yml` file.
- dast
include:
- template: DAST-API.gitlab-ci.yml
```
1. The [configuration file](#configuration-files) has several testing profiles defined with different checks enabled. We recommend that you start with the `Quick` profile. 1. The [configuration file](#configuration-files) has several testing profiles defined with different checks enabled. We recommend that you start with the `Quick` profile.
Testing with this profile completes faster, allowing for easier configuration validation. Testing with this profile completes faster, allowing for easier configuration validation.
Provide the profile by adding the `DAST_API_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file, Provide the profile by adding the `DAST_API_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file.
substituting `Quick` for the profile you choose:
```yaml
stages:
- dast
include:
- template: DAST-API.gitlab-ci.yml
variables:
DAST_API_PROFILE: Quick
```
1. Provide the location of the HAR file. You can provide the location as a file path 1. Provide the location of the HAR file. You can provide the location as a file path
or URL. [URL support was introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285020) in GitLab 13.10 and later. Specify the location by adding the `DAST_API_HAR` variable: or URL. [URL support was introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285020) in GitLab 13.10 and later. Specify the location by adding the `DAST_API_HAR` variable.
```yaml
stages:
- dast
include:
- template: DAST-API.gitlab-ci.yml
variables:
DAST_API_PROFILE: Quick
DAST_API_HAR: test-api-recording.har
```
1. The target API instance's base URL is also required. Provide it by using the `DAST_API_TARGET_URL` 1. The target API instance's base URL is also required. Provide it by using the `DAST_API_TARGET_URL`
variable or an `environment_url.txt` file. variable or an `environment_url.txt` file.
...@@ -238,20 +162,20 @@ To configure DAST API scanning to use a HAR file: ...@@ -238,20 +162,20 @@ To configure DAST API scanning to use a HAR file:
automatically parses that file to find its scan target. You can see an automatically parses that file to find its scan target. You can see an
[example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). [example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml).
Here's an example of using `DAST_API_TARGET_URL`: Complete example configuration of using an HAR file:
```yaml ```yaml
stages: stages:
- dast - dast
include: include:
- template: DAST-API.gitlab-ci.yml - template: DAST-API.gitlab-ci.yml
variables: variables:
DAST_API_PROFILE: Quick DAST_API_PROFILE: Quick
DAST_API_HAR: test-api-recording.har DAST_API_HAR: test-api-recording.har
DAST_API_TARGET_URL: http://test-deployment/ DAST_API_TARGET_URL: http://test-deployment/
``` ```
This is a minimal configuration for DAST API. From here you can: This is a minimal configuration for DAST API. From here you can:
...@@ -259,11 +183,6 @@ This is a minimal configuration for DAST API. From here you can: ...@@ -259,11 +183,6 @@ This is a minimal configuration for DAST API. From here you can:
- [Add authentication](#authentication). - [Add authentication](#authentication).
- Learn how to [handle false positives](#handling-false-positives). - Learn how to [handle false positives](#handling-false-positives).
WARNING:
**NEVER** run DAST API testing against a production server. Not only can it perform *any* function that
the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting
data. Only run DAST API against a test server.
### Postman Collection ### Postman Collection
The [Postman API Client](https://www.postman.com/product/api-client/) is a popular tool that The [Postman API Client](https://www.postman.com/product/api-client/) is a popular tool that
...@@ -281,51 +200,20 @@ Postman Collection files may contain sensitive information such as authenticatio ...@@ -281,51 +200,20 @@ Postman Collection files may contain sensitive information such as authenticatio
and session cookies. We recommend that you review the Postman Collection file contents before adding and session cookies. We recommend that you review the Postman Collection file contents before adding
them to a repository. them to a repository.
To configure DAST API scanning to use a Postman Collection file: #### DAST API scanning with a Postman Collection file
1. To use DAST API, you must [include](../../../ci/yaml/index.md#includetemplate) To configure DAST API to use a Postman Collection file that provides information about the target
the [`DAST-API.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml) API to test:
that's provided as part of your GitLab installation. To do so, add the following to your
`.gitlab-ci.yml` file:
```yaml 1. [Include](../../../ci/yaml/index.md#includetemplate)
stages: the [`DAST-API.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml).
- dast
include:
- template: DAST-API.gitlab-ci.yml
```
1. The [configuration file](#configuration-files) has several testing profiles defined with different checks enabled. We recommend that you start with the `Quick` profile. 1. The [configuration file](#configuration-files) has several testing profiles defined with different checks enabled. We recommend that you start with the `Quick` profile.
Testing with this profile completes faster, allowing for easier configuration validation. Testing with this profile completes faster, allowing for easier configuration validation.
Provide the profile by adding the `DAST_API_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file, Provide the profile by adding the `DAST_API_PROFILE` CI/CD variable to your `.gitlab-ci.yml` file.
substituting `Quick` for the profile you choose:
```yaml
stages:
- dast
include:
- template: DAST-API.gitlab-ci.yml
variables:
DAST_API_PROFILE: Quick
```
1. Provide the location of the Postman Collection file. You can provide the location as a file or URL. Specify the location by adding the `DAST_API_POSTMAN_COLLECTION` variable:
```yaml
stages:
- dast
include: 1. Provide the location of the Postman Collection file as either a file or URL. Specify the location by adding the `DAST_API_POSTMAN_COLLECTION` variable.
- template: DAST-API.gitlab-ci.yml
variables:
DAST_API_PROFILE: Quick
DAST_API_POSTMAN_COLLECTION: postman-collection_serviceA.json
```
1. The target API instance's base URL is also required. Provide it by using the `DAST_API_TARGET_URL` 1. The target API instance's base URL is also required. Provide it by using the `DAST_API_TARGET_URL`
variable or an `environment_url.txt` file. variable or an `environment_url.txt` file.
...@@ -336,20 +224,20 @@ To configure DAST API scanning to use a Postman Collection file: ...@@ -336,20 +224,20 @@ To configure DAST API scanning to use a Postman Collection file:
automatically parses that file to find its scan target. You can see an automatically parses that file to find its scan target. You can see an
[example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). [example of this in our Auto DevOps CI YAML](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml).
Here's an example of using `DAST_API_TARGET_URL`: Complete example configuration of using a Postman collection:
```yaml ```yaml
stages: stages:
- dast - dast
include: include:
- template: DAST-API.gitlab-ci.yml - template: DAST-API.gitlab-ci.yml
variables: variables:
DAST_API_PROFILE: Quick DAST_API_PROFILE: Quick
DAST_API_POSTMAN_COLLECTION: postman-collection_serviceA.json DAST_API_POSTMAN_COLLECTION: postman-collection_serviceA.json
DAST_API_TARGET_URL: http://test-deployment/ DAST_API_TARGET_URL: http://test-deployment/
``` ```
This is a minimal configuration for DAST API. From here you can: This is a minimal configuration for DAST API. From here you can:
...@@ -357,12 +245,7 @@ This is a minimal configuration for DAST API. From here you can: ...@@ -357,12 +245,7 @@ This is a minimal configuration for DAST API. From here you can:
- [Add authentication](#authentication). - [Add authentication](#authentication).
- Learn how to [handle false positives](#handling-false-positives). - Learn how to [handle false positives](#handling-false-positives).
WARNING: ##### Postman variables
**NEVER** run DAST API testing against a production server. Not only can it perform *any* function that
the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting
data. Only run DAST API against a test server.
#### Postman variables
Postman allows the developer to define placeholders that can be used in different parts of the Postman allows the developer to define placeholders that can be used in different parts of the
requests. These placeholders are called variables, as explained in [Using variables](https://learning.postman.com/docs/sending-requests/variables/). requests. These placeholders are called variables, as explained in [Using variables](https://learning.postman.com/docs/sending-requests/variables/).
...@@ -386,7 +269,7 @@ Postman file. For example, Postman does not export environment-scoped variables ...@@ -386,7 +269,7 @@ Postman file. For example, Postman does not export environment-scoped variables
file. file.
By default, the DAST API scanner uses the Postman file to resolve Postman variable values. If a JSON file By default, the DAST API scanner uses the Postman file to resolve Postman variable values. If a JSON file
is set in a GitLab CI environment variable `DAST_API_POSTMAN_COLLECTION_VARIABLES`, then the JSON is set in a GitLab CI/CD environment variable `DAST_API_POSTMAN_COLLECTION_VARIABLES`, then the JSON
file takes precedence to get Postman variable values. file takes precedence to get Postman variable values.
WARNING: WARNING:
...@@ -419,12 +302,12 @@ values. For example: ...@@ -419,12 +302,12 @@ values. For example:
} }
``` ```
### Authentication ## Authentication
Authentication is handled by providing the authentication token as a header or cookie. You can Authentication is handled by providing the authentication token as a header or cookie. You can
provide a script that performs an authentication flow or calculates the token. provide a script that performs an authentication flow or calculates the token.
#### HTTP Basic Authentication ### HTTP Basic Authentication
[HTTP basic authentication](https://en.wikipedia.org/wiki/Basic_access_authentication) [HTTP basic authentication](https://en.wikipedia.org/wiki/Basic_access_authentication)
is an authentication method built in to the HTTP protocol and used in conjunction with is an authentication method built in to the HTTP protocol and used in conjunction with
...@@ -454,23 +337,23 @@ variables: ...@@ -454,23 +337,23 @@ variables:
DAST_API_HTTP_PASSWORD: $TEST_API_PASSWORD DAST_API_HTTP_PASSWORD: $TEST_API_PASSWORD
``` ```
#### Bearer Tokens ### Bearer tokens
Bearer tokens are used by several different authentication mechanisms, including OAuth2 and JSON Web Bearer tokens are used by several different authentication mechanisms, including OAuth2 and JSON Web
Tokens (JWT). Bearer tokens are transmitted using the `Authorization` HTTP header. To use bearer Tokens (JWT). Bearer tokens are transmitted using the `Authorization` HTTP header. To use Bearer
tokens with DAST API, you need one of the following: tokens with DAST API, you need one of the following:
- A token that doesn't expire - A token that doesn't expire.
- A way to generate a token that lasts the length of testing - A way to generate a token that lasts the length of testing.
- A Python script that DAST API can call to generate the token - A Python script that DAST API can call to generate the token.
##### Token doesn't expire #### Token doesn't expire
If the bearer token doesn't expire, use the `DAST_API_OVERRIDES_ENV` variable to provide it. This If the Bearer token doesn't expire, use the `DAST_API_OVERRIDES_ENV` variable to provide it. This
variable's content is a JSON snippet that provides headers and cookies to add to DAST API's variable's content is a JSON snippet that provides headers and cookies to add to DAST API's
outgoing HTTP requests. outgoing HTTP requests.
Follow these steps to provide the bearer token with `DAST_API_OVERRIDES_ENV`: Follow these steps to provide the Bearer token with `DAST_API_OVERRIDES_ENV`:
1. [Create a CI/CD variable](../../../ci/variables/index.md#custom-cicd-variables), 1. [Create a CI/CD variable](../../../ci/variables/index.md#custom-cicd-variables),
for example `TEST_API_BEARERAUTH`, with the value for example `TEST_API_BEARERAUTH`, with the value
...@@ -500,9 +383,9 @@ Follow these steps to provide the bearer token with `DAST_API_OVERRIDES_ENV`: ...@@ -500,9 +383,9 @@ Follow these steps to provide the bearer token with `DAST_API_OVERRIDES_ENV`:
1. To validate that authentication is working, run an DAST API test and review the job logs 1. To validate that authentication is working, run an DAST API test and review the job logs
and the test API's application logs. and the test API's application logs.
##### Token generated at test runtime #### Token generated at test runtime
If the bearer token must be generated and doesn't expire during testing, you can provide to DAST API a file containing the token. A prior stage and job, or part of the DAST API job, can If the Bearer token must be generated and doesn't expire during testing, you can provide DAST API a file that has the token. A prior stage and job, or part of the DAST API job, can
generate this file. generate this file.
DAST API expects to receive a JSON file with the following structure: DAST API expects to receive a JSON file with the following structure:
...@@ -537,14 +420,14 @@ variables: ...@@ -537,14 +420,14 @@ variables:
To validate that authentication is working, run an DAST API test and review the job logs and To validate that authentication is working, run an DAST API test and review the job logs and
the test API's application logs. the test API's application logs.
##### Token has short expiration #### Token has short expiration
If the bearer token must be generated and expires prior to the scan's completion, you can provide a If the Bearer token must be generated and expires prior to the scan's completion, you can provide a
program or script for the DAST API scanner to execute on a provided interval. The provided script runs in program or script for the DAST API scanner to execute on a provided interval. The provided script runs in
an Alpine Linux container that has Python 3 and Bash installed. If the Python script requires an Alpine Linux container that has Python 3 and Bash installed. If the Python script requires
additional packages, it must detect this and install the packages at runtime. additional packages, it must detect this and install the packages at runtime.
The script must create a JSON file containing the bearer token in a specific format: The script must create a JSON file containing the Bearer token in a specific format:
```json ```json
{ {
...@@ -580,7 +463,7 @@ variables: ...@@ -580,7 +463,7 @@ variables:
To validate that authentication is working, run an DAST API test and review the job logs and the test API's application logs. See the [overrides section](#overrides) for more information about override commands. To validate that authentication is working, run an DAST API test and review the job logs and the test API's application logs. See the [overrides section](#overrides) for more information about override commands.
### Configuration files ## Configuration files
To get you started quickly, GitLab provides the configuration file To get you started quickly, GitLab provides the configuration file
[`gitlab-dast-api-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/dast/-/blob/master/config/gitlab-dast-api-config.yml). [`gitlab-dast-api-config.yml`](https://gitlab.com/gitlab-org/security-products/analyzers/dast/-/blob/master/config/gitlab-dast-api-config.yml).
...@@ -588,12 +471,12 @@ This file has several testing profiles that perform various numbers of tests. Th ...@@ -588,12 +471,12 @@ This file has several testing profiles that perform various numbers of tests. Th
profile increases as the test numbers go up. To use a configuration file, add it to your profile increases as the test numbers go up. To use a configuration file, add it to your
repository's root as `.gitlab/gitlab-dast-api-config.yml`. repository's root as `.gitlab/gitlab-dast-api-config.yml`.
#### Profiles ### Profiles
The following profiles are pre-defined in the default configuration file. Profiles The following profiles are pre-defined in the default configuration file. Profiles
can be added, removed, and modified by creating a custom configuration. can be added, removed, and modified by creating a custom configuration.
##### Quick #### Quick
- Application Information Check - Application Information Check
- Cleartext Authentication Check - Cleartext Authentication Check
...@@ -608,7 +491,7 @@ can be added, removed, and modified by creating a custom configuration. ...@@ -608,7 +491,7 @@ can be added, removed, and modified by creating a custom configuration.
- Token Check - Token Check
- XML Injection Check - XML Injection Check
##### Full #### Full
- Application Information Check - Application Information Check
- Cleartext AuthenticationCheck - Cleartext AuthenticationCheck
...@@ -628,7 +511,7 @@ can be added, removed, and modified by creating a custom configuration. ...@@ -628,7 +511,7 @@ can be added, removed, and modified by creating a custom configuration.
- Token Check - Token Check
- XML Injection Check - XML Injection Check
### Available CI/CD variables ## Available CI/CD variables
| CI/CD variable | Description | | CI/CD variable | Description |
|------------------------------------------------------|--------------------| |------------------------------------------------------|--------------------|
...@@ -654,7 +537,7 @@ can be added, removed, and modified by creating a custom configuration. ...@@ -654,7 +537,7 @@ can be added, removed, and modified by creating a custom configuration.
|`DAST_API_SERVICE_START_TIMEOUT` | How long to wait for target API to become available in seconds. Default is 300 seconds. | |`DAST_API_SERVICE_START_TIMEOUT` | How long to wait for target API to become available in seconds. Default is 300 seconds. |
|`DAST_API_TIMEOUT` | How long to wait for API responses in seconds. Default is 30 seconds. | |`DAST_API_TIMEOUT` | How long to wait for API responses in seconds. Default is 30 seconds. |
### Overrides ## Overrides
DAST API provides a method to add or override specific items in your request, for example: DAST API provides a method to add or override specific items in your request, for example:
...@@ -812,7 +695,7 @@ It is changed to: ...@@ -812,7 +695,7 @@ It is changed to:
You can provide this JSON document as a file or environment variable. You may also provide a command You can provide this JSON document as a file or environment variable. You may also provide a command
to generate the JSON document. The command can run at intervals to support values that expire. to generate the JSON document. The command can run at intervals to support values that expire.
#### Using a file ### Using a file
To provide the overrides JSON as a file, the `DAST_API_OVERRIDES_FILE` CI/CD variable is set. The path is relative to the job current working directory. To provide the overrides JSON as a file, the `DAST_API_OVERRIDES_FILE` CI/CD variable is set. The path is relative to the job current working directory.
...@@ -832,7 +715,7 @@ variables: ...@@ -832,7 +715,7 @@ variables:
DAST_API_OVERRIDES_FILE: dast-api-overrides.json DAST_API_OVERRIDES_FILE: dast-api-overrides.json
``` ```
#### Using a CI/CD variable ### Using a CI/CD variable
To provide the overrides JSON as a CI/CD variable, use the `DAST_API_OVERRIDES_ENV` variable. To provide the overrides JSON as a CI/CD variable, use the `DAST_API_OVERRIDES_ENV` variable.
This allows you to place the JSON as variables that can be masked and protected. This allows you to place the JSON as variables that can be masked and protected.
...@@ -870,7 +753,7 @@ variables: ...@@ -870,7 +753,7 @@ variables:
DAST_API_OVERRIDES_ENV: $SECRET_OVERRIDES DAST_API_OVERRIDES_ENV: $SECRET_OVERRIDES
``` ```
#### Using a command ### Using a command
If the value must be generated or regenerated on expiration, you can provide a program or script for If the value must be generated or regenerated on expiration, you can provide a program or script for
the DAST API scanner to execute on a specified interval. The provided command runs in an Alpine Linux the DAST API scanner to execute on a specified interval. The provided command runs in an Alpine Linux
...@@ -912,7 +795,7 @@ variables: ...@@ -912,7 +795,7 @@ variables:
DAST_API_OVERRIDES_INTERVAL: 300 DAST_API_OVERRIDES_INTERVAL: 300
``` ```
#### Debugging overrides ### Debugging overrides
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334578) in GitLab 14.8. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334578) in GitLab 14.8.
...@@ -963,10 +846,10 @@ def get_auth_response(): ...@@ -963,10 +846,10 @@ def get_auth_response():
# In our example, access token is retrieved from a given endpoint # In our example, access token is retrieved from a given endpoint
try: try:
# Performs a http request, response sample: # Performs a http request, response sample:
# { "Token" : "b5638ae7-6e77-4585-b035-7d9de2e3f6b3" } # { "Token" : "b5638ae7-6e77-4585-b035-7d9de2e3f6b3" }
response = get_auth_response() response = get_auth_response()
# Check that the request is successful. may raise `requests.exceptions.HTTPError` # Check that the request is successful. may raise `requests.exceptions.HTTPError`
response.raise_for_status() response.raise_for_status()
...@@ -993,7 +876,7 @@ except Exception as e: ...@@ -993,7 +876,7 @@ except Exception as e:
logging.error(f'Error, unknown error while retrieving access token. Error message: {e}') logging.error(f'Error, unknown error while retrieving access token. Error message: {e}')
raise raise
# computes object that holds overrides file content. # computes object that holds overrides file content.
# It uses data fetched from request # It uses data fetched from request
overrides_data = { overrides_data = {
"headers": { "headers": {
...@@ -1068,7 +951,7 @@ variables: ...@@ -1068,7 +951,7 @@ variables:
In the previous sample, you could use the script `user-pre-scan-set-up.sh` to also install new runtimes or applications that later on you could use in our overrides command. In the previous sample, you could use the script `user-pre-scan-set-up.sh` to also install new runtimes or applications that later on you could use in our overrides command.
### Exclude Paths ## Exclude Paths
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211892) in GitLab 14.0. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211892) in GitLab 14.0.
...@@ -1086,7 +969,7 @@ To verify the paths are excluded, review the `Tested Operations` and `Excluded O ...@@ -1086,7 +969,7 @@ To verify the paths are excluded, review the `Tested Operations` and `Excluded O
2021-05-27 21:51:08 [INF] API Security: ------------------------------------------------ 2021-05-27 21:51:08 [INF] API Security: ------------------------------------------------
``` ```
#### Examples ### Examples
This example excludes the `/auth` resource. This does not exclude child resources (`/auth/child`). This example excludes the `/auth` resource. This does not exclude child resources (`/auth/child`).
...@@ -1163,7 +1046,7 @@ pipelines. For more information, see the [Security Dashboard documentation](../s ...@@ -1163,7 +1046,7 @@ pipelines. For more information, see the [Security Dashboard documentation](../s
Once a vulnerability is found, you can interact with it. Read more on how to Once a vulnerability is found, you can interact with it. Read more on how to
[address the vulnerabilities](../vulnerabilities/index.md). [address the vulnerabilities](../vulnerabilities/index.md).
## Handling False Positives ### Handling False Positives
False positives can be handled in several ways: False positives can be handled in several ways:
...@@ -1175,7 +1058,7 @@ False positives can be handled in several ways: ...@@ -1175,7 +1058,7 @@ False positives can be handled in several ways:
- Turn off the Check producing the false positive. This prevents the check from generating any - Turn off the Check producing the false positive. This prevents the check from generating any
vulnerabilities. Example checks are the SQL Injection Check, and JSON Hijacking Check. vulnerabilities. Example checks are the SQL Injection Check, and JSON Hijacking Check.
### Turn off a Check #### Turn off a Check
Checks perform testing of a specific type and can be turned on and off for specific configuration Checks perform testing of a specific type and can be turned on and off for specific configuration
profiles. The provided [configuration files](#configuration-files) define several profiles that you profiles. The provided [configuration files](#configuration-files) define several profiles that you
...@@ -1233,7 +1116,7 @@ This results in the following YAML: ...@@ -1233,7 +1116,7 @@ This results in the following YAML:
- Name: XmlInjectionCheck - Name: XmlInjectionCheck
``` ```
### Turn off an Assertion for a Check #### Turn off an Assertion for a Check
Assertions detect vulnerabilities in tests produced by checks. Many checks support multiple Assertions such as Log Analysis, Response Analysis, and Status Code. When a vulnerability is found, the Assertion used is provided. To identify which Assertions are on by default, see the Checks default configuration in the configuration file. The section is called `Checks`. Assertions detect vulnerabilities in tests produced by checks. Many checks support multiple Assertions such as Log Analysis, Response Analysis, and Status Code. When a vulnerability is found, the Assertion used is provided. To identify which Assertions are on by default, see the Checks default configuration in the configuration file. The section is called `Checks`.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7