Commit a8995452 authored by Nick Gaskill's avatar Nick Gaskill

Merge branch 'docs-api-fuzzing-runner-update' into 'master'

Document OpenAPI v3 and Postman variables support in API Fuzzing

See merge request gitlab-org/gitlab!53646
parents 930656dc 915a20ea
......@@ -25,7 +25,7 @@ you can run fuzz tests as part your CI/CD workflow.
- GraphQL
- Form bodies, JSON, or XML
- One of the following assets to provide APIs to test:
- OpenAPI v2 API definition
- OpenAPI v2 or v3 API definition
- HTTP Archive (HAR) of API requests to test
- Postman Collection v2.0 or v2.1
......@@ -54,7 +54,7 @@ changes, other pipelines, or other scanners) during a scan could cause inaccurat
There are three ways to perform scans. See the configuration section for the one you wish to use:
- [OpenAPI v2 specification](#openapi-specification)
- [OpenAPI v2 or v3 specification](#openapi-specification)
- [HTTP Archive (HAR)](#http-archive-har)
- [Postman Collection v2.0 or v2.1](#postman-collection)
......@@ -71,6 +71,13 @@ API description format for REST APIs. This section shows you how to configure AP
an OpenAPI specification to provide information about the target API to test. OpenAPI specifications
are provided as a file system resource or URL.
API fuzzing uses an OpenAPI document to generate the request body. When a request body is required,
the body generation is limited to these body types:
- `application/x-www-form-urlencoded`
- `multipart/form-data`
- `application/json`
Follow these steps to configure API fuzzing in GitLab with an OpenAPI specification:
1. To use API fuzzing, you must [include](../../../ci/yaml/README.md#includetemplate)
......@@ -100,7 +107,7 @@ Follow these steps to configure API fuzzing in GitLab with an OpenAPI specificat
FUZZAPI_PROFILE: Quick-10
```
1. Provide the location of the OpenAPI v2 specification. You can provide the specification as a file
1. Provide the location of the OpenAPI specification. You can provide the specification as a file
or URL. Specify the location by adding the `FUZZAPI_OPENAPI` variable:
```yaml
......@@ -327,6 +334,60 @@ WARNING:
the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting
data. Only run fuzzing against a test server.
#### Postman variables
Postman allows the developer to define placeholders that can be used in different parts of the
requests. These placeholders are called variables, as explained in [Using variables](https://learning.postman.com/docs/sending-requests/variables/).
You can use variables to store and reuse values in your requests and scripts. For example, you can
edit the collection to add variables to the document:
![Edit collection variable tab View](img/api_fuzzing_postman_collection_edit_variable.png)
You can then use the variables in sections such as URL, headers, and others:
![Edit request using variables View](img/api_fuzzing_postman_request_edit.png)
Variables can be defined at different [scopes](https://learning.postman.com/docs/sending-requests/variables/#variable-scopes)
(for example, Global, Collection, Environment, Local, and Data). In this example, they're defined at
the Environment scope:
![Edit environment variables View](img/api_fuzzing_postman_environment_edit_variable.png)
When you export a Postman collection, only Postman collection variables are exported into the
Postman file. For example, Postman does not export environment-scoped variables into the Postman
file.
By default, the API fuzzer uses the Postman file to resolve Postman variable values. If a JSON file
is set in a GitLab CI environment variable `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`, then the JSON
file takes precedence to get Postman variable values.
Although Postman can export environment variables into a JSON file, the format is not compatible
with the JSON expected by `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`.
Here is an example of using `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`:
```yaml
include:
- template: API-Fuzzing.gitlab-ci.yml
variables:
FUZZAPI_PROFILE: Quick-10
FUZZAPI_POSTMAN_COLLECTION: postman-collection_serviceA.json
FUZZAPI_TARGET_URL: http://test-deployment/
FUZZAPI_POSTMAN_COLLECTION_VARIABLES: variable-collection-dictionary.json
```
The file `variable-collection-dictionary.json` is a JSON document. This JSON is an object with
key-value pairs for properties. The keys are the variables' names, and the values are the variables'
values. For example:
```json
{
"base_url": "http://127.0.0.1/",
"token": "Token 84816165151"
}
```
### Authentication
Authentication is handled by providing the authentication token as a header or cookie. You can
......@@ -502,6 +563,7 @@ repository's root as `.gitlab-api-fuzzing.yml`.
|[`FUZZAPI_OPENAPI`](#openapi-specification) | OpenAPI specification file or URL. |
|[`FUZZAPI_HAR`](#http-archive-har) | HTTP Archive (HAR) file. |
|[`FUZZAPI_POSTMAN_COLLECTION`](#postman-collection) | Postman Collection file. |
|[`FUZZAPI_POSTMAN_COLLECTION_VARIABLES`](#postman-variables) | Path to a JSON file to extract postman variable values. |
|[`FUZZAPI_OVERRIDES_FILE`](#overrides) | Path to a JSON file containing overrides. |
|[`FUZZAPI_OVERRIDES_ENV`](#overrides) | JSON string containing headers to override. |
|[`FUZZAPI_OVERRIDES_CMD`](#overrides) | Overrides command. |
......@@ -523,11 +585,19 @@ repository's root as `.gitlab-api-fuzzing.yml`.
### Overrides
API Fuzzing provides a method to add or override headers and cookies for all outbound HTTP requests.
API Fuzzing provides a method to add or overide specific items in your request, for example:
- Headers
- Cookies
- Query string
- Form data
- JSON nodes
- XML nodes
You can use this to inject semantic version headers, authentication, and so on. The
[authentication section](#authentication) includes examples of using overrides for that purpose.
Overrides use a JSON document to define the headers and cookies:
Overrides use a JSON document, where each type of override is represented by a JSON object:
```json
{
......@@ -538,6 +608,22 @@ Overrides use a JSON document to define the headers and cookies:
"cookies": {
"cookie1": "value",
"cookie2": "value"
},
"query": {
"query-string1": "value",
"query-string2": "value"
},
"body-form": {
"form-param1": "value",
"form-param1": "value",
},
"body-json": {
"json-path1": "value",
"json-path2": "value",
},
"body-xml" : {
"xpath1": "value",
"xpath2": "value",
}
}
```
......@@ -565,7 +651,94 @@ Example of setting both a header and cookie:
}
```
You can provide this JSON document as a file or CI/CD variable. You may also provide a command
Example usage for setting a `body-form` override:
```json
{
"body-form": {
"username": "john.doe"
}
}
```
The override engine uses `body-form` when the request body has only form-data content.
Example usage for setting a `body-json` override:
```json
{
"body-json": {
"$.credentials.access-token": "iddqd!42.$"
}
}
```
Note that each JSON property name in the object `body-json` is set to a [JSON Path](https://goessner.net/articles/JsonPath/)
expression. The JSON Path expression `$.credentials.access-token` identifies the node to be
overridden with the value `iddqd!42.$`. The override engine uses `body-json` when the request body
has only [JSON](https://www.json.org/json-en.html) content.
For example, if the body is set to the following JSON:
```json
{
"credentials" : {
"username" :"john.doe",
"access-token" : "non-valid-password"
}
}
```
It is changed to:
```json
{
"credentials" : {
"username" :"john.doe",
"access-token" : "iddqd!42.$"
}
}
```
Here's an example for setting a `body-xml` override. The first entry overrides an XML attribute and
the second entry overrides an XML element:
```json
{
"body-xml" : {
"/credentials/@isEnabled": "true",
"/credentials/access-token/text()" : "iddqd!42.$"
}
}
```
Note that each JSON property name in the object `body-xml` is set to an
[XPath v2](https://www.w3.org/TR/xpath20/)
expression. The XPath expression `/credentials/@isEnabled` identifies the attribute node to override
with the value `true`. The XPath expression `/credentials/access-token/text()` identifies the
element node to override with the value `iddqd!42.$`. The override engine uses `body-xml` when the
request body has only [XML](https://www.w3.org/XML/)
content.
For example, if the body is set to the following XML:
```xml
<credentials isEnabled="false">
<username>john.doe</username>
<access-token>non-valid-password</access-token>
</credentials>
```
It is changed to:
```xml
<credentials isEnabled="true">
<username>john.doe</username>
<access-token>iddqd!42.$</access-token>
</credentials>
```
You can provide this JSON document as a file or environment variable. You may also provide a command
to generate the JSON document. The command can run at intervals to support values that expire.
#### Using a file
......
---
title: Support OpenAPI v3 and Postman variables in API Fuzzing
merge_request: 53646
author:
type: added
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment