Commit 915a20ea authored by Herber Madrigal's avatar Herber Madrigal Committed by Nick Gaskill

Add more details in postman variables and override

Adding more details in Postman variables
- Explain what the postman variables are
- Show they look in the UI (postman)
- Differentiate  Postman export global vars
  from GitLab custom format.
- Show the CI yaml sample w/ Postman variables
- Indentation issue
- Link FUZZAPI_POSTMAN_COLLECTION_VARIABLES
  to the item 6 or the section

Adding mode details in Overrides
- Mention new capabilities (e.g. body-xml, body-json, ...)
  in 1st paragraph.
- XML override, example on how to change a element value
- XML override, example on how to change a attribute
parent 33e02c42
...@@ -25,7 +25,7 @@ you can run fuzz tests as part your CI/CD workflow. ...@@ -25,7 +25,7 @@ you can run fuzz tests as part your CI/CD workflow.
- GraphQL - GraphQL
- Form bodies, JSON, or XML - Form bodies, JSON, or XML
- One of the following assets to provide APIs to test: - One of the following assets to provide APIs to test:
- OpenAPI v2 API definition - OpenAPI v2 or v3 API definition
- HTTP Archive (HAR) of API requests to test - HTTP Archive (HAR) of API requests to test
- Postman Collection v2.0 or v2.1 - Postman Collection v2.0 or v2.1
...@@ -54,7 +54,7 @@ changes, other pipelines, or other scanners) during a scan could cause inaccurat ...@@ -54,7 +54,7 @@ changes, other pipelines, or other scanners) during a scan could cause inaccurat
There are three ways to perform scans. See the configuration section for the one you wish to use: There are three ways to perform scans. See the configuration section for the one you wish to use:
- [OpenAPI v2 specification](#openapi-specification) - [OpenAPI v2 or v3 specification](#openapi-specification)
- [HTTP Archive (HAR)](#http-archive-har) - [HTTP Archive (HAR)](#http-archive-har)
- [Postman Collection v2.0 or v2.1](#postman-collection) - [Postman Collection v2.0 or v2.1](#postman-collection)
...@@ -71,6 +71,13 @@ API description format for REST APIs. This section shows you how to configure AP ...@@ -71,6 +71,13 @@ API description format for REST APIs. This section shows you how to configure AP
an OpenAPI specification to provide information about the target API to test. OpenAPI specifications an OpenAPI specification to provide information about the target API to test. OpenAPI specifications
are provided as a file system resource or URL. are provided as a file system resource or URL.
API fuzzing uses an OpenAPI document to generate the request body. When a request body is required,
the body generation is limited to these body types:
- `application/x-www-form-urlencoded`
- `multipart/form-data`
- `application/json`
Follow these steps to configure API fuzzing in GitLab with an OpenAPI specification: Follow these steps to configure API fuzzing in GitLab with an OpenAPI specification:
1. To use API fuzzing, you must [include](../../../ci/yaml/README.md#includetemplate) 1. To use API fuzzing, you must [include](../../../ci/yaml/README.md#includetemplate)
...@@ -100,7 +107,7 @@ Follow these steps to configure API fuzzing in GitLab with an OpenAPI specificat ...@@ -100,7 +107,7 @@ Follow these steps to configure API fuzzing in GitLab with an OpenAPI specificat
FUZZAPI_PROFILE: Quick-10 FUZZAPI_PROFILE: Quick-10
``` ```
1. Provide the location of the OpenAPI v2 specification. You can provide the specification as a file 1. Provide the location of the OpenAPI specification. You can provide the specification as a file
or URL. Specify the location by adding the `FUZZAPI_OPENAPI` variable: or URL. Specify the location by adding the `FUZZAPI_OPENAPI` variable:
```yaml ```yaml
...@@ -327,6 +334,60 @@ WARNING: ...@@ -327,6 +334,60 @@ WARNING:
the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting the API can, it may also trigger bugs in the API. This includes actions like modifying and deleting
data. Only run fuzzing against a test server. data. Only run fuzzing against a test server.
#### Postman variables
Postman allows the developer to define placeholders that can be used in different parts of the
requests. These placeholders are called variables, as explained in [Using variables](https://learning.postman.com/docs/sending-requests/variables/).
You can use variables to store and reuse values in your requests and scripts. For example, you can
edit the collection to add variables to the document:
![Edit collection variable tab View](img/api_fuzzing_postman_collection_edit_variable.png)
You can then use the variables in sections such as URL, headers, and others:
![Edit request using variables View](img/api_fuzzing_postman_request_edit.png)
Variables can be defined at different [scopes](https://learning.postman.com/docs/sending-requests/variables/#variable-scopes)
(for example, Global, Collection, Environment, Local, and Data). In this example, they're defined at
the Environment scope:
![Edit environment variables View](img/api_fuzzing_postman_environment_edit_variable.png)
When you export a Postman collection, only Postman collection variables are exported into the
Postman file. For example, Postman does not export environment-scoped variables into the Postman
file.
By default, the API fuzzer uses the Postman file to resolve Postman variable values. If a JSON file
is set in a GitLab CI environment variable `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`, then the JSON
file takes precedence to get Postman variable values.
Although Postman can export environment variables into a JSON file, the format is not compatible
with the JSON expected by `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`.
Here is an example of using `FUZZAPI_POSTMAN_COLLECTION_VARIABLES`:
```yaml
include:
- template: API-Fuzzing.gitlab-ci.yml
variables:
FUZZAPI_PROFILE: Quick-10
FUZZAPI_POSTMAN_COLLECTION: postman-collection_serviceA.json
FUZZAPI_TARGET_URL: http://test-deployment/
FUZZAPI_POSTMAN_COLLECTION_VARIABLES: variable-collection-dictionary.json
```
The file `variable-collection-dictionary.json` is a JSON document. This JSON is an object with
key-value pairs for properties. The keys are the variables' names, and the values are the variables'
values. For example:
```json
{
"base_url": "http://127.0.0.1/",
"token": "Token 84816165151"
}
```
### Authentication ### Authentication
Authentication is handled by providing the authentication token as a header or cookie. You can Authentication is handled by providing the authentication token as a header or cookie. You can
...@@ -502,6 +563,7 @@ repository's root as `.gitlab-api-fuzzing.yml`. ...@@ -502,6 +563,7 @@ repository's root as `.gitlab-api-fuzzing.yml`.
|[`FUZZAPI_OPENAPI`](#openapi-specification) | OpenAPI specification file or URL. | |[`FUZZAPI_OPENAPI`](#openapi-specification) | OpenAPI specification file or URL. |
|[`FUZZAPI_HAR`](#http-archive-har) | HTTP Archive (HAR) file. | |[`FUZZAPI_HAR`](#http-archive-har) | HTTP Archive (HAR) file. |
|[`FUZZAPI_POSTMAN_COLLECTION`](#postman-collection) | Postman Collection file. | |[`FUZZAPI_POSTMAN_COLLECTION`](#postman-collection) | Postman Collection file. |
|[`FUZZAPI_POSTMAN_COLLECTION_VARIABLES`](#postman-variables) | Path to a JSON file to extract postman variable values. |
|[`FUZZAPI_OVERRIDES_FILE`](#overrides) | Path to a JSON file containing overrides. | |[`FUZZAPI_OVERRIDES_FILE`](#overrides) | Path to a JSON file containing overrides. |
|[`FUZZAPI_OVERRIDES_ENV`](#overrides) | JSON string containing headers to override. | |[`FUZZAPI_OVERRIDES_ENV`](#overrides) | JSON string containing headers to override. |
|[`FUZZAPI_OVERRIDES_CMD`](#overrides) | Overrides command. | |[`FUZZAPI_OVERRIDES_CMD`](#overrides) | Overrides command. |
...@@ -523,11 +585,19 @@ repository's root as `.gitlab-api-fuzzing.yml`. ...@@ -523,11 +585,19 @@ repository's root as `.gitlab-api-fuzzing.yml`.
### Overrides ### Overrides
API Fuzzing provides a method to add or override headers and cookies for all outbound HTTP requests. API Fuzzing provides a method to add or overide specific items in your request, for example:
- Headers
- Cookies
- Query string
- Form data
- JSON nodes
- XML nodes
You can use this to inject semantic version headers, authentication, and so on. The You can use this to inject semantic version headers, authentication, and so on. The
[authentication section](#authentication) includes examples of using overrides for that purpose. [authentication section](#authentication) includes examples of using overrides for that purpose.
Overrides use a JSON document to define the headers and cookies: Overrides use a JSON document, where each type of override is represented by a JSON object:
```json ```json
{ {
...@@ -538,6 +608,22 @@ Overrides use a JSON document to define the headers and cookies: ...@@ -538,6 +608,22 @@ Overrides use a JSON document to define the headers and cookies:
"cookies": { "cookies": {
"cookie1": "value", "cookie1": "value",
"cookie2": "value" "cookie2": "value"
},
"query": {
"query-string1": "value",
"query-string2": "value"
},
"body-form": {
"form-param1": "value",
"form-param1": "value",
},
"body-json": {
"json-path1": "value",
"json-path2": "value",
},
"body-xml" : {
"xpath1": "value",
"xpath2": "value",
} }
} }
``` ```
...@@ -565,7 +651,94 @@ Example of setting both a header and cookie: ...@@ -565,7 +651,94 @@ Example of setting both a header and cookie:
} }
``` ```
You can provide this JSON document as a file or CI/CD variable. You may also provide a command Example usage for setting a `body-form` override:
```json
{
"body-form": {
"username": "john.doe"
}
}
```
The override engine uses `body-form` when the request body has only form-data content.
Example usage for setting a `body-json` override:
```json
{
"body-json": {
"$.credentials.access-token": "iddqd!42.$"
}
}
```
Note that each JSON property name in the object `body-json` is set to a [JSON Path](https://goessner.net/articles/JsonPath/)
expression. The JSON Path expression `$.credentials.access-token` identifies the node to be
overridden with the value `iddqd!42.$`. The override engine uses `body-json` when the request body
has only [JSON](https://www.json.org/json-en.html) content.
For example, if the body is set to the following JSON:
```json
{
"credentials" : {
"username" :"john.doe",
"access-token" : "non-valid-password"
}
}
```
It is changed to:
```json
{
"credentials" : {
"username" :"john.doe",
"access-token" : "iddqd!42.$"
}
}
```
Here's an example for setting a `body-xml` override. The first entry overrides an XML attribute and
the second entry overrides an XML element:
```json
{
"body-xml" : {
"/credentials/@isEnabled": "true",
"/credentials/access-token/text()" : "iddqd!42.$"
}
}
```
Note that each JSON property name in the object `body-xml` is set to an
[XPath v2](https://www.w3.org/TR/xpath20/)
expression. The XPath expression `/credentials/@isEnabled` identifies the attribute node to override
with the value `true`. The XPath expression `/credentials/access-token/text()` identifies the
element node to override with the value `iddqd!42.$`. The override engine uses `body-xml` when the
request body has only [XML](https://www.w3.org/XML/)
content.
For example, if the body is set to the following XML:
```xml
<credentials isEnabled="false">
<username>john.doe</username>
<access-token>non-valid-password</access-token>
</credentials>
```
It is changed to:
```xml
<credentials isEnabled="true">
<username>john.doe</username>
<access-token>iddqd!42.$</access-token>
</credentials>
```
You can provide this JSON document as a file or environment variable. You may also provide a command
to generate the JSON document. The command can run at intervals to support values that expire. to generate the JSON document. The command can run at intervals to support values that expire.
#### Using a file #### Using a file
......
---
title: Support OpenAPI v3 and Postman variables in API Fuzzing
merge_request: 53646
author:
type: added
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment