Commit 9d3eeca6 authored by Marcel Amirault's avatar Marcel Amirault

Merge branch '276207-vale-substitutions-warning-part-06' into 'master'

Docs: clean up some Vale substitution warnings

See merge request gitlab-org/gitlab!52975
parents 0c63329d decbc2ab
...@@ -35,21 +35,21 @@ verification methods: ...@@ -35,21 +35,21 @@ verification methods:
| Git | Object pools for forked project deduplication | Geo with Gitaly | _Not implemented_ | | Git | Object pools for forked project deduplication | Geo with Gitaly | _Not implemented_ |
| Git | Project Snippets | Geo with Gitaly | _Not implemented_ | | Git | Project Snippets | Geo with Gitaly | _Not implemented_ |
| Git | Personal Snippets | Geo with Gitaly | _Not implemented_ | | Git | Personal Snippets | Geo with Gitaly | _Not implemented_ |
| Blobs | User uploads _(filesystem)_ | Geo with API | _Not implemented_ | | Blobs | User uploads _(file system)_ | Geo with API | _Not implemented_ |
| Blobs | User uploads _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | | Blobs | User uploads _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ |
| Blobs | LFS objects _(filesystem)_ | Geo with API | _Not implemented_ | | Blobs | LFS objects _(file system)_ | Geo with API | _Not implemented_ |
| Blobs | LFS objects _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | | Blobs | LFS objects _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ |
| Blobs | CI job artifacts _(filesystem)_ | Geo with API | _Not implemented_ | | Blobs | CI job artifacts _(file system)_ | Geo with API | _Not implemented_ |
| Blobs | CI job artifacts _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | | Blobs | CI job artifacts _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ |
| Blobs | Archived CI build traces _(filesystem)_ | Geo with API | _Not implemented_ | | Blobs | Archived CI build traces _(file system)_ | Geo with API | _Not implemented_ |
| Blobs | Archived CI build traces _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | | Blobs | Archived CI build traces _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ |
| Blobs | Container registry _(filesystem)_ | Geo with API/Docker API | _Not implemented_ | | Blobs | Container registry _(file system)_ | Geo with API/Docker API | _Not implemented_ |
| Blobs | Container registry _(object storage)_ | Geo with API/Managed/Docker API (*2*) | _Not implemented_ | | Blobs | Container registry _(object storage)_ | Geo with API/Managed/Docker API (*2*) | _Not implemented_ |
| Blobs | Package registry _(filesystem)_ | Geo with API | _Not implemented_ | | Blobs | Package registry _(file system)_ | Geo with API | _Not implemented_ |
| Blobs | Package registry _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | | Blobs | Package registry _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ |
| Blobs | Versioned Terraform State _(filesystem)_ | Geo with API | _Not implemented_ | | Blobs | Versioned Terraform State _(file system)_ | Geo with API | _Not implemented_ |
| Blobs | Versioned Terraform State _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | | Blobs | Versioned Terraform State _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ |
| Blobs | External Merge Request Diffs _(filesystem)_ | Geo with API | _Not implemented_ | | Blobs | External Merge Request Diffs _(file system)_ | Geo with API | _Not implemented_ |
| Blobs | External Merge Request Diffs _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | | Blobs | External Merge Request Diffs _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ |
- (*1*): Redis replication can be used as part of HA with Redis sentinel. It's not used between Geo nodes. - (*1*): Redis replication can be used as part of HA with Redis sentinel. It's not used between Geo nodes.
...@@ -63,8 +63,8 @@ is responsible for allowing access and operations on the locally stored Git repo ...@@ -63,8 +63,8 @@ is responsible for allowing access and operations on the locally stored Git repo
on a machine with a single disk, multiple disks mounted as a single mount-point (like with a RAID array), on a machine with a single disk, multiple disks mounted as a single mount-point (like with a RAID array),
or using LVM. or using LVM.
It requires no special filesystem and can work with NFS or a mounted Storage Appliance (there may be It requires no special file system and can work with NFS or a mounted Storage Appliance (there may be
performance limitations when using a remote filesystem). performance limitations when using a remote file system).
Communication is done via Gitaly's own gRPC API. There are three possible ways of synchronization: Communication is done via Gitaly's own gRPC API. There are three possible ways of synchronization:
...@@ -88,13 +88,13 @@ Both types will be synced to a secondary node. ...@@ -88,13 +88,13 @@ Both types will be synced to a secondary node.
GitLab stores files and blobs such as Issue attachments or LFS objects into either: GitLab stores files and blobs such as Issue attachments or LFS objects into either:
- The filesystem in a specific location. - The file system in a specific location.
- An [Object Storage](../../object_storage.md) solution. Object Storage solutions can be: - An [Object Storage](../../object_storage.md) solution. Object Storage solutions can be:
- Cloud based like Amazon S3 Google Cloud Storage. - Cloud based like Amazon S3 Google Cloud Storage.
- Hosted by you (like MinIO). - Hosted by you (like MinIO).
- A Storage Appliance that exposes an Object Storage-compatible API. - A Storage Appliance that exposes an Object Storage-compatible API.
When using the filesystem store instead of Object Storage, you need to use network mounted filesystems When using the file system store instead of Object Storage, you need to use network mounted file systems
to run GitLab when using more than one server. to run GitLab when using more than one server.
With respect to replication and verification: With respect to replication and verification:
......
...@@ -29,7 +29,7 @@ WARNING: ...@@ -29,7 +29,7 @@ WARNING:
This is a [**beta** feature](https://about.gitlab.com/handbook/product/#beta) and is not ready yet for production use at any scale. The main limitations are a lack of testing at scale and no verification of any replicated data. This is a [**beta** feature](https://about.gitlab.com/handbook/product/#beta) and is not ready yet for production use at any scale. The main limitations are a lack of testing at scale and no verification of any replicated data.
**Secondary** nodes can replicate files stored on the **primary** node regardless of **Secondary** nodes can replicate files stored on the **primary** node regardless of
whether they are stored on the local filesystem or in object storage. whether they are stored on the local file system or in object storage.
To enable GitLab replication, you must: To enable GitLab replication, you must:
......
...@@ -194,7 +194,7 @@ from [owasp.org](https://owasp.org/). ...@@ -194,7 +194,7 @@ from [owasp.org](https://owasp.org/).
administrator via SSH, and then back out to the **secondary** node in the same manner. administrator via SSH, and then back out to the **secondary** node in the same manner.
In particular, this includes the PostgreSQL replication credentials and a secret In particular, this includes the PostgreSQL replication credentials and a secret
key (`db_key_base`) which is used to decrypt certain columns in the database. key (`db_key_base`) which is used to decrypt certain columns in the database.
The `db_key_base` secret is stored unencrypted on the filesystem, in The `db_key_base` secret is stored unencrypted on the file system, in
`/etc/gitlab/gitlab-secrets.json`, along with a number of other secrets. There is `/etc/gitlab/gitlab-secrets.json`, along with a number of other secrets. There is
no at-rest protection for them. no at-rest protection for them.
...@@ -217,7 +217,7 @@ from [owasp.org](https://owasp.org/). ...@@ -217,7 +217,7 @@ from [owasp.org](https://owasp.org/).
- **Secondary** nodes and **primary** nodes interact via HTTP/HTTPS (secured with JSON web - **Secondary** nodes and **primary** nodes interact via HTTP/HTTPS (secured with JSON web
tokens) and via PostgreSQL streaming replication. tokens) and via PostgreSQL streaming replication.
- Within a **primary** node or **secondary** node, the SSOT is the filesystem and the database - Within a **primary** node or **secondary** node, the SSOT is the file system and the database
(including Geo tracking database on **secondary** node). The various internal components (including Geo tracking database on **secondary** node). The various internal components
are orchestrated to make alterations to these stores. are orchestrated to make alterations to these stores.
...@@ -231,7 +231,7 @@ from [owasp.org](https://owasp.org/). ...@@ -231,7 +231,7 @@ from [owasp.org](https://owasp.org/).
### What data is or may need to be encrypted and what key management requirements have been defined? ### What data is or may need to be encrypted and what key management requirements have been defined?
- Neither **primary** nodes or **secondary** nodes encrypt Git repository or filesystem data at - Neither **primary** nodes or **secondary** nodes encrypt Git repository or file system data at
rest. A subset of database columns are encrypted at rest using the `db_otp_key`. rest. A subset of database columns are encrypted at rest using the `db_otp_key`.
- A static secret shared across all hosts in a GitLab deployment. - A static secret shared across all hosts in a GitLab deployment.
- In transit, data should be encrypted, although the application does permit - In transit, data should be encrypted, although the application does permit
...@@ -287,5 +287,5 @@ from [owasp.org](https://owasp.org/). ...@@ -287,5 +287,5 @@ from [owasp.org](https://owasp.org/).
### What application auditing requirements have been defined? How are audit and debug logs accessed, stored, and secured? ### What application auditing requirements have been defined? How are audit and debug logs accessed, stored, and secured?
- Structured JSON log is written to the filesystem, and can also be ingested - Structured JSON log is written to the file system, and can also be ingested
into a Kibana installation for further analysis. into a Kibana installation for further analysis.
...@@ -113,7 +113,7 @@ In this example: ...@@ -113,7 +113,7 @@ In this example:
- Repositories are stored on a virtual storage called `storage-1`. - Repositories are stored on a virtual storage called `storage-1`.
- Three Gitaly nodes provide `storage-1` access: `gitaly-1`, `gitaly-2`, and `gitaly-3`. - Three Gitaly nodes provide `storage-1` access: `gitaly-1`, `gitaly-2`, and `gitaly-3`.
- The three Gitaly nodes store data on their filesystems. - The three Gitaly nodes store data on their file systems.
### Virtual storage or direct Gitaly storage ### Virtual storage or direct Gitaly storage
......
...@@ -188,7 +188,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. ...@@ -188,7 +188,7 @@ Learn how to install, configure, update, and maintain your GitLab instance.
## Git configuration options ## Git configuration options
- [Server hooks](server_hooks.md): Server hooks (on the filesystem) for when webhooks aren't enough. - [Server hooks](server_hooks.md): Server hooks (on the file system) for when webhooks aren't enough.
- [Git LFS configuration](lfs/index.md): Learn how to configure LFS for GitLab. - [Git LFS configuration](lfs/index.md): Learn how to configure LFS for GitLab.
- [Housekeeping](housekeeping.md): Keep your Git repositories tidy and fast. - [Housekeeping](housekeeping.md): Keep your Git repositories tidy and fast.
- [Configuring Git Protocol v2](git_protocol.md): Git protocol version 2 support. - [Configuring Git Protocol v2](git_protocol.md): Git protocol version 2 support.
......
...@@ -20,10 +20,10 @@ From GitLab 14.0, technical support for NFS for Git repositories ...@@ -20,10 +20,10 @@ From GitLab 14.0, technical support for NFS for Git repositories
will no longer be provided. Upgrade to [Gitaly Cluster](gitaly/praefect.md) will no longer be provided. Upgrade to [Gitaly Cluster](gitaly/praefect.md)
as soon as possible. as soon as possible.
Filesystem performance can impact overall GitLab performance, especially for File system performance can impact overall GitLab performance, especially for
actions that read or write to Git repositories. For steps you can use to test actions that read or write to Git repositories. For steps you can use to test
filesystem performance, see file system performance, see
[Filesystem Performance Benchmarking](operations/filesystem_benchmarking.md). [File system Performance Benchmarking](operations/filesystem_benchmarking.md).
## Known kernel version incompatibilities ## Known kernel version incompatibilities
...@@ -408,7 +408,7 @@ For supported database architecture, see our documentation about ...@@ -408,7 +408,7 @@ For supported database architecture, see our documentation about
### Finding the requests that are being made to NFS ### Finding the requests that are being made to NFS
In case of NFS-related problems, it can be helpful to trace In case of NFS-related problems, it can be helpful to trace
the filesystem requests that are being made by using `perf`: the file system requests that are being made by using `perf`:
```shell ```shell
sudo perf trace -e 'nfs4:*' -p $(pgrep -fd ',' puma && pgrep -fd ',' unicorn) sudo perf trace -e 'nfs4:*' -p $(pgrep -fd ',' puma && pgrep -fd ',' unicorn)
......
...@@ -557,11 +557,11 @@ supported by consolidated configuration form, refer to the following guides: ...@@ -557,11 +557,11 @@ supported by consolidated configuration form, refer to the following guides:
| [Terraform state files](terraform_state.md#using-object-storage) | Yes | | [Terraform state files](terraform_state.md#using-object-storage) | Yes |
| [GitLab Pages content](pages/index.md#using-object-storage) | Yes | | [GitLab Pages content](pages/index.md#using-object-storage) | Yes |
### Other alternatives to filesystem storage ### Other alternatives to file system storage
If you're working to [scale out](reference_architectures/index.md) your GitLab implementation, If you're working to [scale out](reference_architectures/index.md) your GitLab implementation,
or add fault tolerance and redundancy, you may be or add fault tolerance and redundancy, you may be
looking at removing dependencies on block or network filesystems. looking at removing dependencies on block or network file systems.
See the following additional guides and See the following additional guides and
[note that Pages requires disk storage](#gitlab-pages-requires-nfs): [note that Pages requires disk storage](#gitlab-pages-requires-nfs):
......
...@@ -4,15 +4,15 @@ group: Distribution ...@@ -4,15 +4,15 @@ group: Distribution
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
--- ---
# Filesystem Performance Benchmarking # File system Performance Benchmarking
Filesystem performance has a big impact on overall GitLab performance, File system performance has a big impact on overall GitLab performance,
especially for actions that read or write to Git repositories. This information especially for actions that read or write to Git repositories. This information
will help benchmark filesystem performance against known good and bad real-world will help benchmark file system performance against known good and bad real-world
systems. systems.
Normally when talking about filesystem performance the biggest concern is Normally when talking about file system performance the biggest concern is
with Network Filesystems (NFS). However, even some local disks can have slow with Network File Systems (NFS). However, even some local disks can have slow
I/O. The information on this page can be used for either scenario. I/O. The information on this page can be used for either scenario.
## Executing benchmarks ## Executing benchmarks
...@@ -77,7 +77,7 @@ available on the system. It's possible to receive good results on this ...@@ -77,7 +77,7 @@ available on the system. It's possible to receive good results on this
test but still have poor performance due to read speed and various other test but still have poor performance due to read speed and various other
factors. factors.
The following one-line commands provide a quick benchmark for filesystem write and read The following one-line commands provide a quick benchmark for file system write and read
performance. This will write 1,000 small files to the directory in which it is performance. This will write 1,000 small files to the directory in which it is
executed, and then read the same 1,000 files. executed, and then read the same 1,000 files.
...@@ -125,4 +125,4 @@ sys 0m1.663s ...@@ -125,4 +125,4 @@ sys 0m1.663s
``` ```
From experience with multiple customers, this task should take under 10 From experience with multiple customers, this task should take under 10
seconds to indicate good filesystem performance. seconds to indicate good file system performance.
...@@ -27,10 +27,10 @@ Keep your GitLab instance up and running smoothly. ...@@ -27,10 +27,10 @@ Keep your GitLab instance up and running smoothly.
indexed lookup to the GitLab database](fast_ssh_key_lookup.md), and/or indexed lookup to the GitLab database](fast_ssh_key_lookup.md), and/or
by [doing away with user SSH keys stored on GitLab entirely in favor by [doing away with user SSH keys stored on GitLab entirely in favor
of SSH certificates](ssh_certificates.md). of SSH certificates](ssh_certificates.md).
- [Filesystem Performance Benchmarking](filesystem_benchmarking.md): Filesystem - [File System Performance Benchmarking](filesystem_benchmarking.md): File system
performance can have a big impact on GitLab performance, especially for actions performance can have a big impact on GitLab performance, especially for actions
that read or write Git repositories. This information will help benchmark that read or write Git repositories. This information will help benchmark
filesystem performance against known good and bad real-world systems. file system performance against known good and bad real-world systems.
- [The Rails Console](rails_console.md): Provides a way to interact with your GitLab instance from the command line. - [The Rails Console](rails_console.md): Provides a way to interact with your GitLab instance from the command line.
Used for troubleshooting a problem or retrieving some data that can only be done through direct access to GitLab. Used for troubleshooting a problem or retrieving some data that can only be done through direct access to GitLab.
- [ChatOps Scripts](https://gitlab.com/gitlab-com/chatops): The GitLab.com Infrastructure team uses this repository to house - [ChatOps Scripts](https://gitlab.com/gitlab-com/chatops): The GitLab.com Infrastructure team uses this repository to house
......
...@@ -845,7 +845,7 @@ open /etc/ssl/ca-bundle.pem: permission denied ...@@ -845,7 +845,7 @@ open /etc/ssl/ca-bundle.pem: permission denied
``` ```
The use of a `chroot` jail makes this error misleading, as it is not The use of a `chroot` jail makes this error misleading, as it is not
referring to `/etc/ssl` on the root filesystem. referring to `/etc/ssl` on the root file system.
The fix is to correct the source file permissions and restart Pages: The fix is to correct the source file permissions and restart Pages:
......
...@@ -266,7 +266,7 @@ Rough numbers for calls to `open` and `openat` (used to access files) on various ...@@ -266,7 +266,7 @@ Rough numbers for calls to `open` and `openat` (used to access files) on various
Slow storage can cause the dreaded `DeadlineExceeded` error in Gitaly. Slow storage can cause the dreaded `DeadlineExceeded` error in Gitaly.
Also [see this entry](../operations/filesystem_benchmarking.md) Also [see this entry](../operations/filesystem_benchmarking.md)
in the handbook for quick tests customers can perform to check their filesystem performance. in the handbook for quick tests customers can perform to check their file system performance.
Keep in mind that timing information from `strace` is often somewhat inaccurate, so Keep in mind that timing information from `strace` is often somewhat inaccurate, so
small differences should not be considered significant. small differences should not be considered significant.
......
...@@ -179,9 +179,9 @@ POST /groups/:id/wikis/attachments ...@@ -179,9 +179,9 @@ POST /groups/:id/wikis/attachments
| `file` | string | yes | The attachment to be uploaded | | `file` | string | yes | The attachment to be uploaded |
| `branch` | string | no | The name of the branch. Defaults to the wiki repository default branch | | `branch` | string | no | The name of the branch. Defaults to the wiki repository default branch |
To upload a file from your filesystem, use the `--form` argument. This causes To upload a file from your file system, use the `--form` argument. This causes
cURL to post data using the header `Content-Type: multipart/form-data`. cURL to post data using the header `Content-Type: multipart/form-data`.
The `file=` parameter must point to a file on your filesystem and be preceded The `file=` parameter must point to a file on your file system and be preceded
by `@`. For example: by `@`. For example:
```shell ```shell
......
...@@ -175,9 +175,9 @@ POST /projects/:id/wikis/attachments ...@@ -175,9 +175,9 @@ POST /projects/:id/wikis/attachments
| `file` | string | yes | The attachment to be uploaded | | `file` | string | yes | The attachment to be uploaded |
| `branch` | string | no | The name of the branch. Defaults to the wiki repository default branch | | `branch` | string | no | The name of the branch. Defaults to the wiki repository default branch |
To upload a file from your filesystem, use the `--form` argument. This causes To upload a file from your file system, use the `--form` argument. This causes
cURL to post data using the header `Content-Type: multipart/form-data`. cURL to post data using the header `Content-Type: multipart/form-data`.
The `file=` parameter must point to a file on your filesystem and be preceded The `file=` parameter must point to a file on your file system and be preceded
by `@`. For example: by `@`. For example:
```shell ```shell
......
...@@ -801,7 +801,7 @@ NOTE: ...@@ -801,7 +801,7 @@ NOTE:
The shared runners on GitLab.com use the `overlay2` driver by default. The shared runners on GitLab.com use the `overlay2` driver by default.
By default, when using `docker:dind`, Docker uses the `vfs` storage driver which By default, when using `docker:dind`, Docker uses the `vfs` storage driver which
copies the filesystem on every run. This is a disk-intensive operation copies the file system on every run. This is a disk-intensive operation
which can be avoided if a different driver is used, for example `overlay2`. which can be avoided if a different driver is used, for example `overlay2`.
### Requirements ### Requirements
......
...@@ -111,7 +111,7 @@ with [domain expertise](#domain-experts). ...@@ -111,7 +111,7 @@ with [domain expertise](#domain-experts).
[GitLab Licensing and Compatibility documentation](licensing.md). [GitLab Licensing and Compatibility documentation](licensing.md).
1. If your merge request includes adding a new UI/UX paradigm (*1*), it must be 1. If your merge request includes adding a new UI/UX paradigm (*1*), it must be
**approved by a [UX lead](https://about.gitlab.com/company/team/)**. **approved by a [UX lead](https://about.gitlab.com/company/team/)**.
1. If your merge request includes a new dependency or a filesystem change, it must be 1. If your merge request includes a new dependency or a file system change, it must be
**approved by a [Distribution team member](https://about.gitlab.com/company/team/)**. See how to work with the [Distribution team](https://about.gitlab.com/handbook/engineering/development/enablement/distribution/#how-to-work-with-distribution) for more details. **approved by a [Distribution team member](https://about.gitlab.com/company/team/)**. See how to work with the [Distribution team](https://about.gitlab.com/handbook/engineering/development/enablement/distribution/#how-to-work-with-distribution) for more details.
1. If your merge request includes documentation changes, it must be **approved 1. If your merge request includes documentation changes, it must be **approved
by a [Technical writer](https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments)**, based on by a [Technical writer](https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments)**, based on
......
...@@ -212,7 +212,7 @@ the contribution acceptance criteria below: ...@@ -212,7 +212,7 @@ the contribution acceptance criteria below:
1. Changes do not degrade performance: 1. Changes do not degrade performance:
- Avoid repeated polling of endpoints that require a significant amount of overhead. - Avoid repeated polling of endpoints that require a significant amount of overhead.
- Check for N+1 queries via the SQL log or [`QueryRecorder`](../merge_request_performance_guidelines.md). - Check for N+1 queries via the SQL log or [`QueryRecorder`](../merge_request_performance_guidelines.md).
- Avoid repeated access of the filesystem. - Avoid repeated access of the file system.
- Use [polling with ETag caching](../polling.md) if needed to support real-time features. - Use [polling with ETag caching](../polling.md) if needed to support real-time features.
1. If the merge request adds any new libraries (gems, JavaScript libraries, etc.), 1. If the merge request adds any new libraries (gems, JavaScript libraries, etc.),
they should conform to our [Licensing guidelines](../licensing.md). See those they should conform to our [Licensing guidelines](../licensing.md). See those
......
...@@ -75,7 +75,7 @@ your models _unless_ absolutely required and only when approved by database ...@@ -75,7 +75,7 @@ your models _unless_ absolutely required and only when approved by database
specialists. For example, if each row in a table has a corresponding file on a specialists. For example, if each row in a table has a corresponding file on a
file system it may be tempting to add a `after_destroy` hook. This however file system it may be tempting to add a `after_destroy` hook. This however
introduces non database logic to a model, and means we can no longer rely on introduces non database logic to a model, and means we can no longer rely on
foreign keys to remove the data as this would result in the filesystem data foreign keys to remove the data as this would result in the file system data
being left behind. In such a case you should use a service class instead that being left behind. In such a case you should use a service class instead that
takes care of removing non database data. takes care of removing non database data.
......
...@@ -423,14 +423,14 @@ Path Traversal attacks can lead to multiple critical and high severity issues, l ...@@ -423,14 +423,14 @@ Path Traversal attacks can lead to multiple critical and high severity issues, l
### When to consider ### When to consider
When working with user-controlled filenames/paths and filesystem APIs. When working with user-controlled filenames/paths and file system APIs.
### Mitigation and prevention ### Mitigation and prevention
In order to prevent Path Traversal vulnerabilities, user-controlled filenames or paths should be validated before being processed. In order to prevent Path Traversal vulnerabilities, user-controlled filenames or paths should be validated before being processed.
- Comparing user input against an allowlist of allowed values or verifying that it only contains allowed characters. - Comparing user input against an allowlist of allowed values or verifying that it only contains allowed characters.
- After validating the user supplied input, it should be appended to the base directory and the path should be canonicalized using the filesystem API. - After validating the user supplied input, it should be appended to the base directory and the path should be canonicalized using the file system API.
#### GitLab specific validations #### GitLab specific validations
......
...@@ -32,7 +32,7 @@ disk in a temporary file so you can perform some checks on it. When the checks ...@@ -32,7 +32,7 @@ disk in a temporary file so you can perform some checks on it. When the checks
pass, you make the file official. In scenarios like this please follow these pass, you make the file official. In scenarios like this please follow these
rules: rules:
- Store the temporary file under `shared/tmp`, i.e. on the same filesystem you - Store the temporary file under `shared/tmp`, i.e. on the same file system you
want the official file to be on. want the official file to be on.
- Use move/rename operations when operating on the file instead of copy - Use move/rename operations when operating on the file instead of copy
operations where possible, because renaming a file is much faster than operations where possible, because renaming a file is much faster than
......
...@@ -202,7 +202,7 @@ When using regular expressions to validate user input that is passed as an argum ...@@ -202,7 +202,7 @@ When using regular expressions to validate user input that is passed as an argum
If you don't, an attacker could use this to execute commands with potentially harmful effect. If you don't, an attacker could use this to execute commands with potentially harmful effect.
For example, when a project's `import_url` is validated like below, the user could trick GitLab into cloning from a Git repository on the local filesystem. For example, when a project's `import_url` is validated like below, the user could trick GitLab into cloning from a Git repository on the local file system.
```ruby ```ruby
validates :import_url, format: { with: URI.regexp(%w(ssh git http https)) } validates :import_url, format: { with: URI.regexp(%w(ssh git http https)) }
......
...@@ -16,7 +16,7 @@ of GitLab on which it was created. The best way to migrate your repositories ...@@ -16,7 +16,7 @@ of GitLab on which it was created. The best way to migrate your repositories
from one server to another is through backup restore. from one server to another is through backup restore.
WARNING: WARNING:
GitLab doesn't back up items that aren't stored in the filesystem. If you're GitLab doesn't back up items that aren't stored in the file system. If you're
using [object storage](../administration/object_storage.md), be sure to enable using [object storage](../administration/object_storage.md), be sure to enable
backups with your object storage provider, if desired. backups with your object storage provider, if desired.
...@@ -1042,12 +1042,12 @@ is being discussed in [issue #17517](https://gitlab.com/gitlab-org/gitlab/-/issu ...@@ -1042,12 +1042,12 @@ is being discussed in [issue #17517](https://gitlab.com/gitlab-org/gitlab/-/issu
If your GitLab server contains a lot of Git repository data, you may find the If your GitLab server contains a lot of Git repository data, you may find the
GitLab backup script to be too slow. In this case you can consider using GitLab backup script to be too slow. In this case you can consider using
filesystem snapshots as part of your backup strategy. file system snapshots as part of your backup strategy.
Example: Amazon EBS Example: Amazon EBS
> A GitLab server using Omnibus GitLab hosted on Amazon AWS. > A GitLab server using Omnibus GitLab hosted on Amazon AWS.
> An EBS drive containing an ext4 filesystem is mounted at `/var/opt/gitlab`. > An EBS drive containing an ext4 file system is mounted at `/var/opt/gitlab`.
> In this case you could make an application backup by taking an EBS snapshot. > In this case you could make an application backup by taking an EBS snapshot.
> The backup includes all repositories, uploads and PostgreSQL data. > The backup includes all repositories, uploads and PostgreSQL data.
...@@ -1055,7 +1055,7 @@ Example: LVM snapshots + rsync ...@@ -1055,7 +1055,7 @@ Example: LVM snapshots + rsync
> A GitLab server using Omnibus GitLab, with an LVM logical volume mounted at `/var/opt/gitlab`. > A GitLab server using Omnibus GitLab, with an LVM logical volume mounted at `/var/opt/gitlab`.
> Replicating the `/var/opt/gitlab` directory using rsync would not be reliable because too many files would change while rsync is running. > Replicating the `/var/opt/gitlab` directory using rsync would not be reliable because too many files would change while rsync is running.
> Instead of rsync-ing `/var/opt/gitlab`, we create a temporary LVM snapshot, which we mount as a read-only filesystem at `/mnt/gitlab_backup`. > Instead of rsync-ing `/var/opt/gitlab`, we create a temporary LVM snapshot, which we mount as a read-only file system at `/mnt/gitlab_backup`.
> Now we can have a longer running rsync job which creates a consistent replica on the remote server. > Now we can have a longer running rsync job which creates a consistent replica on the remote server.
> The replica includes all repositories, uploads and PostgreSQL data. > The replica includes all repositories, uploads and PostgreSQL data.
...@@ -1370,7 +1370,7 @@ To get your registry working again: ...@@ -1370,7 +1370,7 @@ To get your registry working again:
sudo chown -R registry:registry /var/opt/gitlab/gitlab-rails/shared/registry/docker sudo chown -R registry:registry /var/opt/gitlab/gitlab-rails/shared/registry/docker
``` ```
If you changed the default filesystem location for the registry, run `chown` If you changed the default file system location for the registry, run `chown`
against your custom location, instead of `/var/opt/gitlab/gitlab-rails/shared/registry/docker`. against your custom location, instead of `/var/opt/gitlab/gitlab-rails/shared/registry/docker`.
### Backup fails to complete with Gzip error ### Backup fails to complete with Gzip error
......
...@@ -74,7 +74,7 @@ I, [2020-01-08T20:51:17.148765 #43765] INFO -- : Removed unreferenced LFS files ...@@ -74,7 +74,7 @@ I, [2020-01-08T20:51:17.148765 #43765] INFO -- : Removed unreferenced LFS files
Clean up project upload files if they don't exist in GitLab database. Clean up project upload files if they don't exist in GitLab database.
### Clean up project upload files from filesystem ### Clean up project upload files from file system
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20863) in GitLab 11.2. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20863) in GitLab 11.2.
......
...@@ -15,7 +15,7 @@ type: concepts, howto ...@@ -15,7 +15,7 @@ type: concepts, howto
GitLab provides liveness and readiness probes to indicate service health and GitLab provides liveness and readiness probes to indicate service health and
reachability to required services. These probes report on the status of the reachability to required services. These probes report on the status of the
database connection, Redis connection, and access to the filesystem. These database connection, Redis connection, and access to the file system. These
endpoints [can be provided to schedulers like Kubernetes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) to hold endpoints [can be provided to schedulers like Kubernetes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) to hold
traffic until the system is ready or restart the container as needed. traffic until the system is ready or restart the container as needed.
......
...@@ -69,7 +69,7 @@ Examples of both configurations can be found here: ...@@ -69,7 +69,7 @@ Examples of both configurations can be found here:
The [OpenAPI Specification](https://www.openapis.org/) (formerly the Swagger Specification) is an The [OpenAPI Specification](https://www.openapis.org/) (formerly the Swagger Specification) is an
API description format for REST APIs. This section shows you how to configure API fuzzing by using API description format for REST APIs. This section shows you how to configure API fuzzing by using
an OpenAPI specification to provide information about the target API to test. OpenAPI specifications an OpenAPI specification to provide information about the target API to test. OpenAPI specifications
are provided as a filesystem resource or URL. are provided as a file system resource or URL.
Follow these steps to configure API fuzzing in GitLab with an OpenAPI specification: Follow these steps to configure API fuzzing in GitLab with an OpenAPI specification:
......
...@@ -30,7 +30,7 @@ There are two approaches to SVN to Git migration: ...@@ -30,7 +30,7 @@ There are two approaches to SVN to Git migration:
migration. It creates a writable Git mirror of a local or remote Subversion migration. It creates a writable Git mirror of a local or remote Subversion
repository and that way you can use both Subversion and Git as long as you like. repository and that way you can use both Subversion and Git as long as you like.
It requires access to your GitLab server as it talks with the Git repositories It requires access to your GitLab server as it talks with the Git repositories
directly in a filesystem level. directly in a file system level.
### SubGit prerequisites ### SubGit prerequisites
......
...@@ -411,7 +411,7 @@ to generate a new key. You'll have to update the other repository with the new ...@@ -411,7 +411,7 @@ to generate a new key. You'll have to update the other repository with the new
key to keep the mirror running. key to keep the mirror running.
NOTE: NOTE:
The generated keys are stored in the GitLab database, not in the filesystem. Therefore, The generated keys are stored in the GitLab database, not in the file system. Therefore,
SSH public key authentication for mirrors cannot be used in a pre-receive hook. SSH public key authentication for mirrors cannot be used in a pre-receive hook.
## Forcing an update **(FREE)** ## Forcing an update **(FREE)**
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment