Merge branch 'patch-232' into 'master'

Object storage for job artifacts with or without incremental logging

Closes #247375

See merge request gitlab-org/gitlab!41978

doc/administration/job_artifacts.md

@@ -99,8 +99,13 @@ artifacts, you can use an object storage like AWS S3 instead.
 This configuration relies on valid AWS credentials to be configured already.
 Use an object storage option like AWS S3 to store job artifacts.
 
+If you configure GitLab to store artifacts on object storage, you may also want to
+[eliminate local disk usage for job logs](job_logs.md#prevent-local-disk-usage).
+In both cases, job logs are archived and moved to object storage when the job completes.
+
 DANGER: **Danger:**
-If you configure GitLab to store CI logs and artifacts on object storage, you must also enable [incremental logging](job_logs.md#new-incremental-logging-architecture). Otherwise, job logs will disappear or not be saved.
+In a multi-server setup you must use one of the options to
+[eliminate local disk usage for job logs](job_logs.md#prevent-local-disk-usage), or job logs could be lost.
 
 [Read more about using object storage with GitLab](object_storage.md).
 

doc/administration/job_logs.md

@@ -65,6 +65,15 @@ job logs are automatically migrated to it along with the other job artifacts.
 
 See "Phase 4: uploading" in [Data flow](#data-flow) to learn about the process.
 
+## Prevent local disk usage
+
+If you want to avoid any local disk usage for job logs,
+you can do so using one of the following options:
+
+- Enable the [beta incremental logging](#new-incremental-logging-architecture) feature.
+- Set the [job logs location](#changing-the-job-logs-local-location)
+  to an NFS drive.
+
 ## How to remove job logs
 
 There isn't a way to automatically expire old job logs, but it's safe to remove

doc/administration/object_storage.md

@@ -503,7 +503,7 @@ supported by consolidated configuration form, refer to the following guides:
 |Object storage type|Supported by consolidated configuration?|
 |-------------------|----------------------------------------|
 | [Backups](../raketasks/backup_restore.md#uploading-backups-to-a-remote-cloud-storage)|No|
-| [Job artifacts](job_artifacts.md#using-object-storage) and [incremental logging](job_logs.md#new-incremental-logging-architecture) | Yes |
+| [Job artifacts](job_artifacts.md#using-object-storage) including archived job logs | Yes |
 | [LFS objects](lfs/index.md#storing-lfs-objects-in-remote-object-storage) | Yes |
 | [Uploads](uploads.md#using-object-storage) | Yes |
 | [Container Registry](packages/container_registry.md#use-object-storage) (optional feature) | No |
@@ -520,12 +520,13 @@ supported by consolidated configuration form, refer to the following guides:
 If you're working to [scale out](reference_architectures/index.md) your GitLab implementation,
 or add fault tolerance and redundancy, you may be
 looking at removing dependencies on block or network filesystems.
-See the following guides and
+See the following additional guides and
 [note that Pages requires disk storage](#gitlab-pages-requires-nfs):
 
 1. Make sure the [`git` user home directory](https://docs.gitlab.com/omnibus/settings/configuration.html#moving-the-home-directory-for-a-user) is on local disk.
 1. Configure [database lookup of SSH keys](operations/fast_ssh_key_lookup.md)
    to eliminate the need for a shared `authorized_keys` file.
+1. [Prevent local disk usage for job logs](job_logs.md#prevent-local-disk-usage).
 
 ## Warnings, limitations, and known issues
 
@@ -569,7 +570,8 @@ The dependency on disk storage also prevents Pages being deployed using the
 ### Incremental logging is required for CI to use object storage
 
 If you configure GitLab to use object storage for CI logs and artifacts,
-[you must also enable incremental logging](job_artifacts.md#using-object-storage).
+you can avoid [local disk usage for job logs](job_logs.md#data-flow) by enabling
+[beta incremental logging](job_logs.md#new-incremental-logging-architecture).
 
 ### Proxy Download