Skip to content

G
gitlab-ce
Commit a8d8616c authored by Craig Norris's avatar Craig Norris
Browse files
Options

Merge branch 'axil-sidekiq-s3' into 'master' 

Mention object storage in Sidekiq and Rails reference architectures

See merge request gitlab-org/gitlab!50852
parents b028eb64 7cdcda16
Show whitespace changes
Inline Side-by-side
Showing with 230 additions and 9 deletions
doc/administration/reference_architectures/10k_users.md
View file @ a8d8616c
......@@ -1812,8 +1812,9 @@ To configure Praefect with TLS:
## Configure Sidekiq
Sidekiq requires connections to the Redis, PostgreSQL and Gitaly instances.
The following IPs will be used as an example:
Sidekiq requires connection to the [Redis](#configure-redis),
[PostgreSQL](#configure-postgresql) and [Gitaly](#configure-gitaly) instances.
[Object storage](#configure-the-object-storage) is also required to be configured.
- `10.6.0.101`: Sidekiq 1
- `10.6.0.102`: Sidekiq 2
......@@ -1927,6 +1928,25 @@ To configure the Sidekiq nodes, on each one:
# Rails Status for prometheus
gitlab_rails['monitoring_whitelist'] = ['10.6.0.121/32', '127.0.0.0/8']
#############################
### Object storage ###
#############################
# This is an example for configuring Object Storage on GCP
# Replace this config with your chosen Object Storage provider as desired
gitlab_rails['object_store']['connection'] = {
'provider' => 'Google',
'google_project' => '<gcp-project-name>',
'google_json_key_location' => '<path-to-gcp-service-account-key>'
}
gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>"
```
1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace
......@@ -1947,6 +1967,7 @@ You can also run [multiple Sidekiq processes](../operations/extra_sidekiq_proces
## Configure GitLab Rails
This section describes how to configure the GitLab application (Rails) component.
[Object storage](#configure-the-object-storage) is also required to be configured.
The following IPs will be used as an example:
......@@ -2036,6 +2057,25 @@ On each node perform the following:
# scrape the NGINX metrics
gitlab_rails['monitoring_whitelist'] = ['10.6.0.121/32', '127.0.0.0/8']
nginx['status']['options']['allow'] = ['10.6.0.121/32', '127.0.0.0/8']
#############################
### Object storage ###
#############################
# This is an example for configuring Object Storage on GCP
# Replace this config with your chosen Object Storage provider as desired
gitlab_rails['object_store']['connection'] = {
'provider' => 'Google',
'google_project' => '<gcp-project-name>',
'google_json_key_location' => '<path-to-gcp-service-account-key>'
}
gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>"
```
1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure).
......
doc/administration/reference_architectures/25k_users.md
View file @ a8d8616c
......@@ -1512,8 +1512,9 @@ To configure Gitaly with TLS:
## Configure Sidekiq
Sidekiq requires connections to the Redis, PostgreSQL and Gitaly instances.
The following IPs will be used as an example:
Sidekiq requires connection to the [Redis](#configure-redis),
[PostgreSQL](#configure-postgresql) and [Gitaly](#configure-gitaly) instances.
[Object storage](#configure-the-object-storage) is also required to be configured.
- `10.6.0.101`: Sidekiq 1
- `10.6.0.102`: Sidekiq 2
......@@ -1624,6 +1625,25 @@ To configure the Sidekiq nodes, on each one:
# Rails Status for prometheus
gitlab_rails['monitoring_whitelist'] = ['10.6.0.121/32', '127.0.0.0/8']
#############################
### Object storage ###
#############################
# This is an example for configuring Object Storage on GCP
# Replace this config with your chosen Object Storage provider as desired
gitlab_rails['object_store']['connection'] = {
'provider' => 'Google',
'google_project' => '<gcp-project-name>',
'google_json_key_location' => '<path-to-gcp-service-account-key>'
}
gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>"
```
1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace
......@@ -1644,6 +1664,7 @@ You can also run [multiple Sidekiq processes](../operations/extra_sidekiq_proces
## Configure GitLab Rails
This section describes how to configure the GitLab application (Rails) component.
[Object storage](#configure-the-object-storage) is also required to be configured.
The following IPs will be used as an example:
......@@ -1736,6 +1757,25 @@ On each node perform the following:
# scrape the NGINX metrics
gitlab_rails['monitoring_whitelist'] = ['10.6.0.121/32', '127.0.0.0/8']
nginx['status']['options']['allow'] = ['10.6.0.121/32', '127.0.0.0/8']
#############################
### Object storage ###
#############################
# This is an example for configuring Object Storage on GCP
# Replace this config with your chosen Object Storage provider as desired
gitlab_rails['object_store']['connection'] = {
'provider' => 'Google',
'google_project' => '<gcp-project-name>',
'google_json_key_location' => '<path-to-gcp-service-account-key>'
}
gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>"
```
1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure).
......
doc/administration/reference_architectures/2k_users.md
View file @ a8d8616c
......@@ -657,6 +657,25 @@ On each node perform the following:
gitlab_rails['monitoring_whitelist'] = ['<MONITOR NODE IP>/32', '127.0.0.0/8']
nginx['status']['options']['allow'] = ['<MONITOR NODE IP>/32', '127.0.0.0/8']
#############################
### Object storage ###
#############################
# This is an example for configuring Object Storage on GCP
# Replace this config with your chosen Object Storage provider as desired
gitlab_rails['object_store']['connection'] = {
'provider' => 'Google',
'google_project' => '<gcp-project-name>',
'google_json_key_location' => '<path-to-gcp-service-account-key>'
}
gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>"
## Uncomment and edit the following options if you have set up NFS
##
## Prevent GitLab from starting if NFS data mounts are not available
......
doc/administration/reference_architectures/3k_users.md
View file @ a8d8616c
......@@ -1212,7 +1212,10 @@ To configure Gitaly with TLS:
## Configure Sidekiq
Sidekiq requires connection to the Redis, PostgreSQL and Gitaly instance.
Sidekiq requires connection to the [Redis](#configure-redis),
[PostgreSQL](#configure-postgresql) and [Gitaly](#configure-gitaly) instances.
[Object storage](#configure-the-object-storage) is also required to be configured.
The following IPs will be used as an example:
- `10.6.0.71`: Sidekiq 1
......@@ -1307,6 +1310,25 @@ To configure the Sidekiq nodes, one each one:
# Rails Status for prometheus
gitlab_rails['monitoring_whitelist'] = ['10.6.0.81/32', '127.0.0.0/8']
gitlab_rails['prometheus_address'] = '10.6.0.81:9090'
#############################
### Object storage ###
#############################
# This is an example for configuring Object Storage on GCP
# Replace this config with your chosen Object Storage provider as desired
gitlab_rails['object_store']['connection'] = {
'provider' => 'Google',
'google_project' => '<gcp-project-name>',
'google_json_key_location' => '<path-to-gcp-service-account-key>'
}
gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>"
```
1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure).
......@@ -1337,6 +1359,7 @@ You can also run [multiple Sidekiq processes](../operations/extra_sidekiq_proces
## Configure GitLab Rails
This section describes how to configure the GitLab application (Rails) component.
[Object storage](#configure-the-object-storage) is also required to be configured.
On each node perform the following:
......@@ -1454,6 +1477,25 @@ On each node perform the following:
#web_server['gid'] = 9001
#registry['uid'] = 9002
#registry['gid'] = 9002
#############################
### Object storage ###
#############################
# This is an example for configuring Object Storage on GCP
# Replace this config with your chosen Object Storage provider as desired
gitlab_rails['object_store']['connection'] = {
'provider' => 'Google',
'google_project' => '<gcp-project-name>',
'google_json_key_location' => '<path-to-gcp-service-account-key>'
}
gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>"
```
1. If you're using [Gitaly with TLS support](#gitaly-tls-support), make sure the
......
doc/administration/reference_architectures/50k_users.md
View file @ a8d8616c
......@@ -1512,8 +1512,9 @@ To configure Gitaly with TLS:
## Configure Sidekiq
Sidekiq requires connections to the Redis, PostgreSQL and Gitaly instances.
The following IPs will be used as an example:
Sidekiq requires connection to the [Redis](#configure-redis),
[PostgreSQL](#configure-postgresql) and [Gitaly](#configure-gitaly) instances.
[Object storage](#configure-the-object-storage) is also required to be configured.
- `10.6.0.101`: Sidekiq 1
- `10.6.0.102`: Sidekiq 2
......@@ -1624,6 +1625,25 @@ To configure the Sidekiq nodes, on each one:
# Rails Status for prometheus
gitlab_rails['monitoring_whitelist'] = ['10.6.0.121/32', '127.0.0.0/8']
#############################
### Object storage ###
#############################
# This is an example for configuring Object Storage on GCP
# Replace this config with your chosen Object Storage provider as desired
gitlab_rails['object_store']['connection'] = {
'provider' => 'Google',
'google_project' => '<gcp-project-name>',
'google_json_key_location' => '<path-to-gcp-service-account-key>'
}
gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>"
```
1. Copy the `/etc/gitlab/gitlab-secrets.json` file from your Consul server, and replace
......@@ -1644,6 +1664,7 @@ You can also run [multiple Sidekiq processes](../operations/extra_sidekiq_proces
## Configure GitLab Rails
This section describes how to configure the GitLab application (Rails) component.
[Object storage](#configure-the-object-storage) is also required to be configured.
The following IPs will be used as an example:
......@@ -1736,6 +1757,25 @@ On each node perform the following:
# scrape the NGINX metrics
gitlab_rails['monitoring_whitelist'] = ['10.6.0.121/32', '127.0.0.0/8']
nginx['status']['options']['allow'] = ['10.6.0.121/32', '127.0.0.0/8']
#############################
### Object storage ###
#############################
# This is an example for configuring Object Storage on GCP
# Replace this config with your chosen Object Storage provider as desired
gitlab_rails['object_store']['connection'] = {
'provider' => 'Google',
'google_project' => '<gcp-project-name>',
'google_json_key_location' => '<path-to-gcp-service-account-key>'
}
gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>"
```
1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure).
......
doc/administration/reference_architectures/5k_users.md
View file @ a8d8616c
......@@ -1211,8 +1211,9 @@ To configure Gitaly with TLS:
## Configure Sidekiq
Sidekiq requires connection to the Redis, PostgreSQL and Gitaly instance.
The following IPs will be used as an example:
Sidekiq requires connection to the [Redis](#configure-redis),
[PostgreSQL](#configure-postgresql) and [Gitaly](#configure-gitaly) instances.
[Object storage](#configure-the-object-storage) is also required to be configured.
- `10.6.0.71`: Sidekiq 1
- `10.6.0.72`: Sidekiq 2
......@@ -1306,6 +1307,25 @@ To configure the Sidekiq nodes, one each one:
# Rails Status for prometheus
gitlab_rails['monitoring_whitelist'] = ['10.6.0.81/32', '127.0.0.0/8']
gitlab_rails['prometheus_address'] = '10.6.0.81:9090'
#############################
### Object storage ###
#############################
# This is an example for configuring Object Storage on GCP
# Replace this config with your chosen Object Storage provider as desired
gitlab_rails['object_store']['connection'] = {
'provider' => 'Google',
'google_project' => '<gcp-project-name>',
'google_json_key_location' => '<path-to-gcp-service-account-key>'
}
gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>"
```
1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure).
......@@ -1336,6 +1356,7 @@ You can also run [multiple Sidekiq processes](../operations/extra_sidekiq_proces
## Configure GitLab Rails
This section describes how to configure the GitLab application (Rails) component.
[Object storage](#configure-the-object-storage) is also required to be configured.
On each node perform the following:
......@@ -1439,6 +1460,25 @@ On each node perform the following:
nginx['status']['options']['allow'] = ['10.6.0.81/32', '127.0.0.0/8']
gitlab_rails['prometheus_address'] = '10.6.0.81:9090'
#############################
### Object storage ###
#############################
# This is an example for configuring Object Storage on GCP
# Replace this config with your chosen Object Storage provider as desired
gitlab_rails['object_store']['connection'] = {
'provider' => 'Google',
'google_project' => '<gcp-project-name>',
'google_json_key_location' => '<path-to-gcp-service-account-key>'
}
gitlab_rails['object_store']['objects']['artifacts']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['lfs']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['uploads']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['packages']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = "<gcp-bucket-name>"
gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = "<gcp-bucket-name>"
## Uncomment and edit the following options if you have set up NFS
##
## Prevent GitLab from starting if NFS data mounts are not available
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7