Commit 5cd9bf78 authored by Paul Okstad's avatar Paul Okstad Committed by Evan Read

Document Praefect storage node constraints

Praefect has a number of uniqueness constraints for how backend Gitaly
nodes can be named or addressed. This note should help clarify these
constraints when adding additional Gitaly nodes to virtual storages.
parent 72c7fc8c
...@@ -199,12 +199,12 @@ You need the IP/host address for each node. ...@@ -199,12 +199,12 @@ You need the IP/host address for each node.
1. `LOAD_BALANCER_SERVER_ADDRESS`: the IP/host address of the load balancer 1. `LOAD_BALANCER_SERVER_ADDRESS`: the IP/host address of the load balancer
1. `POSTGRESQL_SERVER_ADDRESS`: the IP/host address of the PostgreSQL server 1. `POSTGRESQL_SERVER_ADDRESS`: the IP/host address of the PostgreSQL server
1. `PRAEFECT_HOST`: the IP/host address of the Praefect server 1. `PRAEFECT_HOST`: the IP/host address of the Praefect server
1. `GITALY_HOST`: the IP/host address of each Gitaly server 1. `GITALY_HOST_*`: the IP or host address of each Gitaly server
1. `GITLAB_HOST`: the IP/host address of the GitLab server 1. `GITLAB_HOST`: the IP/host address of the GitLab server
If you are using a cloud provider, you can look up the addresses for each server through your cloud provider's management console. If you are using a cloud provider, you can look up the addresses for each server through your cloud provider's management console.
If you are using Google Cloud Platform, SoftLayer, or any other vendor that provides a virtual private cloud (VPC) you can use the private addresses for each cloud instance (corresponds to “internal address” for Google Cloud Platform) for `PRAEFECT_HOST`, `GITALY_HOST`, and `GITLAB_HOST`. If you are using Google Cloud Platform, SoftLayer, or any other vendor that provides a virtual private cloud (VPC) you can use the private addresses for each cloud instance (corresponds to “internal address” for Google Cloud Platform) for `PRAEFECT_HOST`, `GITALY_HOST_*`, and `GITLAB_HOST`.
#### Secrets #### Secrets
...@@ -465,26 +465,31 @@ application server, or a Gitaly node. ...@@ -465,26 +465,31 @@ application server, or a Gitaly node.
Praefect when communicating with Gitaly nodes in the cluster. This token is Praefect when communicating with Gitaly nodes in the cluster. This token is
distinct from the `PRAEFECT_EXTERNAL_TOKEN`. distinct from the `PRAEFECT_EXTERNAL_TOKEN`.
Replace `GITALY_HOST` with the IP/host address of the each Gitaly node. Replace `GITALY_HOST_*` with the IP or host address of the each Gitaly node.
More Gitaly nodes can be added to the cluster to increase the number of More Gitaly nodes can be added to the cluster to increase the number of
replicas. More clusters can also be added for very large GitLab instances. replicas. More clusters can also be added for very large GitLab instances.
NOTE:
When adding additional Gitaly nodes to a virtual storage, all storage names
within that virtual storage must be unique. Additionally, all Gitaly node
addresses referenced in the Praefect configuration must be unique.
```ruby ```ruby
# Name of storage hash must match storage name in git_data_dirs on GitLab # Name of storage hash must match storage name in git_data_dirs on GitLab
# server ('praefect') and in git_data_dirs on Gitaly nodes ('gitaly-1') # server ('praefect') and in git_data_dirs on Gitaly nodes ('gitaly-1')
praefect['virtual_storages'] = { praefect['virtual_storages'] = {
'default' => { 'default' => {
'gitaly-1' => { 'gitaly-1' => {
'address' => 'tcp://GITALY_HOST:8075', 'address' => 'tcp://GITALY_HOST_1:8075',
'token' => 'PRAEFECT_INTERNAL_TOKEN', 'token' => 'PRAEFECT_INTERNAL_TOKEN',
}, },
'gitaly-2' => { 'gitaly-2' => {
'address' => 'tcp://GITALY_HOST:8075', 'address' => 'tcp://GITALY_HOST_2:8075',
'token' => 'PRAEFECT_INTERNAL_TOKEN' 'token' => 'PRAEFECT_INTERNAL_TOKEN'
}, },
'gitaly-3' => { 'gitaly-3' => {
'address' => 'tcp://GITALY_HOST:8075', 'address' => 'tcp://GITALY_HOST_3:8075',
'token' => 'PRAEFECT_INTERNAL_TOKEN' 'token' => 'PRAEFECT_INTERNAL_TOKEN'
} }
} }
...@@ -919,7 +924,7 @@ Particular attention should be shown to: ...@@ -919,7 +924,7 @@ Particular attention should be shown to:
You need to replace: You need to replace:
- `PRAEFECT_HOST` with the IP address or hostname of the Praefect node - `PRAEFECT_HOST` with the IP address or hostname of the Praefect node
- `GITALY_HOST` with the IP address or hostname of each Gitaly node - `GITALY_HOST_*` with the IP address or hostname of each Gitaly node
```ruby ```ruby
prometheus['scrape_configs'] = [ prometheus['scrape_configs'] = [
...@@ -937,9 +942,9 @@ Particular attention should be shown to: ...@@ -937,9 +942,9 @@ Particular attention should be shown to:
'job_name' => 'praefect-gitaly', 'job_name' => 'praefect-gitaly',
'static_configs' => [ 'static_configs' => [
'targets' => [ 'targets' => [
'GITALY_HOST:9236', # gitaly-1 'GITALY_HOST_1:9236', # gitaly-1
'GITALY_HOST:9236', # gitaly-2 'GITALY_HOST_2:9236', # gitaly-2
'GITALY_HOST:9236', # gitaly-3 'GITALY_HOST_3:9236', # gitaly-3
] ]
] ]
} }
...@@ -1137,7 +1142,7 @@ You can configure: ...@@ -1137,7 +1142,7 @@ You can configure:
'default_replication_factor' => 1, 'default_replication_factor' => 1,
# nodes... # nodes...
'gitaly-1' => { 'gitaly-1' => {
'address' => 'tcp://GITALY_HOST:8075', 'address' => 'tcp://GITALY_HOST_1:8075',
'token' => 'PRAEFECT_INTERNAL_TOKEN', 'token' => 'PRAEFECT_INTERNAL_TOKEN',
}, },
} }
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment