Skip to content

G
gitlab-ce
Commit fc97edb1 authored by Michael Kozono's avatar Michael Kozono
Browse files
Options

Clarify and standardize documentation

parent 60fdb5dc
Show whitespace changes
Inline Side-by-side
Showing with 113 additions and 85 deletions
doc/administration/geo/disaster_recovery/index.md
View file @ fc97edb1
......@@ -83,7 +83,14 @@ must disable the primary.
[update the primary domain DNS record](#step-4-optional-updating-the-primary-domain-dns-record),
you may wish to lower the TTL now to speed up propagation.
### Step 3. Promoting a secondary Geo replica
### Step 3. Promoting a **secondary** node
NOTE: **Note:**
A new **secondary** should not be added at this time. If you want to add a new
**secondary**, do this after you have completed the entire process of promoting
the **secondary** to the **primary**.
#### Promoting a **secondary** node running on a single machine
1. SSH in to your **secondary** and login as root:
......@@ -91,43 +98,66 @@ must disable the primary.
sudo -i
```
1. Edit `/etc/gitlab/gitlab.rb` to reflect its new status as primary by
removing the following line:
1. Edit `/etc/gitlab/gitlab.rb` to reflect its new status as **primary** by
removing any lines that enabled the `geo_secondary_role`:
```ruby
## REMOVE THIS LINE
## In pre-11.5 documentation, the role was enabled as follows. Remove this line.
geo_secondary_role['enable'] = true
## In 11.5+ documentation, the role was enabled as follows. Remove this line.
roles ['geo_secondary_role']
```
A new secondary should not be added at this time. If you want to add a new
secondary, do this after you have completed the entire process of promoting
the secondary to the primary.
1. Promote the secondary to primary. Execute:
1. Promote the **secondary** to **primary**. Execute:
```bash
gitlab-ctl promote-to-primary-node
```
1. Verify you can connect to the newly promoted primary using the URL used
previously for the secondary.
1. Success! The secondary has now been promoted to primary.
1. Verify you can connect to the newly promoted **primary** using the URL used
previously for the **secondary**.
1. Success! The **secondary** has now been promoted to **primary**.
#### Promoting a node with HA
#### Promoting a **secondary** node with HA
The `gitlab-ctl promote-to-primary-node` command cannot be used yet in conjunction with
High Availability or with multiple machines, as it can only perform changes on
a single one.
The `gitlab-ctl promote-to-primary-node` command cannot be used yet in
conjunction with High Availability or with multiple machines, as it can only
perform changes on a **secondary** with only a single machine. Instead, you must
do this manually.
The command above does the following changes:
1. SSH in to the database node in the **secondary** and trigger PostgreSQL to
promote to read-write:
- Promotes the PostgreSQL secondary to primary
- Executes `gitlab-ctl reconfigure` to apply the changes in `/etc/gitlab/gitlab.rb`
- Runs `gitlab-rake geo:set_secondary_as_primary`
```bash
sudo gitlab-pg-ctl promote
```
1. Edit `/etc/gitlab/gitlab.rb` on every machine in the **secondary** to
reflect its new status as **primary** by removing any lines that enabled the
`geo_secondary_role`:
```ruby
## In pre-11.5 documentation, the role was enabled as follows. Remove this line.
geo_secondary_role['enable'] = true
## In 11.5+ documentation, the role was enabled as follows. Remove this line.
roles ['geo_secondary_role']
```
After making these changes [Reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) each
machine so the changes take effect.
1. Promote the **secondary** to **primary**. SSH into a single application
server and execute:
```bash
sudo gitlab-rake geo:set_secondary_as_primary
```
You need to make sure all the affected machines no longer have `roles ['geo_secondary_role']` in
`/etc/gitlab/gitlab.rb`, that you execute the database promotion on the required database nodes
and you execute the `gitlab-rake geo:set_secondary_as_primary` in a machine running the application server.
1. Verify you can connect to the newly promoted **primary** using the URL used
previously for the **secondary**.
1. Success! The **secondary** has now been promoted to **primary**.
### Step 4. (Optional) Updating the primary domain DNS record
......
doc/administration/geo/replication/database.md
View file @ fc97edb1
......@@ -3,7 +3,7 @@
NOTE: **Note:**
This is the documentation for the Omnibus GitLab packages. For installations
from source, follow the
[**database replication for installations from source**][database-source] guide.
[Geo database replication (source)](database_source.md) guide.
NOTE: **Note:**
If your GitLab installation uses external PostgreSQL, the Omnibus roles
......@@ -104,7 +104,9 @@ The following guide assumes that:
If you are using an external database not managed by Omnibus GitLab, you need
to create the replicator user and define a password to it manually.
Check [How to create replication user][database-source-primary] documentation.
For information on how to create a replication user, refer to the
[appropriate step](database_source.md#step-1-configure-the-primary-server)
in [Geo database replication (source)](database_source.md).
1. Configure PostgreSQL to listen on network interfaces
......@@ -348,7 +350,7 @@ The following guide assumes that:
```
For external PostgreSQL instances, [see additional instructions][external postgresql].
If you bring a former primary back online to serve as a secondary then you also need to remove `roles ['geo_primary_role']`.
If you bring a former **primary** node back online to serve as a **secondary** node, then you also need to remove `roles ['geo_primary_role']` or `geo_primary_role['enable'] = true`.
1. Reconfigure GitLab for the changes to take effect:
......@@ -621,8 +623,6 @@ Read the [troubleshooting document](troubleshooting.md).
[tracking]: database_source.md#enable-tracking-database-on-the-secondary-server
[FDW]: https://www.postgresql.org/docs/9.6/static/postgres-fdw.html
[toc]: index.md#using-omnibus-gitlab
[database-source]: database_source.md
[database-source-primary]: database_source.md#step-1-configure-the-primary-server
[rake-maintenance]: ../../raketasks/maintenance.md
[pg-docs-ssl]: https://www.postgresql.org/docs/9.6/static/libpq-ssl.html#LIBPQ-SSL-PROTECTION
[pg-docs-runtime-conn]: https://www.postgresql.org/docs/9.6/static/runtime-config-connection.html
......
doc/administration/geo/replication/high_availability.md
View file @ fc97edb1
......@@ -31,19 +31,22 @@ in setting up this configuration for PostgreSQL and Redis
it is not covered by this Geo HA documentation.
For more information about setting up a highly available PostgreSQL cluster and Redis cluster using the omnibus package see the high availability documentation for
[PostgreSQL][postgresql-ha] and [Redis][redis-ha], respectively.
[PostgreSQL](../../high_availability/database.md) and
[Redis](../../high_availability/redis.md), respectively.
NOTE: **Note:**
It is possible to use cloud hosted services for PostgreSQL and Redis but this is beyond the scope of this document.
## Prerequisites: A working GitLab HA cluster
This cluster will serve as the Geo Primary. Use the
[GitLab HA documentation][gitlab-ha] to set this up.
This cluster will serve as the **primary** node. Use the
[GitLab HA documentation](../../high_availability/README.md) to set this up.
## Configure the working cluster to be a Geo Primary
## Configure the GitLab cluster to be the **primary** node
### Step 1: Configure the Geo Primary Frontend Servers
The following steps enable a GitLab cluster to serve as the **primary** node.
### Step 1: Configure the **primary** frontend servers
1. Edit `/etc/gitlab/gitlab.rb` and add the following:
......@@ -59,63 +62,71 @@ This cluster will serve as the Geo Primary. Use the
gitlab_rails['auto_migrate'] = false
```
After making these changes, [reconfigure GitLab][gitlab-reconfigure] so that they take effect.
After making these changes, [reconfigure GitLab][gitlab-reconfigure] so the changes take effect.
NOTE: **Note:** PostgreSQL and Redis should have already been disabled on the
application servers and the connections configured, during normal GitLab
HA set up. See documentation for
[PostgreSQL][postgresql-ha-configuring-application-nodes] and
[Redis][redis-ha-configuring-the-application-nodes]
application servers, and connections from the application servers to those
services on the backend servers configured, during normal GitLab HA set up. See
high availability configuration documentation for
[PostgreSQL](../../high_availability/database.md#configuring-the-application-nodes)
and [Redis](../../high_availability/redis.md#example-configuration-for-the-gitlab-application).
The Geo Primary database will require modification later, as part of
[Step 2 of Configure a Geo Secondary][step-2-of-configure-a-geo-secondary].
The **primary** database will require modification later, as part of
[step 2](#step-2-configure-the-main-read-only-replica-postgresql-database-on-the-secondary-node).
## Configure a Geo Secondary
## Configure a **secondary** node
A Geo Secondary cluster is similar to any other GitLab HA cluster, with two
A **secondary** cluster is similar to any other GitLab HA cluster, with two
major differences:
1. The main PostgreSQL database is a read-only replica of the Geo Primary's
* The main PostgreSQL database is a read-only replica of the **primary** node's
PostgreSQL database.
1. There is also a single PostgreSQL database per Geo Secondary cluster, called
the "tracking database", which tracks the sync state of various resources.
* There is also a single PostgreSQL database for the **secondary** cluster,
called the "tracking database", which tracks the synchronization state of
various resources.
So, we will set up the HA components one-by-one, and include deviations from
the normal HA setup.
Therefore, we will set up the HA components one-by-one, and include deviations
from the normal HA setup.
### Step 1: Configure the Redis and NFS services on the Geo Secondary
### Step 1: Configure the Redis and NFS services on the **secondary** node
Configure the following services, again using the non-Geo, HA documentation:
Configure the following services, again using the non-Geo high availability
documentation:
* [Redis][redis-ha] for high availability.
* [NFS Server][nfs-ha] which will store data that is synchronized from the Geo primary.
* [Configuring Redis for GitLab HA](../../high_availability/redis.md) for high
availability.
* [NFS](../../high_availability/nfs.md) which will store data that is
synchronized from the **primary** node.
### Step 2: Configure the main read-only replica PostgreSQL database on the Geo Secondary
### Step 2: Configure the main read-only replica PostgreSQL database on the **secondary** node
NOTE: **Note:** This documentation assumes the DB will be run on only a single
machine, rather than as a PostgreSQL cluster.
NOTE: **Note:** The following documentation assumes the database will be run on
only a single machine, rather than as a PostgreSQL cluster.
Configure the [secondary Geo PostgreSQL database][database] as a read-only
secondary of the primary Geo PostgreSQL database. Be sure to follow the
[External PostgreSQL instances][external-postgresql] section.
Configure the [**secondary** database](database.md) as a read-only replica of
the **primary** database. Be sure to follow the
[External PostgreSQL instances](database.md#external-postgresql-instances)
section.
### Step 3: Configure the tracking database on the Geo Secondary
### Step 3: Configure the tracking database on the **secondary** node
NOTE: **Note:** This documentation assumes the tracking DB will be run on only a
single machine, rather than as a PostgreSQL cluster.
NOTE: **Note:** This documentation assumes the tracking database will be run on
only a single machine, rather than as a PostgreSQL cluster.
Configure the [Geo tracking database][tracking-database].
Configure the
[tracking database](database.md#tracking-database-for-the-secondary-nodes).
### Step 4: Configure the Frontend Application servers on the Geo Secondary
### Step 4: Configure the frontend application servers on the **secondary** node
In the architecture overview, there are two machines running the GitLab
application services. These services are enabled selectively in the
configuration.
Configure the application servers following [Configuring GitLab for HA][app-ha],
then make the following modifications:
Configure the application servers following
[Configuring GitLab for HA](../../high_availability/gitlab.md), then make the
following modifications:
1. Edit `/etc/gitlab/gitlab.rb` on each application server in the secondary
1. Edit `/etc/gitlab/gitlab.rb` on each application server in the **secondary**
cluster, and add the following:
```ruby
......@@ -173,7 +184,7 @@ servers connect to the databases.
NOTE: **Note:**
Make sure that current node IP is listed in `postgresql['md5_auth_cidr_addresses']` setting of your remote database.
After making these changes [Reconfigure GitLab][gitlab-reconfigure] so that they take effect.
After making these changes [Reconfigure GitLab][gitlab-reconfigure] so the changes take effect.
On the secondary the following GitLab frontend services will be enabled:
......@@ -190,26 +201,13 @@ On the secondary the following GitLab frontend services will be enabled:
Verify these services by running `sudo gitlab-ctl status` on the frontend
application servers.
### Step 5: Set up the LoadBalancer for the Geo Secondary
### Step 5: Set up the LoadBalancer for the **secondary** node
In this topology there will need to be a load balancers at each geographical
location to route traffic to the application servers.
In this topology, a load balancer is required at each geographic location to
route traffic to the application servers.
See the [Load Balancer for GitLab HA][load-balancer-ha]
documentation for more information.
See [Load Balancer for GitLab HA](../../high_availability/load_balancer.md) for
more information.
[diagram-source]: https://docs.google.com/drawings/d/1z0VlizKiLNXVVVaERFwgsIOuEgjcUqDTWPdQYsE7Z4c/edit
[gitlab-reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure
[redis-ha]: ../../high_availability/redis.md
[redis-ha-configuring-the-application-nodes]: ../../high_availability/redis.md#example-configuration-for-the-gitlab-application
[postgresql-ha]: ../../high_availability/database.md
[postgresql-ha-configuring-application-nodes]: ../../high_availability/database.md#configuring-the-application-nodes
[nfs-ha]: ../../high_availability/nfs.md
[load-balancer-ha]: ../../high_availability/load_balancer.md
[database]: database.md
[tracking-database]: database.md#tracking-database-for-the-secondary-nodes
[external-postgresql]: database.md#external-postgresql-instances
[gitlab-rb-template]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template
[gitlab-ha]: ../../high_availability/README.md
[app-ha]: ../../high_availability/gitlab.md
[step-2-of-configure-a-geo-secondary]: #step-2-configure-the-main-read-only-replica-postgresql-database-on-the-geo-secondary
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7