In the following table you can see what all these settings mean:
| Setting | Description |
| --------- | ----------- |
| Primary | This marks a Geo Node as primary. There can be only one primary, make sure that you first add the primary node and then all the others. |
| URL | Your instance's full URL, in the same way it is configured in `gitlab.yml` (source based installations) or `/etc/gitlab/gitlab.rb` (omnibus installations). |
|Public Key | The SSH public key of the user that your GitLab instance runs on (unless changed, should be the user `git`). That means that you have to go in each Geo Node separately and create an SSH key pair. See the [SSH key creation][ssh-pair] section. |
## Secondary node GitLab setup
>**Note:**
The Geo nodes admin area (**Admin Area > Geo Nodes**) is not used when setting
up the secondary nodes. This is handled at the primary one.
To install a secondary node, you must follow the normal GitLab Enterprise
Edition installation, with some extra requirements:
- You should point your database connection to a [replicated instance](./database.md).
- Your secondary node should be allowed to [communicate via HTTP/HTTPS and
SSH with your primary node (make sure your firewall is not blocking that).
- Don't make any extra steps you would do for a normal new installation
- Don't setup any custom authentication (this will be handled by the `primary` node)
You need to make sure you restored the database backup (that is part of setting
up replication) and that the primary node PostgreSQL instance is ready to
replicate data.
### Database Encryption Key
-[Repositories data replication](#repositories-data-replication)
-[Create SSH key pairs for Geo nodes](#create-ssh-key-pairs-for-geo-nodes)
Remember to add your primary node to the `known_hosts` file of your `git` user.
This will enable `git` operations to authorize against your existing users.
New users and SSH keys updated after this step, will be replicated automatically.
You can find ssh key files and `know_hosts` at `/var/opt/gitlab/.ssh/` in
Omnibus installations or at `/home/git/.ssh/` when following the source
installation guide.
### Ready to use
Your instance should be ready to use. You can visit the Admin area in the
secondary node to check if it's correctly identified as a secondary Geo node and
if Geo is enabled.
If your installation isn't working properly, check the
[troubleshooting](#troubleshooting) section.
## Create SSH key pairs for new Geo nodes
>**Note:**
These are general instructions to create a new SSH key pair for a new Geo node,
either primary or secondary.
---
When adding a new Geo node, you must provide an SSH public key of the user that
your GitLab instance runs on (unless changed, should be the user `git`). This
user will act as a "normal user" who fetches from the primary Geo node.
1. Run the command below on each server that will be a Geo node:
```bash
sudo -u git -H ssh-keygen
```
1. Get the contents of `id_rsa.pub` the was just created:
```
# Omnibus installations
sudo -u git cat /var/opt/gitlab/.ssh/id_rsa.pub
# Installations from source
sudo -u git cat /home/git/.ssh/id_rsa.pub
```
1. Copy them to the admin area of the **primary** node (**Admin Area > Geo Nodes**).
---
If for any reason you generate the key using a different name from the default
`id_rsa`, or you want to generate an extra key only for the repository
...
...
@@ -84,105 +213,86 @@ Host example.com # The FQDN of the primary Geo node
IdentityFile ~/.ssh/mycustom.key # The location of your private key
```
## Primary Node GitLab setup
### Add the primary node to the `known_hosts` file of the secondary nodes
>**Note:**
You will need to setup your database into a **Primary <-> Secondary (read-only)** replication
topology, and your Primary node should always point to a database's Primary
instance. If you haven't done that already, read [database replication](./database.md).
Go to the server that you chose to be your primary, and visit
**Admin Area > Geo Nodes** (`/admin/geo_nodes`) in order to add the Geo nodes.
Although we are looking at the primary Geo node setup, **this is where you also
add any secondary servers as well**.
In the following table you can see what all these settings mean:
| Setting | Description |
| --------- | ----------- |
| Primary | This marks a Geo Node as primary. There can be only one primary, make sure that you first add the primary node and then all the others. |
| URL | Your instance's full URL, in the same way it is configured in `gitlab.yml` (source based installations) or `/etc/gitlab/gitlab.rb` (omnibus installations). |
|Public Key | The SSH public key of the user that your GitLab instance runs on (unless changed, should be the user `git`). That means that you have to go in each Geo Node separately and create an SSH key pair. See the [SSH key creation](#create-ssh-key-pairs-for-geo-nodes) section. |
First, add your primary node by providing its full URL and the public SSH key
you created previously. Make sure to check the box 'This is a primary node'
when adding it. Continue with all secondaries.
This operation is only needed for the secondary nodes.
---
## Secondary Node GitLab setup
>**Note:**
The Geo nodes admin area (**Admin Area > Geo Nodes**) is not used when setting
up the secondary servers. This is handled by the primary server setup.
To install a secondary node, you must follow the normal GitLab Enterprise
Edition installation, with some extra requirements:
- You should point your database connection to a [replicated instance](./database.md).
- Your secondary node should be allowed to communicate via HTTP/HTTPS and
SSH with your primary node (make sure your firewall is not blocking that).
### Database Encryption Key
GitLab stores a unique encryption key in disk that we use to safely store sensitive
data in the database.
Any secondary node must have the exact same value for `db_key_base` as defined in the primary one.
For Omnibus installations it is stored at `/etc/gitlab/gitlab-secrets.json`.
For Source installations it is stored at `/home/git/gitlab/config/secrets.yml`.
### Authorized keys regeneration
The final step will be to regenerate the keys for `.ssh/authorized_keys` using
the following command (HTTPS clone will still work without this extra step).
On the secondary node where the database is [already replicated](./database.md),
run the following:
The secondary nodes need to know the SSH fingerprint of the primary node that
will be used for the Git clone/fetch operations. In order to add it to the
`known_hosts` file, while in the terminal of a secondary node, run the
- By running the command above, `primary` should be `true` when executed in
the primary node, and `false` on any secondary
- Did I defined the correct SSH Key for the node?
- Did I define the correct SSH Key for the node?
- You must create an SSH Key for `git` user
- This key is the one you have to inform at `Admin > Geo`
- Can I SSH from secondary to primary node using `git` user account?
- This is the most obvious cause of problems with repository replication issues.
If you haven't added the primary node's key to `known_hosts`, you will end up with
a lot of failed sidekiq jobs with an error similar to:
```
Gitlab::Shell::Error: Host key verification failed. fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.
```
An easy way to fix is by logging in as the `git` user in the secondary node and run:
```
# remove old entries to your primary gitlab in known_hosts
