@@ -5,27 +5,37 @@ info: To determine the technical writer assigned to the Stage/Group associated w
...
@@ -5,27 +5,37 @@ info: To determine the technical writer assigned to the Stage/Group associated w
type:howto
type:howto
---
---
# Versionspecific update instructions
# Version-specific update instructions
Check this document if it includes instructions for the version you are updating.
Review this page for update instructions for your version. These steps
These steps go together with the [general steps](updating_the_geo_nodes.md#general-update-steps)
accompany the [general steps](updating_the_geo_nodes.md#general-update-steps)
for updating Geo nodes.
for updating Geo nodes.
## Updating to GitLab 13.7
## Updating to GitLab 13.7
We've detected an issue with the `FetchRemove` call that is used by Geo secondaries. This causes performance issues as we execute reference transaction hooks for each updated reference. Please hold off upgrading until this is in [the 13.7.5 patch release.](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/3002). More details are available [in this issue](https://gitlab.com/gitlab-org/git/-/issues/79).
We've detected an issue with the `FetchRemove` call used by Geo secondaries.
This causes performance issues as we execute reference transaction hooks for
each updated reference. Delay any upgrade attempts until this is in the
More details are available [in this issue](https://gitlab.com/gitlab-org/git/-/issues/79).
## Updating to GitLab 13.5
## Updating to GitLab 13.5
In GitLab 13.5, there is a [regression that prevents viewing a list of container repositories and registries](https://gitlab.com/gitlab-org/gitlab/-/issues/285475) on Geo secondaries. This issue is fixed in GitLab 13.6.1 and later.
GitLab 13.5 has a [regression that prevents viewing a list of container repositories and registries](https://gitlab.com/gitlab-org/gitlab/-/issues/285475)
on Geo secondaries. This issue is fixed in GitLab 13.6.1 and later.
## Updating to GitLab 13.3
## Updating to GitLab 13.3
In GitLab 13.3, Geo removed the PostgreSQL [Foreign Data Wrapper](https://www.postgresql.org/docs/11/postgres-fdw.html) dependency for the tracking database.
In GitLab 13.3, Geo removed the PostgreSQL [Foreign Data Wrapper](https://www.postgresql.org/docs/11/postgres-fdw.html)
dependency for the tracking database.
The FDW server, user, and the extension will be removed during the upgrade process on each **secondary** node. The GitLab settings related to the FDW in the `/etc/gitlab/gitlab.rb` have been deprecated and can be safely removed.
The FDW server, user, and the extension will be removed during the upgrade
process on each secondary node. The GitLab settings related to the FDW in the
`/etc/gitlab/gitlab.rb` have been deprecated and can be safely removed.
There are some scenarios like using an external PostgreSQL instance for the tracking database where the FDW settings must be removed manually. Enter the PostgreSQL console of that instance and remove them:
There are some scenarios like using an external PostgreSQL instance for the
tracking database where the FDW settings must be removed manually. Enter the
PostgreSQL console of that instance and remove them:
```shell
```shell
DROP SERVER gitlab_secondary CASCADE;
DROP SERVER gitlab_secondary CASCADE;
...
@@ -40,7 +50,10 @@ upgrade to GitLab 13.4 or later.
...
@@ -40,7 +50,10 @@ upgrade to GitLab 13.4 or later.
## Updating to GitLab 13.2
## Updating to GitLab 13.2
In GitLab 13.2, promoting a secondary node to a primary while the secondary is paused fails. **Do not pause replication before promoting a secondary.** If the node is paused, please resume before promoting. To avoid this issue, upgrade to GitLab 13.4 or later.
In GitLab 13.2, promoting a secondary node to a primary while the secondary is
paused fails. Do not pause replication before promoting a secondary. If the
node is paused, be sure to resume before promoting. To avoid this issue,
upgrade to GitLab 13.4 or later.
## Updating to GitLab 13.0
## Updating to GitLab 13.0
...
@@ -63,12 +76,13 @@ GitLab 12.9.0 through GitLab 12.9.3 are affected by [a bug that stops
...
@@ -63,12 +76,13 @@ GitLab 12.9.0 through GitLab 12.9.3 are affected by [a bug that stops
Only upgrade to GitLab 12.7.5 or later. Do not upgrade to versions 12.7.0
Only upgrade to GitLab 12.7.5 or later. Do not upgrade to versions 12.7.0
through 12.7.4 because there is [an initialization order
through 12.7.4 because there is [an initialization order bug](https://gitlab.com/gitlab-org/gitlab/-/issues/199672) that causes Geo secondaries to set the incorrect database connection pool size.
bug](https://gitlab.com/gitlab-org/gitlab/-/issues/199672) that causes Geo
[The fix](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/24021) was
**secondaries** to set the incorrect database connection pool size. [The
fix](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/24021) was
shipped in 12.7.5.
shipped in 12.7.5.
By default, GitLab 12.7 will attempt to automatically update the embedded
By default, GitLab 12.7 attempts to update the embedded PostgreSQL server
PostgreSQL server to 10.9 from 9.6, which requires downtime on secondaries
version from 9.6 to 10.9, which requires downtime on secondaries while
while reinitializing streaming replication. For the recommended procedure, see
reinitializing streaming replication. For the recommended procedure, see the
the [Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/database.html#upgrading-a-geo-instance).
GitLab 12.2 includes the following minor PostgreSQL updates:
GitLab 12.2 includes the following minor PostgreSQL updates:
- To version `9.6.14` if you run PostgreSQL 9.6.
- To version `9.6.14`, if you run PostgreSQL 9.6.
- To version `10.9` if you run PostgreSQL 10.
- To version `10.9`, if you run PostgreSQL 10.
This update will occur even if major PostgreSQL updates are disabled.
This update occurs even if major PostgreSQL updates are disabled.
Before [refreshing Foreign Data Wrapper during a Geo upgrade](https://docs.gitlab.com/omnibus/update/README.html#run-post-deployment-migrations-and-checks),
Before [refreshing Foreign Data Wrapper during a Geo upgrade](https://docs.gitlab.com/omnibus/update/README.html#run-post-deployment-migrations-and-checks),
restart the Geo tracking database:
restart the Geo tracking database:
...
@@ -189,14 +206,15 @@ restart the Geo tracking database:
...
@@ -189,14 +206,15 @@ restart the Geo tracking database:
sudo gitlab-ctl restart geo-postgresql
sudo gitlab-ctl restart geo-postgresql
```
```
The restart avoids a version mismatch when PostgreSQL tries to load the FDW extension.
The restart avoids a version mismatch when PostgreSQL tries to load the FDW
extension.
## Updating to GitLab 12.1
## Updating to GitLab 12.1
WARNING:
WARNING:
If the existing PostgreSQL server version is 9.6.x, it is recommended to
If the existing PostgreSQL server version is 9.6.x, we recommend upgrading to
upgrade to GitLab 12.4 or later. By default, GitLab 12.1 attempts to update the
GitLab 12.4 or later. By default, GitLab 12.1 attempts to update the embedded
embedded PostgreSQL server from 9.6 to 10.9. In certain circumstances, it will
PostgreSQL server version from 9.6 to 10.9. In certain circumstances, it can
Still in the configuration file, locate and remove the `trust_auth_cidr_address`:
```ruby
postgresql['trust_auth_cidr_addresses']=['127.0.0.1/32','5.6.7.8/32']# <- Remove this
```
1.**(secondary)** Reconfigure and restart:
```shell
sudo gitlab-ctl reconfigure
sudo gitlab-ctl restart
```
## Updating to GitLab 10.5
For Geo Disaster Recovery to work with minimum downtime, your **secondary** node
should use the same set of secrets as the **primary** node. However, setup instructions
prior to the 10.5 release only synchronized the `db_key_base` secret.
To rectify this error on existing installations, you should **overwrite** the
contents of `/etc/gitlab/gitlab-secrets.json` on each **secondary** node with the
contents of `/etc/gitlab/gitlab-secrets.json` on the **primary** node, then run the
following command on each **secondary** node:
```shell
sudo gitlab-ctl reconfigure
```
If you do not perform this step, you may find that two-factor authentication
[is broken following DR](troubleshooting.md#two-factor-authentication-is-broken-after-a-failover).
To prevent SSH requests to the newly promoted **primary** node from failing
due to SSH host key mismatch when updating the **primary** node domain's DNS record
you should perform the step to [Manually replicate **primary** SSH host keys](configuration.md#step-2-manually-replicate-the-primary-nodes-ssh-host-keys) in each
**secondary** node.
## Updating to GitLab 10.3
### Support for SSH repository synchronization removed
In GitLab 10.2, synchronizing secondaries over SSH was deprecated. In 10.3,
support is removed entirely. All installations will switch to the HTTP/HTTPS
cloning method instead. Before updating, ensure that all your Geo nodes are
configured to use this method and that it works for your installation. In
particular, ensure that [Git access over HTTP/HTTPS is enabled](configuration.md#step-5-enable-git-access-over-httphttps).
Synchronizing repositories over the public Internet using HTTP is insecure, so
you should ensure that you have HTTPS configured before updating. Note that
file synchronization is **also** insecure in these cases!
## Updating to GitLab 10.2
### Secure PostgreSQL replication
Support for TLS-secured PostgreSQL replication has been added. If you are
currently using PostgreSQL replication across the open internet without an
external means of securing the connection (e.g., a site-to-site VPN), then you
should immediately reconfigure your **primary** and **secondary** PostgreSQL instances
according to the [updated instructions](../setup/database.md).
If you *are* securing the connections externally and wish to continue doing so,
ensure you include the new option `--sslmode=prefer` in future invocations of
`gitlab-ctl replicate-geo-database`.
### HTTPS repository sync
Support for replicating repositories and wikis over HTTP/HTTPS has been added.
Replicating over SSH has been deprecated, and support for this option will be
removed in a future release.
To switch to HTTP/HTTPS replication, log into the **primary** node as an admin and visit
**Admin Area > Geo** (`/admin/geo/nodes`). For each **secondary** node listed,
press the "Edit" button, change the "Repository cloning" setting from
"SSH (deprecated)" to "HTTP/HTTPS", and press "Save changes". This should take
effect immediately.
Any new secondaries should be created using HTTP/HTTPS replication - this is the
default setting.
After you've verified that HTTP/HTTPS replication is working, you should remove
the now-unused SSH keys from your secondaries, as they may cause problems if the
**secondary** node if ever promoted to a **primary** node:
1.**(secondary)** Login to **all** your **secondary** nodes and run:
```ruby
sudo-ugit-Hrm~git/.ssh/id_rsa~git/.ssh/id_rsa.pub
```
### Hashed Storage
WARNING:
Hashed storage is in **Alpha**. It is considered experimental and not
production-ready. See [Hashed Storage](../../repository_storage_types.md) for more detail.
If you previously enabled Hashed Storage and migrated all your existing
projects to Hashed Storage, disabling hashed storage will not migrate projects
to their previous project based storage path. As such, once enabled and
migrated we recommend leaving Hashed Storage enabled.
## Updating to GitLab 10.1
WARNING:
Hashed storage is in **Alpha**. It is considered experimental and not
production-ready. See [Hashed Storage](../../repository_storage_types.md) for more detail.
[Hashed storage](../../repository_storage_types.md) was introduced in
GitLab 10.0, and a [migration path](../../raketasks/storage.md) for
existing repositories was added in GitLab 10.1.
## Updating to GitLab 10.0
In GitLab 10.0 and later, we require all **Geo** systems to [use SSH key lookups via
the database](../../operations/fast_ssh_key_lookup.md) to avoid having to maintain consistency of the
`authorized_keys` file for SSH access. Failing to do this will prevent users
from being able to clone via SSH.
Note that in older versions of Geo, attachments downloaded on the **secondary**
nodes would be saved to the wrong directory. We recommend that you do the
