Commit d5e72a3b authored by Sean McGivern's avatar Sean McGivern

Merge branch 'docs/upgrades-for-background-migrations' into 'master'

Update the guides for updating GitLab and adding background migrations

See merge request !13284
parents 9b216686 d5cb2943
...@@ -7,6 +7,11 @@ storing data in a single JSON column the data is stored in a separate table. ...@@ -7,6 +7,11 @@ storing data in a single JSON column the data is stored in a separate table.
## When To Use Background Migrations ## When To Use Background Migrations
>**Note:**
When adding background migrations _you must_ make sure they are announced in the
monthly release post along with an estimate of how long it will take to complete
the migrations.
In the vast majority of cases you will want to use a regular Rails migration In the vast majority of cases you will want to use a regular Rails migration
instead. Background migrations should _only_ be used when migrating _data_ in instead. Background migrations should _only_ be used when migrating _data_ in
tables that have so many rows this process would take hours when performed in a tables that have so many rows this process would take hours when performed in a
...@@ -91,6 +96,10 @@ BackgroundMigrationWorker.perform_bulk_in(5.minutes, jobs) ...@@ -91,6 +96,10 @@ BackgroundMigrationWorker.perform_bulk_in(5.minutes, jobs)
## Cleaning Up ## Cleaning Up
>**Note:**
Cleaning up any remaining background migrations _must_ be done in either a major
or minor release, you _must not_ do this in a patch release.
Because background migrations can take a long time you can't immediately clean Because background migrations can take a long time you can't immediately clean
things up after scheduling them. For example, you can't drop a column that's things up after scheduling them. For example, you can't drop a column that's
used in the migration process as this would cause jobs to fail. This means that used in the migration process as this would cause jobs to fail. This means that
......
...@@ -35,12 +35,11 @@ Please don't depend on GitLab-specific code since it can change in future ...@@ -35,12 +35,11 @@ Please don't depend on GitLab-specific code since it can change in future
versions. If needed copy-paste GitLab code into the migration to make it forward versions. If needed copy-paste GitLab code into the migration to make it forward
compatible. compatible.
## Commit Guidelines ## Schema Changes
Each migration **must** be added in its own commit with a descriptive commit Migrations that make changes to the database schema (e.g. adding a column) can
message. If a commit adds a migration it _should only_ include the migration and only be added in the monthly release, patch releases may only contain data
any corresponding changes to `db/schema.rb`. This makes it easy to revert a migrations _unless_ schema changes are absolutely required to solve a problem.
database migration without accidentally reverting other changes.
## Downtime Tagging ## Downtime Tagging
...@@ -224,9 +223,9 @@ add_column(:projects, :foo, :integer, default: 10, limit: 8) ...@@ -224,9 +223,9 @@ add_column(:projects, :foo, :integer, default: 10, limit: 8)
## Timestamp column type ## Timestamp column type
By default, Rails uses the `timestamp` data type that stores timestamp data without timezone information. By default, Rails uses the `timestamp` data type that stores timestamp data without timezone information.
The `timestamp` data type is used by calling either the `add_timestamps` or the `timestamps` method. The `timestamp` data type is used by calling either the `add_timestamps` or the `timestamps` method.
Also Rails converts the `:datetime` data type to the `timestamp` one. Also Rails converts the `:datetime` data type to the `timestamp` one.
Example: Example:
......
...@@ -34,17 +34,67 @@ update them are in [a separate document][omnidocker]. ...@@ -34,17 +34,67 @@ update them are in [a separate document][omnidocker].
## Upgrading without downtime ## Upgrading without downtime
Starting with GitLab 9.1.0 it's possible to upgrade to a newer major, minor, or patch version of GitLab Starting with GitLab 9.1.0 it's possible to upgrade to a newer major, minor, or
without having to take your GitLab instance offline. However, for this to work patch version of GitLab without having to take your GitLab instance offline.
there are the following requirements: However, for this to work there are the following requirements:
1. You can only upgrade 1 minor release at a time. So from 9.1 to 9.2, not to 9.3. 1. You can only upgrade 1 minor release at a time. So from 9.1 to 9.2, not to
2. You have to be on the most recent patch release. For example, if 9.1.15 is the last 9.3.
release of 9.1 then you can safely upgrade from that version to any 9.2.x version.
However, if you are running 9.1.14 you first need to upgrade to 9.1.15.
2. You have to use [post-deployment 2. You have to use [post-deployment
migrations](../development/post_deployment_migrations.md). migrations](../development/post_deployment_migrations.md).
3. You are using PostgreSQL. If you are using MySQL please look at the release post to see if downtime is required. 3. You are using PostgreSQL. If you are using MySQL please look at the release
post to see if downtime is required.
Most of the time you can safely upgrade from a patch release to the next minor
release if the patch release is not the latest. For example, upgrading from
9.1.1 to 9.2.0 should be safe even if 9.1.2 has been released. We do recommend
you check the release posts of any releases between your current and target
version just in case they include any migrations that may require you to upgrade
1 release at a time.
Some releases may also include so called "background migrations". These
migrations are performed in the background by Sidekiq and are often used for
migrating data. Background migrations are only added in the monthly releases.
Certain major/minor releases may require a set of background migrations to be
finished. To guarantee this such a release will process any remaining jobs
before continuing the upgrading procedure. While this won't require downtime
(if the above conditions are met) we recommend users to keep at least 1 week
between upgrading major/minor releases, allowing the background migrations to
finish. The time necessary to complete these migrations can be reduced by
increasing the number of Sidekiq workers that can process jobs in the
`background_migration` queue.
As a rule of thumb, any database smaller than 10 GB won't take too much time to
upgrade; perhaps an hour at most per minor release. Larger databases however may
require more time, but this is highly dependent on the size of the database and
the migrations that are being performed.
### Examples
To help explain this, let's look at some examples.
**Example 1:** You are running a large GitLab installation using version 9.4.2,
which is the latest patch release of 9.4. When GitLab 9.5.0 is released this
installation can be safely upgraded to 9.5.0 without requiring downtime if the
requirements mentioned above are met. You can also skip 9.5.0 and upgrade to
9.5.1 once it's released, but you **can not** upgrade straight to 9.6.0; you
_have_ to first upgrade to a 9.5.x release.
**Example 2:** You are running a large GitLab installation using version 9.4.2,
which is the latest patch release of 9.4. GitLab 9.5 includes some background
migrations, and 10.0 will require these to be completed (processing any
remaining jobs for you). Skipping 9.5 is not possible without downtime, and due
to the background migrations would require potentially hours of downtime
depending on how long it takes for the background migrations to complete. To
work around this you will have to upgrade to 9.5.x first, then wait at least a
week before upgrading to 10.0.
**Example 3:** You use MySQL as the database for GitLab. Any upgrade to a new
major/minor release will require downtime. If a release includes any background
migrations this could potentially lead to hours of downtime, depending on the
size of your database. To work around this you will have to use PostgreSQL and
meet the other online upgrade requirements mentioned above.
## Upgrading between editions ## Upgrading between editions
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment