@@ -14,7 +14,7 @@ If you have any doubts about the consistency of the data on this node, we recomm
...
@@ -14,7 +14,7 @@ If you have any doubts about the consistency of the data on this node, we recomm
Since the former **primary** node will be out of sync with the current **primary** node, the first step is to bring the former **primary** node up to date. Note, deletion of data stored on disk like
Since the former **primary** node will be out of sync with the current **primary** node, the first step is to bring the former **primary** node up to date. Note, deletion of data stored on disk like
repositories and uploads will not be replayed when bringing the former **primary** node back
repositories and uploads will not be replayed when bringing the former **primary** node back
into sync, which may result in increased disk usage.
into sync, which may result in increased disk usage.
Alternatively, you can [set up a new **secondary** GitLab instance][setup-geo] to avoid this.
Alternatively, you can [set up a new **secondary** GitLab instance](../replication/index.md#setup-instructions) to avoid this.
To bring the former **primary** node up to date:
To bring the former **primary** node up to date:
...
@@ -25,28 +25,28 @@ To bring the former **primary** node up to date:
...
@@ -25,28 +25,28 @@ To bring the former **primary** node up to date:
sudo gitlab-ctl start
sudo gitlab-ctl start
```
```
NOTE: **Note:** If you [disabled the **primary** node permanently][disaster-recovery-disable-primary],
NOTE: **Note:** If you [disabled the **primary** node permanently](index.md#step-2-permanently-disable-the-primary-node),
you need to undo those steps now. For Debian/Ubuntu you just need to run
you need to undo those steps now. For Debian/Ubuntu you just need to run
`sudo systemctl enable gitlab-runsvdir`. For CentOS 6, you need to install
`sudo systemctl enable gitlab-runsvdir`. For CentOS 6, you need to install
the GitLab instance from scratch and set it up as a **secondary** node by
the GitLab instance from scratch and set it up as a **secondary** node by
following [Setup instructions][setup-geo]. In this case, you don't need to follow the next step.
following [Setup instructions](../replication/index.md#setup-instructions). In this case, you don't need to follow the next step.
NOTE: **Note:** If you [changed the DNS records](index.md#step-4-optional-updating-the-primary-domain-dns-record)
NOTE: **Note:** If you [changed the DNS records](index.md#step-4-optional-updating-the-primary-domain-dns-record)
for this node during disaster recovery procedure you may need to [block
for this node during disaster recovery procedure you may need to [block
all the writes to this node](planned_failover.md#prevent-updates-to-the-primary-node)
all the writes to this node](planned_failover.md#prevent-updates-to-the-primary-node)
during this procedure.
during this procedure.
1.[Setup database replication][database-replication]. Note that in this
1.[Setup database replication](../replication/database.md). Note that in this
case, **primary** node refers to the current **primary** node, and **secondary** node refers to the
case, **primary** node refers to the current **primary** node, and **secondary** node refers to the
former **primary** node.
former **primary** node.
If you have lost your original **primary** node, follow the
If you have lost your original **primary** node, follow the
[setup instructions][setup-geo] to set up a new **secondary** node.
[setup instructions](../replication/index.md#setup-instructions) to set up a new **secondary** node.
## Promote the **secondary** node to **primary** node
## Promote the **secondary** node to **primary** node
When the initial replication is complete and the **primary** node and **secondary** node are
When the initial replication is complete and the **primary** node and **secondary** node are
closely in sync, you can do a [planned failover].
closely in sync, you can do a [planned failover](planned_failover.md).
## Restore the **secondary** node
## Restore the **secondary** node
...
@@ -54,8 +54,3 @@ If your objective is to have two nodes again, you need to bring your **secondary
...
@@ -54,8 +54,3 @@ If your objective is to have two nodes again, you need to bring your **secondary
node back online as well by repeating the first step
node back online as well by repeating the first step
([configure the former **primary** node to be a **secondary** node](#configure-the-former-primary-node-to-be-a-secondary-node))
([configure the former **primary** node to be a **secondary** node](#configure-the-former-primary-node-to-be-a-secondary-node))
| Uploads | **Yes** | [No][upload-verification] | Verified only on transfer, or manually (*1*)|
| Uploads | **Yes** | [No](https://gitlab.com/groups/gitlab-org/-/epics/1817) | Verified only on transfer, or manually (*1*) |
| LFS objects | **Yes** | [No][lfs-verification] | Verified only on transfer, or manually (*1*). Unavailable for new LFS objects in 11.11.x and 12.0.x (*2*). |
| LFS objects | **Yes** | [No](https://gitlab.com/gitlab-org/gitlab/issues/8922) | Verified only on transfer, or manually (*1*). Unavailable for new LFS objects in 11.11.x and 12.0.x (*2*). |
| CI job artifacts (other than traces) | **Yes** | [No][artifact-verification] | Verified only manually (*1*) |
| CI job artifacts (other than traces) | **Yes** | [No](https://gitlab.com/gitlab-org/gitlab/issues/8923) | Verified only manually (*1*) |
| Archived traces | **Yes** | [No][artifact-verification] | Verified only on transfer, or manually (*1*)|
| Archived traces | **Yes** | [No](https://gitlab.com/gitlab-org/gitlab/issues/8923) | Verified only on transfer, or manually (*1*) |
| Personal snippets | **Yes** | **Yes** | |
| Personal snippets | **Yes** | **Yes** | |
| Project snippets | **Yes** | **Yes** | |
| Project snippets | **Yes** | **Yes** | |
| Object pools for forked project deduplication | **Yes** | No | |
| Object pools for forked project deduplication | **Yes** | No | |
| [Server-side Git Hooks][custom-hooks] | No | No | |
| [Server-side Git Hooks](../../custom_hooks.md) | No | No | |
| [Elasticsearch integration][elasticsearch] | [No][elasticsearch-replication] | No | |
| [Elasticsearch integration](../../../integration/elasticsearch.md) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/1186) | No | |
| [GitLab Pages][gitlab-pages] | [No][pages-replication] | No | |
| [GitLab Pages](../../pages/index.md) | [No](https://gitlab.com/groups/gitlab-org/-/epics/589) | No | |
| [Container Registry][container-registry] | **Yes** | No | |
| [Container Registry](../../packages/container_registry.md) | **Yes** | No | |
| [NPM Registry][npm-registry] | [No][packages-replication] | No | |
| [NPM Registry](../../../user/packages/npm_registry/index.md) | [No](https://gitlab.com/groups/gitlab-org/-/epics/2346) | No | |
| [Maven Repository][maven-repository] | [No][packages-replication] | No | |
| [Maven Repository](../../../user/packages/maven_repository/index.md) | [No](https://gitlab.com/groups/gitlab-org/-/epics/2346) | No | |
| [Conan Repository][conan-repository] | [No][packages-replication] | No | |
| [Conan Repository](../../../user/packages/conan_repository/index.md) | [No](https://gitlab.com/groups/gitlab-org/-/epics/2346) | No | |
| [NuGet Repository][nuget-repository] | [No][packages-replication] | No | |
| [NuGet Repository](../../../user/packages/nuget_repository/index.md) | [No](https://gitlab.com/groups/gitlab-org/-/epics/2346) | No | |
After you set up the [database replication and configure the Geo nodes][req], use your closest GitLab node as you would a normal standalone GitLab instance.
After you set up the [database replication and configure the Geo nodes](index.md#setup-instructions), use your closest GitLab node as you would a normal standalone GitLab instance.
Pushing directly to a **secondary** node (for both HTTP, SSH including Git LFS) was [introduced](https://about.gitlab.com/releases/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3.
Pushing directly to a **secondary** node (for both HTTP, SSH including Git LFS) was [introduced](https://about.gitlab.com/releases/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3.
1.[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
1.[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
1. Note the Redis node's IP address or hostname, port, and
1. Note the Redis node's IP address or hostname, port, and
Redis password. These will be necessary when configuring the GitLab
Redis password. These will be necessary when configuring the GitLab
application servers later.
application servers later.
...
@@ -88,13 +88,13 @@ Continue configuration of other components by going back to the
...
@@ -88,13 +88,13 @@ Continue configuration of other components by going back to the
### High Availability with GitLab Omnibus **(PREMIUM ONLY)**
### High Availability with GitLab Omnibus **(PREMIUM ONLY)**
> Experimental Redis Sentinel support was [introduced in GitLab 8.11][ce-1877].
> Experimental Redis Sentinel support was [introduced in GitLab 8.11](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/1877).
Starting with 8.14, Redis Sentinel is no longer experimental.
Starting with 8.14, Redis Sentinel is no longer experimental.
If you've used it with versions `< 8.14` before, please check the updated
If you've used it with versions `< 8.14` before, please check the updated
documentation here.
documentation here.
High Availability with [Redis] is possible using a **Master** x **Slave**
High Availability with [Redis](https://redis.io/) is possible using a **Master** x **Slave**
topology with a [Redis Sentinel][sentinel] service to watch and automatically
topology with a [Redis Sentinel](https://redis.io/topics/sentinel) service to watch and automatically
start the failover procedure.
start the failover procedure.
You can choose to install and manage Redis and Sentinel yourself, use
You can choose to install and manage Redis and Sentinel yourself, use
...
@@ -107,7 +107,7 @@ Omnibus GitLab packages.
...
@@ -107,7 +107,7 @@ Omnibus GitLab packages.
> [Redis Security](https://redis.io/topics/security) documentation for more
> [Redis Security](https://redis.io/topics/security) documentation for more
> information. We recommend using a combination of a Redis password and tight
> information. We recommend using a combination of a Redis password and tight
> firewall rules to secure your Redis service.
> firewall rules to secure your Redis service.
> - You are highly encouraged to read the [Redis Sentinel][sentinel] documentation
> - You are highly encouraged to read the [Redis Sentinel](https://redis.io/topics/sentinel) documentation
> before configuring Redis HA with GitLab to fully understand the topology and
> before configuring Redis HA with GitLab to fully understand the topology and
> architecture.
> architecture.
> - This is the documentation for the Omnibus GitLab packages. For installations
> - This is the documentation for the Omnibus GitLab packages. For installations
...
@@ -296,7 +296,7 @@ multiple ways to configure Redis HA. Omnibus GitLab packages have Redis and/or
...
@@ -296,7 +296,7 @@ multiple ways to configure Redis HA. Omnibus GitLab packages have Redis and/or
Redis Sentinel bundled with them so you only need to focus on configuration.
Redis Sentinel bundled with them so you only need to focus on configuration.
Pick the one that suits your needs.
Pick the one that suits your needs.
-[Installations from source][source]: You need to install Redis and Sentinel
-[Installations from source](../../install/installation.md): You need to install Redis and Sentinel
yourself. Use the [Redis HA installation from source](redis_source.md)
yourself. Use the [Redis HA installation from source](redis_source.md)
documentation.
documentation.
-[Omnibus GitLab **Community Edition** (CE) package](https://about.gitlab.com/install/?version=ce): Redis is bundled, so you
-[Omnibus GitLab **Community Edition** (CE) package](https://about.gitlab.com/install/?version=ce): Redis is bundled, so you
...
@@ -341,7 +341,7 @@ The prerequisites for a HA Redis setup are the following:
...
@@ -341,7 +341,7 @@ The prerequisites for a HA Redis setup are the following:
change the default ones).
change the default ones).
1. The server that hosts the GitLab application must be able to access the
1. The server that hosts the GitLab application must be able to access the
Redis nodes.
Redis nodes.
1. Protect the nodes from access from external networks ([Internet][it]), using
1. Protect the nodes from access from external networks ([Internet](https://gitlab.com/gitlab-org/gitlab-foss/uploads/c4cc8cd353604bd80315f9384035ff9e/The_Internet_IT_Crowd.png)), using
firewall.
firewall.
### Step 1. Configuring the master Redis instance
### Step 1. Configuring the master Redis instance
...
@@ -381,7 +381,7 @@ The prerequisites for a HA Redis setup are the following:
...
@@ -381,7 +381,7 @@ The prerequisites for a HA Redis setup are the following:
gitlab_rails['auto_migrate']=false
gitlab_rails['auto_migrate']=false
```
```
1.[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
1.[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
> Note: You can specify multiple roles like sentinel and Redis as:
> Note: You can specify multiple roles like sentinel and Redis as:
> `roles ['redis_sentinel_role', 'redis_master_role']`. Read more about high
> `roles ['redis_sentinel_role', 'redis_master_role']`. Read more about high
...
@@ -429,7 +429,7 @@ The prerequisites for a HA Redis setup are the following:
...
@@ -429,7 +429,7 @@ The prerequisites for a HA Redis setup are the following:
sudo touch /etc/gitlab/skip-auto-reconfigure
sudo touch /etc/gitlab/skip-auto-reconfigure
```
```
1.[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
1.[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
1. Go through the steps again for all the other slave nodes.
1. Go through the steps again for all the other slave nodes.
> Note: You can specify multiple roles like sentinel and Redis as:
> Note: You can specify multiple roles like sentinel and Redis as:
...
@@ -561,7 +561,7 @@ multiple machines with the Sentinel daemon.
...
@@ -561,7 +561,7 @@ multiple machines with the Sentinel daemon.
Only the primary GitLab application server should handle migrations.
Only the primary GitLab application server should handle migrations.
1.[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
1.[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
1. Go through the steps again for all the other Sentinel nodes.
1. Go through the steps again for all the other Sentinel nodes.
### Step 4. Configuring the GitLab application
### Step 4. Configuring the GitLab application
...
@@ -598,7 +598,7 @@ which ideally should not have Redis or Sentinels on it for a HA setup.
...
@@ -598,7 +598,7 @@ which ideally should not have Redis or Sentinels on it for a HA setup.
]
]
```
```
1.[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
1.[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
## Switching from an existing single-machine installation to Redis HA
## Switching from an existing single-machine installation to Redis HA
...
@@ -677,7 +677,7 @@ sentinel['quorum'] = 2
...
@@ -677,7 +677,7 @@ sentinel['quorum'] = 2
# sentinel['failover_timeout'] = 60000
# sentinel['failover_timeout'] = 60000
```
```
[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
### Example configuration for Redis slave 1 and Sentinel 2
### Example configuration for Redis slave 1 and Sentinel 2
...
@@ -699,7 +699,7 @@ sentinel['quorum'] = 2
...
@@ -699,7 +699,7 @@ sentinel['quorum'] = 2
# sentinel['failover_timeout'] = 60000
# sentinel['failover_timeout'] = 60000
```
```
[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
### Example configuration for Redis slave 2 and Sentinel 3
### Example configuration for Redis slave 2 and Sentinel 3
...
@@ -721,7 +721,7 @@ sentinel['quorum'] = 2
...
@@ -721,7 +721,7 @@ sentinel['quorum'] = 2
# sentinel['failover_timeout'] = 60000
# sentinel['failover_timeout'] = 60000
```
```
[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
### Example configuration for the GitLab application
### Example configuration for the GitLab application
[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect.
[Reconfigure Omnibus GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
## Enable Monitoring
## Enable Monitoring
...
@@ -862,7 +862,7 @@ mailroom['enable'] = false
...
@@ -862,7 +862,7 @@ mailroom['enable'] = false
redis['master']=false
redis['master']=false
```
```
You can find the relevant attributes defined in [`gitlab_rails.rb`][omnifile].
You can find the relevant attributes defined in [`gitlab_rails.rb`](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/libraries/gitlab_rails.rb).
## Troubleshooting
## Troubleshooting
...
@@ -929,7 +929,7 @@ repl_backlog_histlen:0
...
@@ -929,7 +929,7 @@ repl_backlog_histlen:0
If you get an error like: `Redis::CannotConnectError: No sentinels available.`,
If you get an error like: `Redis::CannotConnectError: No sentinels available.`,
there may be something wrong with your configuration files or it can be related
there may be something wrong with your configuration files or it can be related
to [this issue][gh-531].
to [this issue](https://github.com/redis/redis-rb/issues/531).
You must make sure you are defining the same value in `redis['master_name']`
You must make sure you are defining the same value in `redis['master_name']`
and `redis['master_pasword']` as you defined for your sentinel node.
and `redis['master_pasword']` as you defined for your sentinel node.
...
@@ -1001,14 +1001,3 @@ Read more on High Availability:
...
@@ -1001,14 +1001,3 @@ Read more on High Availability:
1.[Configure NFS](nfs.md)
1.[Configure NFS](nfs.md)
1.[Configure the GitLab application servers](gitlab.md)
1.[Configure the GitLab application servers](gitlab.md)
1.[Configure the load balancers](load_balancer.md)
1.[Configure the load balancers](load_balancer.md)
"description":"Vulnerabilities reported on the projects in the group and its subgroups. Available only when feature flag `first_class_vulnerabilities` is enabled",
"args":[
{
"name":"projectId",
"description":"Filter vulnerabilities by project",
"type":{
"kind":"LIST",
"name":null,
"ofType":{
"kind":"NON_NULL",
"name":null,
"ofType":{
"kind":"SCALAR",
"name":"ID",
"ofType":null
}
}
},
"defaultValue":null
},
{
"name":"reportType",
"description":"Filter vulnerabilities by report type",
"type":{
"kind":"LIST",
"name":null,
"ofType":{
"kind":"NON_NULL",
"name":null,
"ofType":{
"kind":"ENUM",
"name":"VulnerabilityReportType",
"ofType":null
}
}
},
"defaultValue":null
},
{
"name":"severity",
"description":"Filter vulnerabilities by severity",
"type":{
"kind":"LIST",
"name":null,
"ofType":{
"kind":"NON_NULL",
"name":null,
"ofType":{
"kind":"ENUM",
"name":"VulnerabilitySeverity",
"ofType":null
}
}
},
"defaultValue":null
},
{
"name":"state",
"description":"Filter vulnerabilities by state",
"type":{
"kind":"LIST",
"name":null,
"ofType":{
"kind":"NON_NULL",
"name":null,
"ofType":{
"kind":"ENUM",
"name":"VulnerabilityState",
"ofType":null
}
}
},
"defaultValue":null
},
{
"name":"after",
"description":"Returns the elements in the list that come after the specified cursor.",
"type":{
"kind":"SCALAR",
"name":"String",
"ofType":null
},
"defaultValue":null
},
{
"name":"before",
"description":"Returns the elements in the list that come before the specified cursor.",
"type":{
"kind":"SCALAR",
"name":"String",
"ofType":null
},
"defaultValue":null
},
{
"name":"first",
"description":"Returns the first _n_ elements from the list.",
"type":{
"kind":"SCALAR",
"name":"Int",
"ofType":null
},
"defaultValue":null
},
{
"name":"last",
"description":"Returns the last _n_ elements from the list.",
The [`Project#feature_available?`](https://gitlab.com/gitlab-org/gitlab/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/app/models/project_feature.rb#L63-68),
[`Namespace#feature_available?`][namespace-fa] (EE), and
[`Namespace#feature_available?`](https://gitlab.com/gitlab-org/gitlab/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/ee/namespace.rb#L71-85)(EE), and
[`License.feature_available?`][license-fa] (EE) methods all implicitly check for
[`License.feature_available?`](https://gitlab.com/gitlab-org/gitlab/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/license.rb#L293-300)(EE) methods all implicitly check for
a by default enabled feature flag with the same name as the provided argument.
a by default enabled feature flag with the same name as the provided argument.
For example if a feature is license-gated, there's no need to add an additional
For example if a feature is license-gated, there's no need to add an additional
...
@@ -49,10 +49,6 @@ feature flag once the feature has reached general availability.
...
@@ -49,10 +49,6 @@ feature flag once the feature has reached general availability.
You'd still want to use an explicit `Feature.enabled?` check if your new feature
You'd still want to use an explicit `Feature.enabled?` check if your new feature
In May 2019, Bob Van Landuyt hosted a [Deep Dive] on GitLab's [Gitaly project] and how to contribute to it as a Ruby developer, to share his domain specific knowledge with anyone who may work in this part of the code base in the future. You can find the [recording on YouTube], and the slides on [Google Slides] and in [PDF]. Everything covered in this deep dive was accurate as of GitLab 11.11, and while specific details may have changed since then, it should still serve as a good introduction.
In May 2019, Bob Van Landuyt hosted a [Deep Dive](https://gitlab.com/gitlab-org/create-stage/issues/1)
on GitLab's [Gitaly project](https://gitlab.com/gitlab-org/gitaly) and how to contribute to it as a
Ruby developer, to share his domain specific knowledge with anyone who may work in this part of the