Commit 7c8c5936 authored by David Fernandez's avatar David Fernandez

Merge branch '334494-enable-sidekiq-lb-by-default' into 'master'

Enable Sidekiq load balancing by default

See merge request gitlab-org/gitlab!65669
parents 9ef27335 a36a0b84
...@@ -104,32 +104,19 @@ the following. This balances the load between `host1.example.com` and ...@@ -104,32 +104,19 @@ the following. This balances the load between `host1.example.com` and
1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect. 1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect.
### Enable the load balancer for Sidekiq ### Load balancing for Sidekiq
Sidekiq mostly writes to the database, which means that most of its traffic hits the > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334494) in GitLab 14.1, load balancing for Sidekick is enabled by default.
primary database.
Some background jobs can use database replicas to read application state. Sidekiq jobs mostly write to the primary database, but there are read-only jobs that can benefit
from the use of Sidekiq load balancing.
These jobs can use load balancing and database replicas to read the application state.
This allows to offload the primary database. This allows to offload the primary database.
Load balancing is disabled by default in Sidekiq. When enabled, we can define For Sidekiq, we can define
[the data consistency](../development/sidekiq_style_guide.md#job-data-consistency-strategies) [data consistency](../development/sidekiq_style_guide.md#job-data-consistency-strategies)
requirements for a specific job. requirements for a specific job.
To enable it, define the `ENABLE_LOAD_BALANCING_FOR_SIDEKIQ` variable to the environment, as shown below.
For Omnibus installations:
```ruby
gitlab_rails['env'] = {"ENABLE_LOAD_BALANCING_FOR_SIDEKIQ" => "true"}
```
For installations from source:
```shell
export ENABLE_LOAD_BALANCING_FOR_SIDEKIQ="true"
```
## Service Discovery ## Service Discovery
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/5883) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.0. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/5883) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.0.
......
...@@ -478,7 +478,7 @@ of reading a stale record is non-zero due to replicas potentially lagging behind ...@@ -478,7 +478,7 @@ of reading a stale record is non-zero due to replicas potentially lagging behind
When the number of jobs that rely on the database increases, ensuring immediate data consistency When the number of jobs that rely on the database increases, ensuring immediate data consistency
can put unsustainable load on the primary database server. We therefore added the ability to use can put unsustainable load on the primary database server. We therefore added the ability to use
[database load-balancing in Sidekiq workers](../administration/database_load_balancing.md#enable-the-load-balancer-for-sidekiq). [database load balancing for Sidekiq workers](../administration/database_load_balancing.md#load-balancing-for-sidekiq).
By configuring a worker's `data_consistency` field, we can then allow the scheduler to target read replicas By configuring a worker's `data_consistency` field, we can then allow the scheduler to target read replicas
under several strategies outlined below. under several strategies outlined below.
......
...@@ -85,7 +85,6 @@ module Gitlab ...@@ -85,7 +85,6 @@ module Gitlab
# Returns true if load balancing is to be enabled. # Returns true if load balancing is to be enabled.
def self.enable? def self.enable?
return false if Gitlab::Runtime.rake? return false if Gitlab::Runtime.rake?
return false if Gitlab::Runtime.sidekiq? && !Gitlab::Utils.to_boolean(ENV['ENABLE_LOAD_BALANCING_FOR_SIDEKIQ'], default: false)
return false unless self.configured? return false unless self.configured?
true true
......
...@@ -142,10 +142,10 @@ RSpec.describe Gitlab::Database::LoadBalancing do ...@@ -142,10 +142,10 @@ RSpec.describe Gitlab::Database::LoadBalancing do
expect(described_class.enable?).to eq(false) expect(described_class.enable?).to eq(false)
end end
it 'returns false when Sidekiq is being used' do it 'returns true when Sidekiq is being used' do
allow(Gitlab::Runtime).to receive(:sidekiq?).and_return(true) allow(Gitlab::Runtime).to receive(:sidekiq?).and_return(true)
expect(described_class.enable?).to eq(false) expect(described_class.enable?).to eq(true)
end end
it 'returns false when running inside a Rake task' do it 'returns false when running inside a Rake task' do
...@@ -170,18 +170,6 @@ RSpec.describe Gitlab::Database::LoadBalancing do ...@@ -170,18 +170,6 @@ RSpec.describe Gitlab::Database::LoadBalancing do
expect(described_class.enable?).to eq(true) expect(described_class.enable?).to eq(true)
end end
context 'when ENABLE_LOAD_BALANCING_FOR_SIDEKIQ environment variable is set' do
before do
stub_env('ENABLE_LOAD_BALANCING_FOR_SIDEKIQ', 'true')
end
it 'returns true when Sidekiq is being used' do
allow(Gitlab::Runtime).to receive(:sidekiq?).and_return(true)
expect(described_class.enable?).to eq(true)
end
end
end end
describe '.configured?' do describe '.configured?' do
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment