Commit 3a426da1 authored by Sean McGivern's avatar Sean McGivern

Rename queue-selector to experimental-queue-selector

We want the ability to react quickly to changes, so while this is
documented, we need to make sure that people are aware that using this
while it's still experimental is a significant risk of having to deal
with breaking changes.
parent 95a74031
......@@ -86,9 +86,18 @@ you list:
> [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/issues/45) in [GitLab Starter](https://about.gitlab.com/pricing/) 12.8.
In addition to selecting queues by name, as above, the `queue_selector`
option allows queue groups to be selected in a more general way using
the following components:
CAUTION: **Caution:**
As this is marked as **experimental**, it is subject to change at any
time, including **breaking backwards compatibility**. This is so that we
can react to changes we need for our GitLab.com deployment. We have a
tracking issue open to [remove the experimental
designation](https://gitlab.com/gitlab-com/gl-infra/scalability/issues/147)
from this feature; please comment there if you are interested in using
this in your own deployment.
In addition to selecting queues by name, as above, the
`experimental_queue_selector` option allows queue groups to be selected
in a more general way using the following components:
- Attributes that can be selected.
- Operators used to construct a query.
......@@ -97,7 +106,8 @@ the following components:
From the [list of all available
attributes](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/workers/all_queues.yml),
`queue_selector` allows selecting of queues by the following attributes:
`experimental_queue_selector` allows selecting of queues by the
following attributes:
- `feature_category` - the [GitLab feature
category](https://about.gitlab.com/direction/maturity/#category-maturity) the
......@@ -122,8 +132,8 @@ considered false.
### Available operators
`queue_selector` supports the following operators, listed from highest
to lowest precedence:
`experimental_queue_selector` supports the following operators, listed
from highest to lowest precedence:
- `|` - the logical OR operator. For example, `query_a|query_b` (where `query_a`
and `query_b` are queries made up of the other operators here) will include
......@@ -150,7 +160,7 @@ In `/etc/gitlab/gitlab.rb`:
```ruby
sidekiq_cluster['enable'] = true
sidekiq_cluster['queue_selector'] = true
sidekiq_cluster['experimental_queue_selector'] = true
sidekiq_cluster['queue_groups'] = [
# Run all non-CPU-bound queues that are latency sensitive
'resource_boundary!=cpu&latency_sensitive=true',
......
......@@ -45,7 +45,7 @@ module Gitlab
queue_names = SidekiqConfig::CliMethods.worker_queues(@rails_path)
queue_groups =
if @queue_selector
if @experimental_queue_selector
# When using the experimental queue query syntax, we treat
# each queue group as a worker attribute query, and resolve
# the queues for the queue group using this query.
......@@ -160,8 +160,8 @@ module Gitlab
@rails_path = path
end
opt.on('--queue-selector', 'EXPERIMENTAL: Run workers based on the provided selector') do |queue_selector|
@queue_selector = queue_selector
opt.on('--experimental-queue-selector', 'EXPERIMENTAL: Run workers based on the provided selector') do |experimental_queue_selector|
@experimental_queue_selector = experimental_queue_selector
end
opt.on('-n', '--negate', 'Run workers for all queues in sidekiq_queues.yml except the given ones') do
......
......@@ -7,7 +7,7 @@ describe 'ee/bin/sidekiq-cluster' do
where(:args, :included, :excluded) do
%w[--negate cronjob] | '-qdefault,1' | '-qcronjob,1'
%w[--queue-selector resource_boundary=cpu] | '-qupdate_merge_requests,1' | '-qdefault,1'
%w[--experimental-queue-selector resource_boundary=cpu] | '-qupdate_merge_requests,1' | '-qdefault,1'
end
with_them do
......
......@@ -79,7 +79,7 @@ describe Gitlab::SidekiqCluster::CLI do
end
end
context 'with --queue-selector' do
context 'with --experimental-queue-selector' do
where do
{
'memory-bound queues' => {
......@@ -125,7 +125,7 @@ describe Gitlab::SidekiqCluster::CLI do
[]
end
cli.run(%W(--queue-selector #{query}))
cli.run(%W(--experimental-queue-selector #{query}))
end
it 'works when negated' do
......@@ -137,7 +137,7 @@ describe Gitlab::SidekiqCluster::CLI do
[]
end
cli.run(%W(--negate --queue-selector #{query}))
cli.run(%W(--negate --experimental-queue-selector #{query}))
end
end
......@@ -147,13 +147,13 @@ describe Gitlab::SidekiqCluster::CLI do
.with([['chat_notification'], ['project_export']], default_options)
.and_return([])
cli.run(%w(--queue-selector feature_category=chatops&latency_sensitive=true resource_boundary=memory&feature_category=source_code_management))
cli.run(%w(--experimental-queue-selector feature_category=chatops&latency_sensitive=true resource_boundary=memory&feature_category=importers))
end
it 'errors on an invalid query multiple queue groups correctly' do
expect(Gitlab::SidekiqCluster).not_to receive(:start)
expect { cli.run(%w(--queue-selector unknown_field=chatops)) }
expect { cli.run(%w(--experimental-queue-selector unknown_field=chatops)) }
.to raise_error(Gitlab::SidekiqConfig::CliMethods::QueryError)
end
end
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment