Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
1
Merge Requests
1
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
nexedi
gitlab-ce
Commits
ccd26150
Commit
ccd26150
authored
Feb 10, 2020
by
Sean McGivern
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Rename queue-query-syntax to queue-selector
queue-query-syntax is really hard to type!
parent
b6db5a1e
Changes
5
Show whitespace changes
Inline
Side-by-side
Showing
5 changed files
with
16 additions
and
16 deletions
+16
-16
changelogs/unreleased/an-sidekiq-query.yml
changelogs/unreleased/an-sidekiq-query.yml
+1
-1
doc/administration/operations/extra_sidekiq_processes.md
doc/administration/operations/extra_sidekiq_processes.md
+6
-6
ee/lib/gitlab/sidekiq_cluster/cli.rb
ee/lib/gitlab/sidekiq_cluster/cli.rb
+3
-3
ee/spec/bin/sidekiq_cluster_spec.rb
ee/spec/bin/sidekiq_cluster_spec.rb
+1
-1
ee/spec/lib/gitlab/sidekiq_cluster/cli_spec.rb
ee/spec/lib/gitlab/sidekiq_cluster/cli_spec.rb
+5
-5
No files found.
changelogs/unreleased/an-sidekiq-query.yml
View file @
ccd26150
---
title
:
Add experimental --queue-
query-syntax
option to sidekiq-cluster
title
:
Add experimental --queue-
selector
option to sidekiq-cluster
merge_request
:
18877
author
:
type
:
changed
doc/administration/operations/extra_sidekiq_processes.md
View file @
ccd26150
...
...
@@ -82,11 +82,11 @@ you list:
sudo
gitlab-ctl reconfigure
```
## Queue
query syntax
(experimental)
## Queue
selector
(experimental)
> [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/issues/45) in [GitLab Starter](https://about.gitlab.com/pricing/) 12.8.
In addition to selecting queues by name, as above, the
`queue_
query_syntax
`
In addition to selecting queues by name, as above, the
`queue_
selector
`
option allows queue groups to be selected in a more general way using
the following components:
...
...
@@ -97,7 +97,7 @@ the following components:
From the
[
list of all available
attributes
](
https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/workers/all_queues.yml
)
,
`queue_
query_syntax
`
allows selecting of queues by the following attributes:
`queue_
selector
`
allows selecting of queues by the following attributes:
-
`feature_category`
- the
[
GitLab feature
category
](
https://about.gitlab.com/direction/maturity/#category-maturity
)
the
...
...
@@ -122,8 +122,8 @@ considered false.
### Available operators
`queue_
query_syntax`
supports the following operators, listed from highest to
lowest precedence:
`queue_
selector`
supports the following operators, listed from highest
to
lowest precedence:
-
<code>
</code>
(space) - the logical OR operator. For example,
`query_a
query_b`
(where
`query_a`
and
`query_b`
are queries made up of the other
...
...
@@ -151,7 +151,7 @@ In `/etc/gitlab/gitlab.rb`:
```
ruby
sidekiq_cluster
[
'enable'
]
=
true
sidekiq_cluster
[
'queue_
query_syntax
'
]
=
true
sidekiq_cluster
[
'queue_
selector
'
]
=
true
sidekiq_cluster
[
'queue_groups'
]
=
[
# Run all non-CPU-bound, queues that are latency sensitive
'resource_boundary!=cpu,latency_sensitive=true'
,
...
...
ee/lib/gitlab/sidekiq_cluster/cli.rb
View file @
ccd26150
...
...
@@ -45,7 +45,7 @@ module Gitlab
queue_names
=
SidekiqConfig
::
CliMethods
.
worker_queues
(
@rails_path
)
queue_groups
=
if
@queue_
query_syntax
if
@queue_
selector
# When using the experimental queue query syntax, we treat
# each queue group as a worker attribute query, and resolve
# the queues for the queue group using this query.
...
...
@@ -160,8 +160,8 @@ module Gitlab
@rails_path
=
path
end
opt
.
on
(
'--queue-
query-syntax'
,
'EXPERIMENTAL: Run workers based on the provided selector'
)
do
|
queue_query_syntax
|
@queue_
query_syntax
=
queue_query_syntax
opt
.
on
(
'--queue-
selector'
,
'EXPERIMENTAL: Run workers based on the provided selector'
)
do
|
queue_selector
|
@queue_
selector
=
queue_selector
end
opt
.
on
(
'-n'
,
'--negate'
,
'Run workers for all queues in sidekiq_queues.yml except the given ones'
)
do
...
...
ee/spec/bin/sidekiq_cluster_spec.rb
View file @
ccd26150
...
...
@@ -7,7 +7,7 @@ describe 'ee/bin/sidekiq-cluster' do
where
(
:args
,
:included
,
:excluded
)
do
%w[--negate cronjob]
|
'-qdefault,1'
|
'-qcronjob,1'
%w[--queue-
query-syntax
resource_boundary=cpu]
|
'-qupdate_merge_requests,1'
|
'-qdefault,1'
%w[--queue-
selector
resource_boundary=cpu]
|
'-qupdate_merge_requests,1'
|
'-qdefault,1'
end
with_them
do
...
...
ee/spec/lib/gitlab/sidekiq_cluster/cli_spec.rb
View file @
ccd26150
...
...
@@ -79,7 +79,7 @@ describe Gitlab::SidekiqCluster::CLI do
end
end
context
'with --queue-
query-syntax
'
do
context
'with --queue-
selector
'
do
where
do
{
'memory-bound queues'
=>
{
...
...
@@ -125,7 +125,7 @@ describe Gitlab::SidekiqCluster::CLI do
[]
end
cli
.
run
(
%W(--queue-
query-syntax
#{
query
}
)
)
cli
.
run
(
%W(--queue-
selector
#{
query
}
)
)
end
it
'works when negated'
do
...
...
@@ -137,7 +137,7 @@ describe Gitlab::SidekiqCluster::CLI do
[]
end
cli
.
run
(
%W(--negate --queue-
query-syntax
#{
query
}
)
)
cli
.
run
(
%W(--negate --queue-
selector
#{
query
}
)
)
end
end
...
...
@@ -147,13 +147,13 @@ describe Gitlab::SidekiqCluster::CLI do
.
with
([[
'chat_notification'
],
[
'project_export'
]],
default_options
)
.
and_return
([])
cli
.
run
(
%w(--queue-
query-syntax
feature_category=chatops,latency_sensitive=true resource_boundary=memory,feature_category=source_code_management)
)
cli
.
run
(
%w(--queue-
selector
feature_category=chatops,latency_sensitive=true resource_boundary=memory,feature_category=source_code_management)
)
end
it
'errors on an invalid query multiple queue groups correctly'
do
expect
(
Gitlab
::
SidekiqCluster
).
not_to
receive
(
:start
)
expect
{
cli
.
run
(
%w(--queue-
query-syntax
unknown_field=chatops)
)
}
expect
{
cli
.
run
(
%w(--queue-
selector
unknown_field=chatops)
)
}
.
to
raise_error
(
Gitlab
::
SidekiqConfig
::
CliMethods
::
QueryError
)
end
end
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment