Commit 90330542 authored by Oswaldo Ferreira's avatar Oswaldo Ferreira Committed by Marcia Ramos

Add docs for idempotent workers

This mostly explains how to set a worker as
idempotent and add tests for it.
parent 1d9484d1
......@@ -64,6 +64,63 @@ the extra jobs will take resources away from jobs from workers that were already
there, if the resources available to the Sidekiq process handling the namespace
are not adjusted appropriately.
## Idempotent Jobs
It's known that a job can fail for multiple reasons, for example, network outages or bugs.
In order to address this, Sidekiq has a built-in retry mechanism that is
used by default by most workers within GitLab.
It's expected that a job can run again after a failure without major side-effects for the
application or users, which is why Sidekiq encourages
jobs to be [idempotent and transactional](https://github.com/mperham/sidekiq/wiki/Best-Practices#2-make-your-job-idempotent-and-transactional).
As a general rule, a worker can be considered idempotent if:
- It can safely run multiple times with the same arguments.
- Application side-effects are expected to happen only once
(or side-effects of a second run are not impactful).
A good example of that would be a cache expiration worker.
### Ensuring a worker is idempotent
Make sure the worker tests pass using the following shared example:
```ruby
include_examples 'an idempotent worker' do
it 'marks the MR as merged' do
# Using subject inside this block will process the job multiple times
subject
expect(merge_request.state).to eq('merged')
end
end
```
Use the `perform_multiple` method directly instead of `job.perform` (this
helper method is automatically included for workers).
### Declaring a worker as idempotent
```ruby
class IdempotentWorker
include ApplicationWorker
# Declares a worker is idempotent and can
# safely run multiple times.
idempotent!
# ...
end
```
It's encouraged to only have the `idempotent!` call in the top-most worker class, even if
the `perform` method is defined in another class or module.
NOTE: **Note:**
Note that a cop will fail if the worker class is not marked as idempotent.
Consider skipping the cop if you're not confident your job can safely run multiple times.
## Latency Sensitive Jobs
If a large number of background jobs get scheduled at once, queueing of jobs may
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment