Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
1
Merge Requests
1
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
nexedi
gitlab-ce
Commits
304b43a6
Commit
304b43a6
authored
Jun 10, 2019
by
Oswaldo Ferreira
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Avoid DB timeouts when scheduling migrations
parent
65be6f1a
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
15 additions
and
12 deletions
+15
-12
changelogs/unreleased/osw-fix-post-dep-migration-with-timeout.yml
...gs/unreleased/osw-fix-post-dep-migration-with-timeout.yml
+5
-0
db/post_migrate/20190528180441_enqueue_reset_merge_status.rb
db/post_migrate/20190528180441_enqueue_reset_merge_status.rb
+5
-10
spec/migrations/enqueue_reset_merge_status_spec.rb
spec/migrations/enqueue_reset_merge_status_spec.rb
+5
-2
No files found.
changelogs/unreleased/osw-fix-post-dep-migration-with-timeout.yml
0 → 100644
View file @
304b43a6
---
title
:
Avoid DB timeouts when scheduling migrations
merge_request
:
29437
author
:
type
:
fixed
db/post_migrate/20190528180441_enqueue_reset_merge_status.rb
View file @
304b43a6
...
...
@@ -16,15 +16,10 @@ class EnqueueResetMergeStatus < ActiveRecord::Migration[5.1]
def
up
say
'Scheduling `ResetMergeStatus` jobs'
# We currently have around 135_000 opened, mergeable MRs in GitLab.com. This iteration
# will schedule around 13 batches of 10_000 MRs, which should take around 1 hour to
# complete.
relation
=
MergeRequest
.
where
(
state:
'opened'
,
merge_status:
'can_be_merged'
)
relation
.
each_batch
(
of:
BATCH_SIZE
)
do
|
batch
,
index
|
range
=
batch
.
pluck
(
'MIN(id)'
,
'MAX(id)'
).
first
BackgroundMigrationWorker
.
perform_in
(
index
*
DELAY_INTERVAL
,
MIGRATION
,
range
)
end
# We currently have more than ~5_000_000 merge request records on GitLab.com.
# This means it'll schedule ~500 jobs (10k MRs each) with a 5 minutes gap,
# so this should take ~41 hours for all background migrations to complete.
# ((5_000_000 / 10_000) * 5) / 60 => 41.6666..
queue_background_migration_jobs_by_range_at_intervals
(
MergeRequest
,
MIGRATION
,
DELAY_INTERVAL
,
batch_size:
BATCH_SIZE
)
end
end
spec/migrations/enqueue_reset_merge_status_spec.rb
View file @
304b43a6
...
...
@@ -40,9 +40,12 @@ describe EnqueueResetMergeStatus, :migration, :sidekiq do
.
to
be_scheduled_delayed_migration
(
5
.
minutes
,
1
,
2
)
expect
(
described_class
::
MIGRATION
)
.
to
be_scheduled_delayed_migration
(
10
.
minutes
,
3
,
3
)
.
to
be_scheduled_delayed_migration
(
10
.
minutes
,
3
,
4
)
expect
(
BackgroundMigrationWorker
.
jobs
.
size
).
to
eq
(
2
)
expect
(
described_class
::
MIGRATION
)
.
to
be_scheduled_delayed_migration
(
15
.
minutes
,
5
,
5
)
expect
(
BackgroundMigrationWorker
.
jobs
.
size
).
to
eq
(
3
)
end
end
end
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment