Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
Léo-Paul Géneau
gitlab-ce
Commits
04908026
Commit
04908026
authored
May 07, 2018
by
Toon Claes
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Use push events again to determine if repo needs checking
parent
ca29aa11
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
38 additions
and
30 deletions
+38
-30
app/workers/repository_check/batch_worker.rb
app/workers/repository_check/batch_worker.rb
+2
-10
app/workers/repository_check/single_repository_worker.rb
app/workers/repository_check/single_repository_worker.rb
+14
-8
spec/workers/repository_check/single_repository_worker_spec.rb
...workers/repository_check/single_repository_worker_spec.rb
+22
-12
No files found.
app/workers/repository_check/batch_worker.rb
View file @
04908026
...
@@ -32,16 +32,8 @@ module RepositoryCheck
...
@@ -32,16 +32,8 @@ module RepositoryCheck
# array of ID's. This is OK because we do it only once an hour, because
# array of ID's. This is OK because we do it only once an hour, because
# getting ID's from Postgres is not terribly slow, and because no user
# getting ID's from Postgres is not terribly slow, and because no user
# has to sit and wait for this query to finish.
# has to sit and wait for this query to finish.
def
project_ids
(
batch_size
=
BATCH_SIZE
)
def
project_ids
project_ids
=
never_checked_project_ids
(
batch_size
)
never_checked_project_ids
(
BATCH_SIZE
)
+
old_checked_project_ids
(
BATCH_SIZE
)
remaining_capacity
=
batch_size
-
project_ids
.
count
if
remaining_capacity
>
0
project_ids
+
old_checked_project_ids
(
remaining_capacity
)
else
project_ids
end
end
end
def
never_checked_project_ids
(
batch_size
)
def
never_checked_project_ids
(
batch_size
)
...
...
app/workers/repository_check/single_repository_worker.rb
View file @
04908026
...
@@ -24,18 +24,13 @@ module RepositoryCheck
...
@@ -24,18 +24,13 @@ module RepositoryCheck
end
end
def
repo_healthy?
(
project
)
def
repo_healthy?
(
project
)
return
true
if
project
.
empty_repo?
return
true
unless
has_changes?
(
project
)
git_fsck
(
project
.
repository
)
git_fsck
(
project
.
repository
)
end
end
def
wiki_repo_healthy?
(
project
)
def
wiki_repo_healthy?
(
project
)
return
true
unless
project
.
wiki_enabled?
return
true
unless
has_wiki_changes?
(
project
)
# Historically some projects never had their wiki repos initialized;
# this happens on project creation now. Let's initialize an empty repo
# if it is not already there.
project
.
create_wiki
git_fsck
(
project
.
wiki
.
repository
)
git_fsck
(
project
.
wiki
.
repository
)
end
end
...
@@ -51,8 +46,19 @@ module RepositoryCheck
...
@@ -51,8 +46,19 @@ module RepositoryCheck
false
false
end
end
def
has_
push
es?
(
project
)
def
has_
chang
es?
(
project
)
Project
.
with_push
.
exists?
(
project
.
id
)
Project
.
with_push
.
exists?
(
project
.
id
)
end
end
def
has_wiki_changes?
(
project
)
return
false
unless
project
.
wiki_enabled?
# Historically some projects never had their wiki repos initialized;
# this happens on project creation now. Let's initialize an empty repo
# if it is not already there.
return
false
unless
project
.
create_wiki
has_changes?
(
project
)
end
end
end
end
end
spec/workers/repository_check/single_repository_worker_spec.rb
View file @
04908026
...
@@ -4,8 +4,10 @@ require 'fileutils'
...
@@ -4,8 +4,10 @@ require 'fileutils'
describe
RepositoryCheck
::
SingleRepositoryWorker
do
describe
RepositoryCheck
::
SingleRepositoryWorker
do
subject
(
:worker
)
{
described_class
.
new
}
subject
(
:worker
)
{
described_class
.
new
}
it
'skips when the project repo is empty'
do
it
'skips when the project has no push events'
do
project
=
create
(
:project
,
:wiki_disabled
)
project
=
create
(
:project
,
:repository
,
:wiki_disabled
)
project
.
events
.
destroy_all
break_project
(
project
)
expect
(
worker
).
not_to
receive
(
:git_fsck
)
expect
(
worker
).
not_to
receive
(
:git_fsck
)
...
@@ -14,8 +16,19 @@ describe RepositoryCheck::SingleRepositoryWorker do
...
@@ -14,8 +16,19 @@ describe RepositoryCheck::SingleRepositoryWorker do
expect
(
project
.
reload
.
last_repository_check_failed
).
to
eq
(
false
)
expect
(
project
.
reload
.
last_repository_check_failed
).
to
eq
(
false
)
end
end
it
'fails when the project has push events and a broken repository'
do
project
=
create
(
:project
,
:repository
)
create_push_event
(
project
)
break_project
(
project
)
worker
.
perform
(
project
.
id
)
expect
(
project
.
reload
.
last_repository_check_failed
).
to
eq
(
true
)
end
it
'succeeds when the project repo is valid'
do
it
'succeeds when the project repo is valid'
do
project
=
create
(
:project
,
:repository
,
:wiki_disabled
)
project
=
create
(
:project
,
:repository
,
:wiki_disabled
)
create_push_event
(
project
)
expect
(
worker
).
to
receive
(
:git_fsck
).
and_call_original
expect
(
worker
).
to
receive
(
:git_fsck
).
and_call_original
...
@@ -26,18 +39,10 @@ describe RepositoryCheck::SingleRepositoryWorker do
...
@@ -26,18 +39,10 @@ describe RepositoryCheck::SingleRepositoryWorker do
expect
(
project
.
reload
.
last_repository_check_failed
).
to
eq
(
false
)
expect
(
project
.
reload
.
last_repository_check_failed
).
to
eq
(
false
)
end
end
it
'fails when the project is not empty and a broken repository'
do
project
=
create
(
:project
,
:repository
)
break_project
(
project
)
worker
.
perform
(
project
.
id
)
expect
(
project
.
reload
.
last_repository_check_failed
).
to
eq
(
true
)
end
it
'fails if the wiki repository is broken'
do
it
'fails if the wiki repository is broken'
do
project
=
create
(
:project
,
:wiki_enabled
)
project
=
create
(
:project
,
:
repository
,
:
wiki_enabled
)
project
.
create_wiki
project
.
create_wiki
create_push_event
(
project
)
# Test sanity: everything should be fine before the wiki repo is broken
# Test sanity: everything should be fine before the wiki repo is broken
worker
.
perform
(
project
.
id
)
worker
.
perform
(
project
.
id
)
...
@@ -78,6 +83,10 @@ describe RepositoryCheck::SingleRepositoryWorker do
...
@@ -78,6 +83,10 @@ describe RepositoryCheck::SingleRepositoryWorker do
expect
(
Gitlab
::
Shell
.
new
.
exists?
(
project
.
repository_storage
,
project
.
wiki
.
path
)).
to
eq
(
false
)
expect
(
Gitlab
::
Shell
.
new
.
exists?
(
project
.
repository_storage
,
project
.
wiki
.
path
)).
to
eq
(
false
)
end
end
def
create_push_event
(
project
)
project
.
events
.
create
(
action:
Event
::
PUSHED
,
author_id:
create
(
:user
).
id
)
end
def
break_wiki
(
project
)
def
break_wiki
(
project
)
break_repo
(
wiki_path
(
project
))
break_repo
(
wiki_path
(
project
))
end
end
...
@@ -90,6 +99,7 @@ describe RepositoryCheck::SingleRepositoryWorker do
...
@@ -90,6 +99,7 @@ describe RepositoryCheck::SingleRepositoryWorker do
break_repo
(
project
.
repository
.
path_to_repo
)
break_repo
(
project
.
repository
.
path_to_repo
)
end
end
def
break_repo
(
repo
)
def
break_repo
(
repo
)
# Create or replace blob ffffffffffffffffffffffffffffffffffffffff with an empty file
# Create or replace blob ffffffffffffffffffffffffffffffffffffffff with an empty file
# This will make the repo invalid, _and_ 'git init' cannot fix it.
# This will make the repo invalid, _and_ 'git init' cannot fix it.
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment