Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
G
gitlab-ce
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
1
Merge Requests
1
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
nexedi
gitlab-ce
Commits
24ba0989
Commit
24ba0989
authored
Jun 25, 2018
by
Shinya Maeda
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Added spec for build trace chunk
parent
49467379
Changes
4
Expand all
Hide whitespace changes
Inline
Side-by-side
Showing
4 changed files
with
321 additions
and
180 deletions
+321
-180
app/models/ci/build_trace_chunk.rb
app/models/ci/build_trace_chunk.rb
+11
-10
app/services/concerns/exclusive_lease_lock.rb
app/services/concerns/exclusive_lease_lock.rb
+3
-1
spec/models/ci/build_trace_chunk_spec.rb
spec/models/ci/build_trace_chunk_spec.rb
+302
-169
spec/support/helpers/stub_object_storage.rb
spec/support/helpers/stub_object_storage.rb
+5
-0
No files found.
app/models/ci/build_trace_chunk.rb
View file @
24ba0989
...
...
@@ -8,8 +8,6 @@ module Ci
default_value_for
:data_store
,
:redis
WriteError
=
Class
.
new
(
StandardError
)
CHUNK_SIZE
=
128
.
kilobytes
WRITE_LOCK_RETRY
=
10
WRITE_LOCK_SLEEP
=
0.01
.
seconds
...
...
@@ -65,6 +63,7 @@ module Ci
end
def
truncate
(
offset
=
0
)
raise
ArgumentError
,
'Fog store does not support truncating'
if
fog?
# If data is null, get_data returns Excon::Error::NotFound
raise
ArgumentError
,
'Offset is out of range'
if
offset
>
size
||
offset
<
0
return
if
offset
==
size
# Skip the following process as it doesn't affect anything
...
...
@@ -72,6 +71,8 @@ module Ci
end
def
append
(
new_data
,
offset
)
raise
ArgumentError
,
'Fog store does not support appending'
if
fog?
# If data is null, get_data returns Excon::Error::NotFound
raise
ArgumentError
,
'New data is nil'
unless
new_data
raise
ArgumentError
,
'Offset is out of range'
if
offset
>
size
||
offset
<
0
raise
ArgumentError
,
'Chunk size overflow'
if
CHUNK_SIZE
<
(
offset
+
new_data
.
bytesize
)
...
...
@@ -98,21 +99,17 @@ module Ci
(
start_offset
...
end_offset
)
end
def
data_persisted?
!
redis?
end
def
persist_data!
in_lock
(
*
lock_params
)
do
# Write opetation is atomic
unsafe_
migrate
_to!
(
self
.
class
.
persist_store
)
unsafe_
persist
_to!
(
self
.
class
.
persist_store
)
end
end
private
def
unsafe_
migrate
_to!
(
new_store
)
def
unsafe_
persist
_to!
(
new_store
)
return
if
data_store
==
new_store
.
to_s
r
eturn
unless
size
>
0
r
aise
ArgumentError
,
'Can not persist empty data'
unless
size
>
0
old_store_class
=
self
.
class
.
get_store_class
(
data_store
)
...
...
@@ -130,7 +127,7 @@ module Ci
end
def
unsafe_set_data!
(
value
)
raise
ArgumentError
,
'
too much data
'
if
value
.
bytesize
>
CHUNK_SIZE
raise
ArgumentError
,
'
New data size exceeds chunk size
'
if
value
.
bytesize
>
CHUNK_SIZE
self
.
class
.
get_store_class
(
data_store
).
set_data
(
self
,
value
)
@data
=
value
...
...
@@ -144,6 +141,10 @@ module Ci
Ci
::
BuildTraceChunkFlushWorker
.
perform_async
(
id
)
end
def
data_persisted?
!
redis?
end
def
full?
size
==
CHUNK_SIZE
end
...
...
app/services/concerns/exclusive_lease_lock.rb
View file @
24ba0989
module
ExclusiveLeaseLock
extend
ActiveSupport
::
Concern
FailedToObtainLockError
=
Class
.
new
(
StandardError
)
def
in_lock
(
key
,
ttl:
1
.
minute
,
retry_max:
10
,
sleep_sec:
0.01
.
seconds
)
lease
=
Gitlab
::
ExclusiveLease
.
new
(
key
,
timeout:
ttl
)
retry_count
=
0
...
...
@@ -12,7 +14,7 @@ module ExclusiveLeaseLock
break
if
retry_max
<
(
retry_count
+=
1
)
end
raise
WriteError
,
'Failed to obtain write
lock'
unless
uuid
raise
FailedToObtainLockError
,
'Failed to obtain a
lock'
unless
uuid
return
yield
ensure
...
...
spec/models/ci/build_trace_chunk_spec.rb
View file @
24ba0989
This diff is collapsed.
Click to expand it.
spec/support/helpers/stub_object_storage.rb
View file @
24ba0989
...
...
@@ -20,6 +20,11 @@ module StubObjectStorage
::
Fog
::
Storage
.
new
(
uploader
.
object_store_credentials
).
tap
do
|
connection
|
begin
connection
.
directories
.
create
(
key:
remote_directory
)
# Cleanup remaining files
connection
.
directories
.
each
do
|
directory
|
directory
.
files
.
map
(
&
:destroy
)
end
rescue
Excon
::
Error
::
Conflict
end
end
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment