Commit ab3dd9a1 authored by Rémy Coutable's avatar Rémy Coutable

Merge branch 'fix/importing-io-timing-issue' into 'master'

Fix timing problems running imports on production

Fixes https://gitlab.com/gitlab-com/infrastructure/issues/151

I've found out that in staging, the imported file is not copied fully by the time sidekiq runs the job, this should hopefully fix it. (Tested against staging).

See merge request !5523
parents be7ab1f2 7e77b1fd
......@@ -3,6 +3,8 @@ module Gitlab
class FileImporter
include Gitlab::ImportExport::CommandLineUtil
MAX_RETRIES = 8
def self.import(*args)
new(*args).import
end
......@@ -14,7 +16,10 @@ module Gitlab
def import
FileUtils.mkdir_p(@shared.export_path)
wait_for_archived_file do
decompress_archive
end
rescue => e
@shared.error(e)
false
......@@ -22,6 +27,17 @@ module Gitlab
private
# Exponentially sleep until I/O finishes copying the file
def wait_for_archived_file
MAX_RETRIES.times do |retry_number|
break if File.exist?(@archive_file)
sleep(2**retry_number)
end
yield
end
def decompress_archive
result = untar_zxf(archive: @archive_file, dir: @shared.export_path)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment