Commit 495bd2fa authored by Kirill Smelkov's avatar Kirill Smelkov

gitlab-backup: Unpack *.tar.gz before storing them in git

Starting from 8.2 GitLab backups uploads and other directories not just
as set of files, but as one tarball:

    https://gitlab.com/gitlab-org/gitlab-ce/commit/d3734fbd

and this does not play well with git - now objects are stored as a
one big whole, compressed, so git cannot find good deltas.

So to help git properly deltify and find duplicates, let's unpack/repack
the archives, the same way we already do for database.sql.gz
parent 7c0e3ff2
...@@ -55,6 +55,11 @@ backup_pull() { ...@@ -55,6 +55,11 @@ backup_pull() {
mkdir "$tmpd/gitlab_backup" mkdir "$tmpd/gitlab_backup"
tar -C "$tmpd/gitlab_backup" -xf "$backup_tar" tar -C "$tmpd/gitlab_backup" -xf "$backup_tar"
gzip -d "$tmpd/gitlab_backup/db/database.sql.gz" # unzip so it is better stored in git gzip -d "$tmpd/gitlab_backup/db/database.sql.gz" # unzip so it is better stored in git
# unpack tarballs so files are better stored in git
find "$tmpd/gitlab_backup" -maxdepth 1 -type f -name "*.tar.gz" -exec \
sh -c 'mv {} {}.x && mkdir {} && tar xf {}.x -C {} && rm {}.x' ';'
find "$tmpd/gitlab_backup" -maxdepth 1 -type d -name "*.tar.gz" -empty -exec \
touch {}/.gitlab-backup-keep ';'
# 3. pull gitlab data into git-backup # 3. pull gitlab data into git-backup
...@@ -82,6 +87,11 @@ backup_restore() { ...@@ -82,6 +87,11 @@ backup_restore() {
$GIT_BACKUP restore $HEAD gitlab/misc:"$tmpd/gitlab_backup" $GIT_BACKUP restore $HEAD gitlab/misc:"$tmpd/gitlab_backup"
gzip "$tmpd/gitlab_backup/db/database.sql" # gzip sql dump, as gitlab expects .gz gzip "$tmpd/gitlab_backup/db/database.sql" # gzip sql dump, as gitlab expects .gz
# recreate tarballs from *.tar.gz directories
find "$tmpd/gitlab_backup" -maxdepth 1 -type d -name "*.tar.gz" -exec \
rm -f {}/.gitlab-backup-keep ';'
find "$tmpd/gitlab_backup" -maxdepth 1 -type d -name "*.tar.gz" -exec \
sh -c 'mv {} {}.x && tar cfz {} -C {}.x . && rm -rf {}.x' ';'
# 2. find out backup timestamp as saved by gitlab # 2. find out backup timestamp as saved by gitlab
backup_created_at=`grep :backup_created_at: "$tmpd/gitlab_backup/backup_information.yml" | backup_created_at=`grep :backup_created_at: "$tmpd/gitlab_backup/backup_information.yml" |
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment