Commit 0cd9e8d6 authored by Pawel Chojnacki's avatar Pawel Chojnacki

Remove support for git annex

parent 68edac04
......@@ -10,11 +10,6 @@ class Profiles::KeysController < Profiles::ApplicationController
@key = current_user.keys.find(params[:id])
end
# Back-compat: We need to support this URL since git-annex webapp points to it
def new
redirect_to profile_keys_path
end
def create
@key = current_user.keys.new(key_params)
......
......@@ -7,4 +7,4 @@
%li
The update action will time out after 10 minutes. For big repositories, use a clone/push combination.
%li
The Git LFS/Annex objects will <strong>not</strong> be synced.
The Git LFS objects will <strong>not</strong> be synced.
......@@ -15,7 +15,7 @@
%li
To migrate an SVN repository, check out #{link_to "this document", help_page_path('workflow/importing/migrating_from_svn')}.
%li
The Git LFS/Annex objects will <strong>not</strong> be imported.
The Git LFS objects will <strong>not</strong> be imported.
.form-group
.col-sm-offset-2.col-sm-10
......@@ -26,6 +26,6 @@
Mirror repository
.help-block
Automatically update this project's branches and tags from the upstream
repository every hour. The Git LFS/Annex objects will not be synced.
repository every hour. The Git LFS objects will not be synced.
= f.hidden_field :mirror_user_id, value: current_user.id
---
title: Remove support for Git Annex
merge_request:
author:
......@@ -577,11 +577,6 @@ production: &base
# If you use non-standard ssh port you need to specify it
# ssh_port: 22
# git-annex support (EE only)
# If this setting is set to true, the same setting in config.yml of
# gitlab-shell needs to be set to true
# git_annex_enabled: false
## Git settings
# CAUTION!
# Use the default values unless you really know what you are doing
......
......@@ -429,7 +429,6 @@ Settings.gitlab_shell['ssh_port'] ||= 22
Settings.gitlab_shell['ssh_user'] ||= Settings.gitlab.user
Settings.gitlab_shell['owner_group'] ||= Settings.gitlab.user
Settings.gitlab_shell['ssh_path_prefix'] ||= Settings.send(:build_gitlab_shell_ssh_path_prefix)
Settings.gitlab_shell['git_annex_enabled'] ||= false
#
# Repositories
......
......@@ -21,7 +21,7 @@ resource :profile, only: [:show, :update] do
end
end
resource :preferences, only: [:show, :update]
resources :keys, only: [:index, :show, :new, :create, :destroy]
resources :keys, only: [:index, :show, :create, :destroy]
resources :emails, only: [:index, :create, :destroy]
resources :chat_names, only: [:index, :new, :create, :destroy] do
collection do
......
......@@ -63,7 +63,6 @@
- [Reply by email](administration/reply_by_email.md) Allow users to comment on issues and merge requests by replying to notification emails.
- [Migrate GitLab CI to CE/EE](migrate_ci_to_ce/README.md) Follow this guide to migrate your existing GitLab CI data to GitLab CE/EE.
- [Downgrade back to CE](downgrade_ee_to_ce/README.md) Follow this guide if you need to downgrade from EE to CE.
- [git-annex configuration](workflow/git_annex.md#configuration)
- [Git LFS configuration](workflow/lfs/lfs_administration.md)
- [Housekeeping](administration/housekeeping.md) Keep your Git repository tidy and fast.
- [GitLab Pages configuration](administration/pages/index.md) Configure GitLab Pages.
......
......@@ -308,8 +308,8 @@ Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-mach
++ CI_RUNNER_ID=1337
++ export CI_RUNNER_DESCRIPTION=shared-runners-manager-1.example.com
++ CI_RUNNER_DESCRIPTION=shared-runners-manager-1.example.com
++ export 'CI_RUNNER_TAGS=shared, docker, linux, ruby, mysql, postgres, mongo, git-annex'
++ CI_RUNNER_TAGS='shared, docker, linux, ruby, mysql, postgres, mongo, git-annex'
++ export 'CI_RUNNER_TAGS=shared, docker, linux, ruby, mysql, postgres, mongo'
++ CI_RUNNER_TAGS='shared, docker, linux, ruby, mysql, postgres, mongo'
++ export CI_REGISTRY=registry.example.com
++ CI_REGISTRY=registry.example.com
++ export CI_DEBUG_TRACE=true
......
......@@ -15,13 +15,6 @@ Kerberos and Atlassian Crowd are only available on the Enterprise Edition, so
you should disable these mechanisms before downgrading and you should provide
alternative authentication methods to your users.
### Git Annex
Git Annex is also only available on the Enterprise Edition. This means that if
you have repositories that use Git Annex to store large files, these files will
no longer be easily available via Git. You should consider migrating these
repositories to use Git LFS before downgrading to the Community Edition.
### Remove Jenkins CI Service entries from the database
The `JenkinsService` class is only available on the Enterprise Edition codebase,
......
......@@ -79,7 +79,6 @@ See more details in the [Configure GitLab](configuration.md) step.
- You cannot push code to secondary nodes
- Git LFS is not supported yet
- Git Annex is not supported yet
- Primary node has to be online for OAuth login to happen (existing sessions and git are not affected)
## Frequently Asked Questions
......@@ -100,7 +99,7 @@ We currently replicate project repositories and the whole database. This
means user accounts, issues, merge requests, groups, project data, etc.,
will be available for query.
We currently don't replicate user generated attachments / avatars or any
other file in `public/upload`. We also don't replicate LFS / Annex or
other file in `public/upload`. We also don't replicate LFS or
artifacts data (`shared/folder`).
### Can I git push to a secondary node?
......
......@@ -130,7 +130,6 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project
1. [GitLab Flow Overview](https://about.gitlab.com/2014/09/29/gitlab-flow/)
1. [Always Start with an Issue](https://about.gitlab.com/2016/03/03/start-with-an-issue/)
1. [GitLab Flow Documentation](https://docs.gitlab.com/ee/workflow/gitlab_flow.html)
1. [Git-Annex to Git-LFS Migration Guide](https://docs.gitlab.com/ee/workflow/lfs/migrate_from_git-annex_to_git-lfs.html)
#### 2.5. GitLab Comparisons
......@@ -166,7 +165,7 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project
#### 3.4. Large Files
1. [Big files in Git (Git LFS, Annex) - Video](https://www.youtube.com/watch?v=DawznUxYDe4)
1. [Big files in Git (Git LFS) - Video](https://www.youtube.com/watch?v=DawznUxYDe4)
#### 3.5. LDAP and Active Directory
......
......@@ -167,7 +167,6 @@ Some tickets need specific knowledge or a deep understanding of a particular com
Move on to understanding some of GitLab's more advanced features. You can make use of GitLab.com to understand the features from an end-user perspective and then use your own instance to understand setup and configuration of the feature from an Administrative perspective
- Set up and try [Git Annex](https://docs.gitlab.com/ee/workflow/git_annex.html)
- Set up and try [Git LFS](https://docs.gitlab.com/ee/workflow/lfs/manage_large_binaries_with_git_lfs.html)
- Get to know the [GitLab API](https://docs.gitlab.com/ee/api/README.html), its capabilities and shortcomings
- Learn how to [migrate from SVN to Git](https://docs.gitlab.com/ee/workflow/importing/migrating_from_svn.html)
......
......@@ -17,8 +17,6 @@
- (EE) [File lock](../user/project/file_lock.md)
- [Labels](../user/project/labels.md)
- (EE) [Issue weight](issue_weight.md)
- (EE) [Manage large binaries with git annex](git_annex.md)
- [Git-Annex to Git-LFS Migration Guide](lfs/migrate_from_git_annex_to_git_lfs.html)
- [Notification emails](notifications.md)
- [Project Features](project_features.md)
- [Project forking workflow](forking_workflow.md)
......
# Git annex
> **Warning:** GitLab is deprecating git-annex support
from GitLab 8.17 (2017/02/22) on. It will be [completely
removed][deprecate-annex-issue] in GitLab 9.0 (2017/03/22).
Read through the [migration guide from git-annex to git-lfs][guide].
The biggest limitation of Git, compared to some older centralized version
control systems, has been the maximum size of the repositories.
The general recommendation is to not have Git repositories larger than 1GB to
preserve performance. Although GitLab has no limit (some repositories in GitLab
are over 50GB!), we subscribe to the advice to keep repositories as small as
you can.
Not being able to version control large binaries is a big problem for many
larger organizations.
Videos, photos, audio, compiled binaries and many other types of files are too
large. As a workaround, people keep artwork-in-progress in a Dropbox folder and
only check in the final result. This results in using outdated files, not
having a complete history and increases the risk of losing work.
This problem is solved in GitLab Enterprise Edition by integrating the
[git-annex] application.
`git-annex` allows managing large binaries with Git without checking the
contents into Git.
You check-in only a symlink that contains the SHA-1 of the large binary. If you
need the large binary, you can sync it from the GitLab server over `rsync`, a
very fast file copying tool.
## GitLab git-annex Configuration
`git-annex` is disabled by default in GitLab. Below you will find the
configuration options required to enable it.
### Requirements
`git-annex` needs to be installed both on the server and the client side.
For Debian-like systems (e.g., Debian, Ubuntu) this can be achieved by running:
```
sudo apt-get update && sudo apt-get install git-annex
```
For RedHat-like systems (e.g., CentOS, RHEL) this can be achieved by running:
```
sudo yum install epel-release && sudo yum install git-annex
```
### Configuration for Omnibus packages
For omnibus-gitlab packages, only one configuration setting is needed.
The Omnibus package will internally set the correct options in all locations.
1. In `/etc/gitlab/gitlab.rb` add the following line:
```ruby
gitlab_shell['git_annex_enabled'] = true
```
1. Save the file and [reconfigure GitLab][] for the changes to take effect.
### Configuration for installations from source
There are 2 settings to enable git-annex on your GitLab server.
One is located in `config/gitlab.yml` of the GitLab repository and the other
one is located in `config.yml` of gitlab-shell.
1. In `config/gitlab.yml` add or edit the following lines:
```yaml
gitlab_shell:
git_annex_enabled: true
```
1. In `config.yml` of gitlab-shell add or edit the following lines:
```yaml
git_annex_enabled: true
```
1. Save the files and [restart GitLab][] for the changes to take effect.
## Using GitLab git-annex
>
**Note:**
Your Git remotes must be using the SSH protocol, not HTTP(S).
Here is an example workflow of uploading a very large file and then checking it
into your Git repository:
```bash
git clone git@example.com:group/project.git
git annex init 'My Laptop' # initialize the annex project and give an optional description
cp ~/tmp/debian.iso ./ # copy a large file into the current directory
git annex add debian.iso # add the large file to git annex
git commit -am "Add Debian iso" # commit the file metadata
git annex sync --content # sync the Git repo and large file to the GitLab server
```
The output should look like this:
```
commit
On branch master
Your branch is ahead of 'origin/master' by 1 commit.
(use "git push" to publish your local commits)
nothing to commit, working tree clean
ok
pull origin
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 5 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (5/5), done.
From example.com:group/project
497842b..5162f80 git-annex -> origin/git-annex
ok
(merging origin/git-annex into git-annex...)
(recording state in git...)
copy debian.iso (checking origin...) (to origin...)
SHA256E-s26214400--8092b3d482fb1b7a5cf28c43bc1425c8f2d380e86869c0686c49aa7b0f086ab2.iso
26,214,400 100% 638.88kB/s 0:00:40 (xfr#1, to-chk=0/1)
ok
pull origin
ok
(recording state in git...)
push origin
Counting objects: 15, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (13/13), done.
Writing objects: 100% (15/15), 1.64 KiB | 0 bytes/s, done.
Total 15 (delta 1), reused 0 (delta 0)
To example.com:group/project.git
* [new branch] git-annex -> synced/git-annex
* [new branch] master -> synced/master
ok
```
Your files can be found in the `master` branch, but you'll notice that there
are more branches created by the `annex sync` command.
Git Annex will also create a new directory at `.git/annex/` and will record the
tracked files in the `.git/config` file. The files you assign to be tracked
with `git-annex` will not affect the existing `.git/config` records. The files
are turned into symbolic links that point to data in `.git/annex/objects/`.
The `debian.iso` file in the example will contain the symbolic link:
```
.git/annex/objects/ZW/1k/SHA256E-s82701--6384039733b5035b559efd5a2e25a493ab6e09aabfd5162cc03f6f0ec238429d.png/SHA256E-s82701--6384039733b5035b559efd5a2e25a493ab6e09aabfd5162cc03f6f0ec238429d.iso
```
Use `git annex info` to retrieve the information about the local copy of your
repository.
---
Downloading a single large file is also very simple:
```bash
git clone git@gitlab.example.com:group/project.git
git annex sync # sync Git branches but not the large file
git annex get debian.iso # download the large file
```
To download all files:
```bash
git clone git@gitlab.example.com:group/project.git
git annex sync --content # sync Git branches and download all the large files
```
By using `git-annex` without GitLab, anyone that can access the server can also
access the files of all projects, but GitLab Annex ensures that you can only
access files of projects you have access to (developer, master, or owner role).
## How it works
Internally GitLab uses [GitLab Shell] to handle SSH access and this was a great
integration point for `git-annex`.
There is a setting in gitlab-shell so you can disable GitLab Annex support
if you want to.
## Troubleshooting tips
Differences in version of `git-annex` on the GitLab server and on local machines
can cause `git-annex` to raise unpredicted warnings and errors.
Consult the [Annex upgrade page][annex-upgrade] for more information about
the differences between versions. You can find out which version is installed
on your server by navigating to <https://pkgs.org/download/git-annex> and
searching for your distribution.
Although there is no general guide for `git-annex` errors, there are a few tips
on how to go around the warnings.
### git-annex-shell: Not a git-annex or gcrypt repository.
This warning can appear on the initial `git annex sync --content` and is caused
by differences in `git-annex-shell`. You can read more about it
[in this git-annex issue][issue].
One important thing to note is that despite the warning, the `sync` succeeds
and the files are pushed to the GitLab repository.
If you get hit by this, you can run the following command inside the repository
that the warning was raised:
```
git config remote.origin.annex-ignore false
```
Consecutive runs of `git annex sync --content` **should not** produce this
warning and the output should look like this:
```
commit ok
pull origin
ok
pull origin
ok
push origin
```
[annex-upgrade]: https://git-annex.branchable.com/upgrades/
[deprecate-annex-issue]: https://gitlab.com/gitlab-org/gitlab-ee/issues/1648
[git-annex]: https://git-annex.branchable.com/ "git-annex website"
[gitlab shell]: https://gitlab.com/gitlab-org/gitlab-shell "GitLab Shell repository"
[guide]: lfs/migrate_from_git_annex_to_git_lfs.html
[issue]: https://git-annex.branchable.com/forum/Error_from_git-annex-shell_on_creation_of_gcrypt_special_remote/ "git-annex issue"
[reconfigure GitLab]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure
[restart GitLab]: ../administration/restart_gitlab.md#installations-from-source
......@@ -3,12 +3,6 @@
Managing large files such as audio, video and graphics files has always been one of the shortcomings of Git.
The general recommendation is to not have Git repositories larger than 1GB to preserve performance.
GitLab already supports [managing large files with git annex](http://doc.gitlab.com/ee/workflow/git_annex.html) (EE only), however in certain
environments it is not always convenient to use different commands to differentiate between the large files and regular ones.
Git LFS makes this simpler for the end user by removing the requirement to learn new commands.
<!-- more -->
## How it works
Git LFS client talks with the GitLab server over HTTPS. It uses HTTP Basic Authentication to authorize client requests.
......
......@@ -4,13 +4,6 @@ Managing large files such as audio, video and graphics files has always been one
of the shortcomings of Git. The general recommendation is to not have Git repositories
larger than 1GB to preserve performance.
GitLab already supports [managing large files with git annex](http://docs.gitlab.com/ee/workflow/git_annex.html)
(EE only), however in certain environments it is not always convenient to use
different commands to differentiate between the large files and regular ones.
Git LFS makes this simpler for the end user by removing the requirement to
learn new commands.
## How it works
Git LFS client talks with the GitLab server over HTTPS. It uses HTTP Basic Authentication
......
# Migration guide from Git Annex to Git LFS
>**Note:**
Git Annex support [will be dropped][issue-remove-annex] with GitLab Enterprise
Edition 9.0 which is to be released on March 22nd 2017.
Both [Git Annex][] and [Git LFS][] are tools to manage large files in Git.
## History
Git Annex [was introduced in GitLab Enterprise Edition 7.8][post-3], at a time
where Git LFS didn't yet exist. A few months later, GitLab brought support for
Git LFS in [GitLab 8.2][post-2] and is available for both Community and
Enterprise editions.
## Differences between Git Annex and Git LFS
Some items below are general differences between the two protocols and some are
ones that GitLab developed.
- Git Annex works only through SSH, whereas Git LFS works both with SSH and HTTPS
(SSH support was added in GitLab 8.12).
- Annex files are stored in a sub-directory of the normal repositories, whereas
LFS files are stored outside of the repositories in a place you can define.
- Git Annex requires a more complex setup, but has much more options than Git
LFS. You can compare the commands each one offers by running `man git-annex`
and `man git-lfs`.
- Annex files cannot be browsed directly in GitLab's interface, whereas LFS
files can.
## Migration steps
>**Note:**
Since Git Annex files are stored in a sub-directory of the normal repositories
(`.git/annex/objects`) and LFS files are stored outside of the repositories,
they are not compatible as they are using a different scheme. Therefore, the
migration has to be done manually per repository.
There are basically two steps you need to take in order to migrate from Git
Annex to Git LFS.
### TL; DR
If you know what you are doing and want to skip the reading, this is what you
need to do (we assume you have [git-annex enabled][annex-gitlab-use] in your
repository). Fire up a terminal, navigate to your Git repository and:
1. Disable `git-annex`:
```bash
git annex sync --content
git annex direct
git annex uninit
git annex indirect
```
1. Enable `git-lfs`:
```
git lfs track <files>
git add .
git commit -m "commit message"
git push
```
### Disabling Git Annex in your repo
Before changing anything, make sure you have a backup of your repository first.
There are a couple of ways to do that, but you can simply clone it to another
local path and maybe push it to GitLab if you want a remote backup as well.
Here you'll find a guide on
[how to back up a **git-annex** repository to an external hard drive][bkp-ext-drive].
Since Annex files are stored as objects with symlinks and cannot be directly
modified, we need to first remove those symlinks.
>**Note:**
Make sure the you read about the [`direct` mode][annex-direct] as it contains
useful information that may fit in your use case. Note that `annex direct` is
deprecated in Git Annex version 6, so you may need to upgrade your repository
if the server also has Git Annex 6 installed. Read more in the
[Git Annex troubleshooting tips][annex-tips] section.
1. Use `annex direct`:
```bash
git annex direct
```
The output should be similar to this:
```bash
commit
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean
ok
direct debian.iso ok
direct ok
```
1. Disable Git Annex with [`annex uninit`][uninit]:
```bash
git annex uninit
```
The output should be similar to this:
```bash
unannex debian.iso ok
Deleted branch git-annex (was 2534d2c).
```
This will `unannex` every file in the repository, leaving the original files.
1. Switch back to `indirect` mode:
```bash
git annex indirect
```
The output should be similar to this:
```bash
(merging origin/git-annex into git-annex...)
(recording state in git...)
commit (recording state in git...)
ok
(recording state in git...)
[master fac3194] commit before switching to indirect mode
1 file changed, 1 deletion(-)
delete mode 120000 alpine-virt-3.4.4-x86_64.iso
ok
indirect ok
ok
```
---
At this point, you have two options. Either add, commit and push the files
directly back to GitLab or switch to Git LFS. We will tackle the LFS switch in
the next section.
### Enabling Git LFS in your repo
Git LFS is enabled by default on all GitLab products (GitLab CE, GitLab EE,
GitLab.com), therefore, you don't need to do anything server-side.
1. First, make sure you have `git-lfs` installed locally:
```bash
git lfs help
```
If the terminal doesn't prompt you with a full response on `git-lfs` commands,
[install the Git LFS client][install-lfs] first.
1. Inside the repo, run the following command to initiate LFS:
```bash
git lfs install
```
1. Enable `git-lfs` for the group of files you want to track. You
can track specific files, all files containing the same extension, or an
entire directory:
```bash
git lfs track images/01.png # per file
git lfs track **/*.png # per extension
git lfs track images/ # per directory
```
Once you do that, run `git status` and you'll see `.gitattributes` added
to your repo. It collects all file patterns that you chose to track via
`git-lfs`.
1. Add the files, commit and push them to GitLab:
```bash
git add .
git commit -m "commit message"
git push
```
If your remote is set up with HTTP, you will be asked to enter your login
credentials. If you have [2FA enabled][2fa], make sure to use a
[personal access token][token] instead of your password.
## Removing the Git Annex branches
After the migration finishes successfully, you can remove all `git-annex`
related branches from your repository.
On GitLab, navigate to your project's **Repository ➔ Branches** and delete all
branches created by Git Annex: `git-annex`, and all under `synced/`.
![repository branches](images/git-annex-branches.png)
If there are still some Annex objects inside your repository (`.git/annex/`)
or references inside `.git/config`, run `annex uninit` again:
```bash
git annex uninit
```
## Further Reading
- (Blog Post) [Getting Started with Git FLS][post-1]
- (Blog Post) [Announcing LFS Support in GitLab][post-2]
- (Blog Post) [GitLab Annex Solves the Problem of Versioning Large Binaries with Git][post-3]
- (GitLab Docs) [Git Annex][doc-1]
- (GitLab Docs) [Git LFS][doc-2]
[2fa]: ../../user/profile/account/two_factor_authentication.md
[token]: ../../user/profile/account/two_factor_authentication.html#personal-access-tokens
[annex-tips]: ../git_annex.html#troubleshooting-tips
[annex-direct]: https://git-annex.branchable.com/direct_mode/
[annex-gitlab-use]: ../git_annex.md#using-gitlab-git-annex
[annex-ee]: https://docs.gitlab.com/ee/workflow/git_annex.html
[bkp-ext-drive]: https://www.thomas-krenn.com/en/wiki/Git-annex_Repository_on_an_External_Hard_Drive
[doc-1]: https://docs.gitlab.com/ee/workflow/git_annex.html
[doc-2]: https://docs.gitlab.com/ee/workflow/lfs/manage_large_binaries_with_git_lfs.html
[Git Annex]: http://git-annex.branchable.com/
[Git LFS]: https://git-lfs.github.com/
[install-lfs]: https://git-lfs.github.com/
[issue-remove-annex]: https://gitlab.com/gitlab-org/gitlab-ee/issues/1648
[lfs-track]: https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/#tracking-files-with-lfs
[post-1]: https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/
[post-2]: https://about.gitlab.com/2015/11/23/announcing-git-lfs-support-in-gitlab/
[post-3]: https://about.gitlab.com/2015/02/17/gitlab-annex-solves-the-problem-of-versioning-large-binaries-with-git/
[uninit]: https://git-annex.branchable.com/git-annex-uninit/
......@@ -22,7 +22,7 @@ A few things/limitations to consider:
password, e.g., you want to mirror to GitHub and have 2FA enabled.
- The import will time out after 15 minutes. For repositories that take longer
use a clone/push combination.
- The Git LFS/Annex objects will not be synced. You'll need to push/pull them
- The Git LFS objects will not be synced. You'll need to push/pull them
manually.
## Pulling from a remote repository
......
......@@ -54,8 +54,7 @@ module API
end
def log_user_activity(actor)
commands = Gitlab::GitAccess::DOWNLOAD_COMMANDS +
Gitlab::GitAccess::GIT_ANNEX_COMMANDS
commands = Gitlab::GitAccess::DOWNLOAD_COMMANDS
::Users::ActivityService.new(actor, 'Git SSH').execute if commands.include?(params[:action])
end
......
......@@ -180,9 +180,8 @@ module Backup
return unless Dir.exist?(path)
dir_entries = Dir.entries(path)
%w[annex custom_hooks].each do |entry|
yield(entry) if dir_entries.include?(entry)
end
yield('custom_hooks') if dir_entries.include?('custom_hooks')
end
def prepare
......
......@@ -102,8 +102,6 @@ module Gitlab
return if Gitlab::Git.blank_ref?(@newrev) || !(commit_validation || validate_path_locks?)
commits.each do |commit|
next if commit_from_annex_sync?(commit.safe_message)
if commit_validation
error = check_commit(commit, push_rule)
return error if error
......@@ -226,13 +224,6 @@ module Gitlab
def commits
project.repository.new_commits(@newrev)
end
def commit_from_annex_sync?(commit_message)
return false unless Gitlab.config.gitlab_shell.git_annex_enabled
# Commit message starting with <git-annex in > so avoid push rules on this
commit_message.start_with?('git-annex in')
end
end
end
end
......@@ -16,8 +16,7 @@ module Gitlab
DOWNLOAD_COMMANDS = %w{ git-upload-pack git-upload-archive }.freeze
PUSH_COMMANDS = %w{ git-receive-pack }.freeze
GIT_ANNEX_COMMANDS = %w{ git-annex-shell }.freeze
ALL_COMMANDS = DOWNLOAD_COMMANDS + PUSH_COMMANDS + GIT_ANNEX_COMMANDS
ALL_COMMANDS = DOWNLOAD_COMMANDS + PUSH_COMMANDS
attr_reader :actor, :project, :protocol, :user_access, :authentication_abilities
......@@ -44,8 +43,6 @@ module Gitlab
check_download_access!
when *PUSH_COMMANDS
check_push_access!(changes)
when *GIT_ANNEX_COMMANDS
git_annex_access_check(project, changes)
end
build_status_object(true)
......@@ -131,8 +128,6 @@ module Gitlab
raise UnauthorizedError, "You can't push code on a secondary GitLab Geo node."
end
return if git_annex_branch_sync?(changes)
if deploy_key
check_deploy_key_push_access!
elsif user
......@@ -247,39 +242,5 @@ module Gitlab
def build_status_object(status, message = '')
Gitlab::GitAccessStatus.new(status, message)
end
def git_annex_access_check(project, changes)
raise UnauthorizedError, "git-annex is disabled" unless Gitlab.config.gitlab_shell.git_annex_enabled
unless user && user_access.allowed?
raise UnauthorizedError, "You don't have access"
end
if Gitlab::Geo.enabled? && Gitlab::Geo.secondary?
raise UnauthorizedError, "You can't use git-annex with a secondary GitLab Geo node."
end
unless user.can?(:push_code, project)
raise UnauthorizedError, "You don't have permission"
end
end
def git_annex_branch_sync?(changes)
return false unless Gitlab.config.gitlab_shell.git_annex_enabled
return false if changes.blank?
changes = changes.lines if changes.is_a?(String)
# Iterate over all changes to find if user allowed all of them to be applied
# 0000000000000000000000000000000000000000 3073696294ddd52e9e6b6fc3f429109cac24626f refs/heads/synced/git-annex
# 0000000000000000000000000000000000000000 65be9df0e995d36977e6d76fc5801b7145ce19c9 refs/heads/synced/master
changes.map(&:strip).reject(&:blank?).each do |change|
unless change.end_with?("refs/heads/synced/git-annex") || change.include?("refs/heads/synced/")
return false
end
end
true
end
end
end
......@@ -3,16 +3,6 @@ require 'spec_helper'
describe Profiles::KeysController do
let(:user) { create(:user) }
describe '#new' do
before { sign_in(user) }
it 'redirects to #index' do
get :new
expect(response).to redirect_to(profile_keys_path)
end
end
describe "#get_keys" do
describe "non existant user" do
it "does not generally work" do
......
......@@ -5,11 +5,7 @@ describe Gitlab::GitAccess, lib: true do
let(:project) { create(:project, :repository) }
let(:user) { create(:user) }
let(:actor) { user }
let(:git_annex_changes) do
["6f6d7e7ed97bb5f0054f2b1df789b39ca89b6ff9 570e7b2abdd848b95f2f578043fc23bd6f6fd24d refs/heads/synced/git-annex",
"6f6d7e7ed97bb5f0054f2b1df789b39ca89b6ff9 570e7b2abdd848b95f2f578043fc23bd6f6fd24d refs/heads/synced/named-branch"]
end
let(:git_annex_master_changes) { "6f6d7e7ed97bb5f0054f2b1df789b39ca89b6ff9 570e7b2abdd848b95f2f578043fc23bd6f6fd24d refs/heads/master" }
let(:authentication_abilities) do
[
:read_project,
......@@ -527,96 +523,6 @@ describe Gitlab::GitAccess, lib: true do
run_permission_checks(permissions_matrix)
end
context "when using git annex" do
before do
project.team << [user, :master]
allow_any_instance_of(Repository).to receive(:new_commits).and_return(
project.repository.commits_between('6f6d7e7ed97bb5f0054f2b1df789b39ca89b6ff9', '570e7b2abdd848b95f2f578043fc23bd6f6fd24d')
)
end
describe 'and gitlab geo is enabled in a secondary node' do
before do
allow(Gitlab.config.gitlab_shell).to receive(:git_annex_enabled).and_return(true)
allow(Gitlab::Geo).to receive(:enabled?) { true }
allow(Gitlab::Geo).to receive(:secondary?) { true }
end
it { expect { access.send(:check_push_access!, git_annex_changes) }.to raise_error(described_class::UnauthorizedError) }
end
describe 'and git hooks unset' do
describe 'git annex enabled' do
before { allow(Gitlab.config.gitlab_shell).to receive(:git_annex_enabled).and_return(true) }
it { expect { access.send(:check_push_access!, git_annex_changes) }.not_to raise_error }
end
describe 'git annex disabled' do
before { allow(Gitlab.config.gitlab_shell).to receive(:git_annex_enabled).and_return(false) }
it { expect { access.send(:check_push_access!, git_annex_changes) }.not_to raise_error }
end
end
describe 'and push rules set' do
before { project.create_push_rule }
describe 'check commit author email' do
before do
project.push_rule.update(author_email_regex: "@only.com")
end
describe 'git annex enabled' do
before { allow(Gitlab.config.gitlab_shell).to receive(:git_annex_enabled).and_return(true) }
it { expect { access.send(:check_push_access!, git_annex_changes) }.not_to raise_error }
end
describe 'git annex enabled, push to master branch' do
before do
allow(Gitlab.config.gitlab_shell).to receive(:git_annex_enabled).and_return(true)
allow_any_instance_of(Commit).to receive(:safe_message) { 'git-annex in me@host:~/repo' }
end
it { expect { access.send(:check_push_access!, git_annex_master_changes) }.not_to raise_error }
end
describe 'git annex disabled' do
before do
allow(Gitlab.config.gitlab_shell).to receive(:git_annex_enabled).and_return(false)
end
it { expect { access.send(:check_push_access!, git_annex_changes) }.to raise_error(described_class::UnauthorizedError) }
end
end
describe 'check max file size' do
before do
allow_any_instance_of(Gitlab::Git::Blob).to receive(:size).and_return(5.megabytes.to_i)
project.push_rule.update(max_file_size: 2)
end
describe 'git annex enabled' do
before { allow(Gitlab.config.gitlab_shell).to receive(:git_annex_enabled).and_return(true) }
it { expect(access.check('git-annex-shell', git_annex_changes).allowed?).to be_truthy }
it { expect { access.send(:check_push_access!, git_annex_changes) }.not_to raise_error }
end
describe 'git annex disabled' do
before do
allow(Gitlab.config.gitlab_shell).to receive(:git_annex_enabled).and_return(false)
end
it { expect(access.check('git-annex-shell', git_annex_changes).allowed?).to be_falsey }
it { expect { access.send(:check_push_access!, git_annex_changes) }.to raise_error(described_class::UnauthorizedError) }
end
end
end
end
describe "push_rule_check" do
before do
project.team << [user, :developer]
......
......@@ -108,7 +108,7 @@ describe 'gitlab:app namespace rake task' do
$stdout = orig_stdout
end
describe 'backup creation and deletion using annex and custom_hooks' do
describe 'backup creation and deletion using custom_hooks' do
let(:project) { create(:project) }
let(:user_backup_path) { "repositories/#{project.path_with_namespace}" }
......@@ -132,25 +132,6 @@ describe 'gitlab:app namespace rake task' do
Dir.chdir(@origin_cd)
end
context 'project uses git-annex and successfully creates backup' do
let(:filename) { "annex" }
it 'creates annex.tar and project bundle' do
tar_contents, exit_status = Gitlab::Popen.popen(%W{tar -tvf #{@backup_tar}})
expect(exit_status).to eq(0)
expect(tar_contents).to match(user_backup_path)
expect(tar_contents).to match("#{user_backup_path}/annex.tar")
expect(tar_contents).to match("#{user_backup_path}.bundle")
end
it 'restores files correctly' do
restore_backup
expect(Dir.entries(File.join(project.repository.path, "annex"))).to include("dummy.txt")
end
end
context 'project uses custom_hooks and successfully creates backup' do
let(:filename) { "custom_hooks" }
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment