Merge branch 'docs/git-filter-branch' into 'master'

Document how to use filter-branch to reduce the repo file size

Closes #1270

See merge request !914

doc/user/admin_area/settings/account_and_limit_settings.md

@@ -6,7 +6,7 @@
 
 Repositories within your GitLab instance can grow quickly, especially if you are
 using LFS. Their size can grow exponentially and eat up your storage device quite
-quickly.
+fast.
 
 In order to avoid this from happening, you can set a hard limit for your
 repositories' size. This limit can be set globally, per group, or per project,
@@ -19,24 +19,14 @@ These settings can be found within each project's settings, in a group's
 settings and in the Application Settings area for the global value
 (`/admin/application_settings`).
 
-### Repository size restrictions
+The very first push of a new project cannot be checked for size as of now, so
+the first push will allow you to upload more than the limit dictates, but every
+subsequent push will be denied. LFS objects, however, can be checked on first
+push and **will** be rejected if the sum of their sizes exceeds the maximum
+allowed repository size.
 
-When a project has reached its size limit, you will not be able to push to it,
-create a new merge request, or merge existing ones. You will still be able to
-create new issues, and clone the project though.
-
-Uploading LFS objects will also be denied.
-
-In order to lift these restrictions, the administrator of the GitLab instance
-needs to increase the limit on the particular project that exceeded it.
-
-### Current limitations for the repository size check
-
-The first push of a new project cannot be checked for size as of now, so the first
-push will allow you to upload more than the limit dictates, but every subsequent
-push will be denied.
-
-LFS objects, however, can be checked on first push and **will** be rejected if the
-sum of their sizes exceeds the maximum allowed repository size.
+For more manually purging the files, read the docs on
+[reducing the repository size using Git][repo-size].
 
 [ee-740]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/740
+[repo-size]: ../../project/repository/reducing_the_repo_size_using_git.md

doc/user/project/repository/reducing_the_repo_size_using_git.md

+# Reducing the repository size using Git
+
+A GitLab administrator can set a [repository size limit][admin-repo-size] which
+will prevent you to exceed it.
+
+When a project has reached its size limit, you will not be able to push to it,
+create a new merge request, or merge existing ones. You will still be able to
+create new issues, and clone the project though. Uploading LFS objects will
+also be denied.
+
+In order to lift these restrictions, the administrator of the GitLab instance
+needs to increase the limit on the particular project that exceeded it or you
+need to instruct Git to rewrite changes.
+
+If you exceed the repository size limit, your first thought might be to remove
+some data, make a new commit and push back to the repository. Unfortunately,
+it's not so easy and that workflow won't work. Deleting files in a commit doesn't
+actually reduce the size of the repo since the earlier commits and blobs are
+still around. What you need to do is rewrite history with Git's
+[`filter-branch` option][gitscm].
+
+Note that even with that method, until `git gc` runs on the GitLab side, the
+"removed" commits and blobs will still be around. And if a commit was ever
+included in an MR, or if a build was run for a commit, or if a user commented
+on it, it will be kept around too. So, in these cases the size will not decrease.
+
+The only fool proof way to actually decrease the repository size is to prune all
+the unneeded stuff locally, and then create a new project on GitLab and start
+using that instead.
+
+With that being said, you can try reducing your repository size with the
+following method.
+
+## Using `git filter-branch` to purge files
+
+>
+**Warning:**
+Make sure to first make a copy of your repository since rewriting history will
+purge the files and information you are about to delete. Also make sure to
+inform any collaborators to not use `pull` after your changes, but use `rebase`.
+
+1. Navigate to your repository:
+
+    ```
+    cd my_repository/
+    ```
+
+1. Change to the branch you want to remove the big file from:
+
+    ```
+    git checkout master
+    ```
+
+1. Use `filter-branch` to remove the big file:
+
+    ```
+    git filter-branch --force --tree-filter 'rm -f path/to/big_file.mpg' HEAD
+    ```
+
+1. Instruct Git to purge the unwanted data:
+
+    ```
+    git reflog expire --expire=now --all && git gc --prune=now --aggressive
+    ```
+
+1. Lastly, force push to the repository:
+
+    ```
+    git push --force origin master
+    ```
+
+Your repository should now be below the size limit.
+
+>**Note:**
+As an alternative to `filter-branch`, you can use the `bfg` tool with a
+command like: `bfg --delete-files path/to/big_file.mpg`. Read the
+[BFG Repo-Cleaner][bfg] documentation for more information.
+
+[admin-repo-size]: ../../admin_area/settings/account_and_limit_settings.md#repository-size-limit
+[bfg]: https://rtyley.github.io/bfg-repo-cleaner/
+[gitscm]: https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History#The-Nuclear-Option:-filter-branch