Merge branch 'master' of gitlab.com:gitlab-org/gitlab-ce into ce_upstream

.rubocop.yml

@@ -681,7 +681,7 @@ Style/UnlessElse:
 
 # Checks for %W when interpolation is not needed.
 Style/UnneededCapitalW:
-  Enabled: false
+  Enabled: true
 
 # TODO: Enable UnneededInterpolation Cop.
 # Checks for strings that are just an interpolated expression.

CHANGELOG

@@ -3,9 +3,12 @@ Please view this file on the master branch, on stable branches it's out of date.
 v 8.10.0 (unreleased)
   - Fix commit builds API, return all builds for all pipelines for given commit. !4849
   - Replace Haml with Hamlit to make view rendering faster. !3666
+  - Refactor repository paths handling to allow multiple git mount points
+  - Add Application Setting to configure default Repository Path for new projects
   - Wrap code blocks on Activies and Todos page. !4783 (winniehell)
   - Align flash messages with left side of page content !4959 (winniehell)
   - Display last commit of deleted branch in push events !4699 (winniehell)
+  - Apply the trusted_proxies config to the rack request object for use with rack_attack
   - Add Sidekiq queue duration to transaction metrics.
   - Let Workhorse serve format-patch diffs
   - Make images fit to the size of the viewport !4810
@@ -15,18 +18,39 @@ v 8.10.0 (unreleased)
   - Exclude email check from the standard health check
   - Fix changing issue state columns in milestone view
   - Add notification settings dropdown for groups
+  - Allow importing from Github using Personal Access Tokens. (Eric K Idema)
+  - API: Todos !3188 (Robert Schilling)
   - Fix user creation with stronger minimum password requirements !4054 (nathan-pmt)
   - PipelinesFinder uses git cache data
   - Check for conflicts with existing Project's wiki path when creating a new project.
+  - Don't instantiate a git tree on Projects show default view
   - Remove unused front-end variable -> default_issues_tracker
+  - Better caching of git calls on ProjectsController#show.
   - Add API endpoint for a group issues !4520 (mahcsig)
   - Add Bugzilla integration !4930 (iamtjg)
+  - Metrics for Rouge::Plugins::Redcarpet and Rouge::Formatters::HTMLGitlab
   - Allow [ci skip] to be in any case and allow [skip ci]. !4785 (simon_w)
   - Add basic system information like memory and disk usage to the admin panel
 
-v 8.9.4 (unreleased)
-  - Fixes middle click and double request when navigating through the file browser  !4891
-  - Add sub nav to file page view
+v 8.9.5 (unreleased)
+  - Improve the request / withdraw access button. !4860
+  - Fix assigning shared runners as admins. !4961
+  - Show "locked" label for locked runners on runners admin. !4961
+  - Fixes issues importing events in Import/Export. Import/Export version bumped to 0.1.1
+
+v 8.9.4
+  - Fix privilege escalation issue with OAuth external users.
+  - Ensure references to private repos aren't shown to logged-out users.
+  - Fixed search field blur not removing focus. !4704
+  - Resolve "Sub nav isn't showing on file view". !4890
+  - Fixes middle click and double request when navigating through the file browser. !4891
+  - Fixed URL on label button when filtering. !4897
+  - Fixed commit avatar alignment. !4933
+  - Do not show build retry link when build is active. !4967
+  - Fix restore Rake task warning message output. !4980
+  - Handle external issues in IssueReferenceFilter. !4988
+  - Expiry date on pinned nav cookie. !5009
+  - Updated breakpoint for sidebar pinning. !5019
 
 v 8.9.3
   - Fix encrypted data backwards compatibility after upgrading attr_encrypted gem. !4963
@@ -43,9 +67,7 @@ v 8.9.3
   - Fix missing avatar on system notes. !4954
   - Reduce overhead and optimize ProjectTeam#max_member_access performance. !4973
   - Use update_columns to by_pass all the dirty code on active_record. !4985
-  - Decreased min width of screen to 1280px for pinned sidebar
-  - Fix encrypted data backwards compatibility after upgrading attr_encrypted gem
-  - Update mobile button icons to be more inline with typical UI paradigms
+  - Fix restore Rake task warning message output !4980
 
 v 8.9.2
   - Fix visibility of snippets when searching.
@@ -96,7 +118,6 @@ v 8.9.1
   - Remove duplicate 'New Page' button on edit wiki page
 
 v 8.9.0
-v 8.9.0 (unreleased)
   - Fix group visibility form layout in application settings
   - Fix builds API response not including commit data
   - Fix error when CI job variables key specified but not defined
@@ -246,12 +267,17 @@ v 8.9.0
   - Filter parameters for request_uri value on instrumented transactions.
   - Remove duplicated keys add UNIQUE index to keys fingerprint column
   - ExtractsPath get ref_names from repository cache, if not there access git.
+  - Show a flash warning about the error detail of XHR requests which failed with status code 404 and 500
   - Cache user todo counts from TodoService
   - Ensure Todos counters doesn't count Todos for projects pending delete
   - Add left/right arrows horizontal navigation
   - Add tooltip to pin/unpin navbar
   - Add new sub nav style to Wiki and Graphs sub navigation
 
+v 8.8.7
+  - Fix privilege escalation issue with OAuth external users.
+  - Ensure references to private repos aren't shown to logged-out users.
+
 v 8.8.6
   - Fix visibility of snippets when searching.
   - Update omniauth-saml to 1.6.0 !4951
@@ -386,6 +412,10 @@ v 8.8.0
   - When creating a .gitignore file a dropdown with templates will be provided
   - Shows the issue/MR list search/filter form and corrects the mobile styling for guest users. #17562
 
+v 8.7.9
+  - Fix privilege escalation issue with OAuth external users.
+  - Ensure references to private repos aren't shown to logged-out users.
+
 v 8.7.8
   - Fix visibility of snippets when searching.
   - Update omniauth-saml to 1.6.0 !4951

GITLAB_SHELL_VERSION

-3.0.0
+3.1.0

Gemfile

@@ -262,7 +262,6 @@ group :development do
   gem 'brakeman', '~> 3.3.0', require: false
 
   gem 'letter_opener_web', '~> 1.3.0'
-  gem 'quiet_assets', '~> 1.0.2'
   gem 'rerun', '~> 0.11.0'
   gem 'bullet', require: false
   gem 'rblineprof', platform: :mri, require: false
@@ -314,7 +313,6 @@ group :development, :test do
   gem 'rubocop', '~> 0.40.0', require: false
   gem 'rubocop-rspec', '~> 1.5.0', require: false
   gem 'scss_lint', '~> 0.47.0', require: false
-  gem 'coveralls', '~> 0.8.2', require: false
   gem 'simplecov', '~> 0.11.0', require: false
   gem 'flog', require: false
   gem 'flay', require: false
@@ -361,3 +359,4 @@ gem 'health_check', '~> 1.5.1'
 
 # System information
 gem 'vmstat', '~> 2.1.0'
+gem 'sys-filesystem', '~> 1.1.6'

Gemfile.lock

@@ -141,12 +141,6 @@ GEM
     colorize (0.7.7)
     concurrent-ruby (1.0.2)
     connection_pool (2.2.0)
-    coveralls (0.8.13)
-      json (~> 1.8)
-      simplecov (~> 0.11.0)
-      term-ansicolor (~> 1.3)
-      thor (~> 0.19.1)
-      tins (~> 1.6.0)
     crack (0.4.3)
       safe_yaml (~> 1.0.0)
     creole (0.5.0)
@@ -529,8 +523,6 @@ GEM
     pry-rails (0.3.4)
       pry (>= 0.9.10)
     pyu-ruby-sasl (0.0.3.3)
-    quiet_assets (1.0.3)
-      railties (>= 3.1, < 5.0)
     rack (1.6.4)
     rack-accept (0.4.5)
       rack (>= 0.4)
@@ -730,10 +722,10 @@ GEM
       spring (>= 0.9.1)
     spring-commands-teaspoon (0.0.2)
       spring (>= 0.9.1)
-    sprockets (3.6.0)
+    sprockets (3.6.2)
       concurrent-ruby (~> 1.0)
       rack (> 1, < 3)
-    sprockets-rails (3.0.4)
+    sprockets-rails (3.1.1)
       actionpack (>= 4.0)
       activesupport (>= 4.0)
       sprockets (>= 3.0.0)
@@ -745,6 +737,8 @@ GEM
       activerecord (>= 4.1, < 5.1)
       state_machines-activemodel (>= 0.3.0)
     stringex (2.5.2)
+    sys-filesystem (1.1.6)
+      ffi
     systemu (2.6.5)
     task_list (1.0.2)
       html-pipeline
@@ -753,8 +747,6 @@ GEM
     teaspoon-jasmine (2.2.0)
       teaspoon (>= 1.0.0)
     temple (0.7.7)
-    term-ansicolor (1.3.2)
-      tins (~> 1.0)
     test_after_commit (0.4.2)
       activerecord (>= 3.2)
     thin (1.6.4)
@@ -775,7 +767,6 @@ GEM
       mime-types
       multi_json (~> 1.7)
       twitter-stream (~> 0.1)
-    tins (1.6.0)
     turbolinks (2.5.3)
       coffee-rails
     twitter-stream (0.1.16)
@@ -868,7 +859,6 @@ DEPENDENCIES
   chronic_duration (~> 0.10.6)
   coffee-rails (~> 4.1.0)
   connection_pool (~> 2.0)
-  coveralls (~> 0.8.2)
   creole (~> 0.5.0)
   d3_rails (~> 3.5.0)
   database_cleaner (~> 1.4.0)
@@ -961,7 +951,6 @@ DEPENDENCIES
   poltergeist (~> 1.9.0)
   premailer-rails (~> 1.9.0)
   pry-rails
-  quiet_assets (~> 1.0.2)
   rack-attack (~> 4.3.1)
   rack-cors (~> 0.4.0)
   rack-oauth2 (~> 1.2.1)
@@ -1009,6 +998,7 @@ DEPENDENCIES
   spring-commands-teaspoon (~> 0.0.2)
   sprockets (~> 3.6.0)
   state_machines-activerecord (~> 0.4.0)
+  sys-filesystem (~> 1.1.6)
   task_list (~> 1.0.2)
   teaspoon (~> 1.1.0)
   teaspoon-jasmine (~> 2.2.0)

app/assets/javascripts/application.js.coffee

@@ -188,6 +188,15 @@ $ ->
       else
         buttons.enable()
 
+  $(document).ajaxError (e, xhrObj, xhrSetting, xhrErrorText) ->
+
+    if xhrObj.status is 401
+      new Flash 'You need to be logged in.', 'alert'
+
+    else if xhrObj.status in [ 404, 500 ]
+      new Flash 'Something went wrong on our end.', 'alert'
+
+
   # Show/Hide the profile menu when hovering the account box
   $('.account-box').hover -> $(@).toggleClass('hover')
 
@@ -264,8 +273,8 @@ $ ->
   new Aside()
 
   # Sidenav pinning
-  if $window.width() < 1280 and $.cookie('pin_nav') is 'true'
-    $.cookie('pin_nav', 'false', { path: '/' })
+  if $window.width() < 1024 and $.cookie('pin_nav') is 'true'
+    $.cookie('pin_nav', 'false', { path: '/', expires: 365 * 10 })
     $('.page-with-sidebar')
       .toggleClass('page-sidebar-collapsed page-sidebar-expanded')
       .removeClass('page-sidebar-pinned')
@@ -296,7 +305,7 @@ $ ->
                .toggleClass('header-collapsed header-expanded')
 
       # Save settings
-      $.cookie 'pin_nav', doPinNav, { path: '/' }
+      $.cookie 'pin_nav', doPinNav, { path: '/', expires: 365 * 10 }
 
       if $.cookie('pin_nav') is 'true' or doPinNav
         tooltipText = 'Unpin navigation'

app/assets/javascripts/gfm_auto_complete.js.coffee

@@ -4,7 +4,7 @@ window.GitLab ?= {}
 GitLab.GfmAutoComplete =
   dataLoading: false
   dataLoaded: false
-
+  cachedData: {}
   dataSource: ''
 
   # Emoji
@@ -55,7 +55,7 @@ GitLab.GfmAutoComplete =
     @setupAtWho()
 
     if @dataSource
-      if !@dataLoading
+      if not @dataLoading and not @cachedData
         @dataLoading = true
 
         # We should wait until initializations are done
@@ -70,6 +70,8 @@ GitLab.GfmAutoComplete =
             @loadData(data)
         , 1000)
 
+      if @cachedData?
+        @loadData(@cachedData)
 
   setupAtWho: ->
     # Emoji
@@ -205,6 +207,7 @@ GitLab.GfmAutoComplete =
     $.getJSON(dataSource)
 
   loadData: (data) ->
+    @cachedData = data
     @dataLoaded = true
 
     # load members

app/assets/stylesheets/framework/mobile.scss

@@ -71,6 +71,10 @@
     display: none;
   }
 
+  .group-right-buttons {
+    display: none;
+  }
+
   .container .title {
     padding-left: 15px !important;
   }

app/assets/stylesheets/framework/variables.scss

@@ -7,7 +7,7 @@ $gutter_collapsed_width: 62px;
 $gutter_width: 290px;
 $gutter_inner_width: 258px;
 $sidebar-transition-duration: .15s;
-$sidebar-breakpoint: 1280px;
+$sidebar-breakpoint: 1024px;
 
 /*
  * UI elements

app/assets/stylesheets/pages/groups.scss

@@ -46,15 +46,18 @@
 }
 
 .groups-cover-block {
-
   .container-fluid {
     position: relative;
   }
 
-  .access-request-button {
+  .group-right-buttons {
+    position: absolute;
+    right: 16px;
+    .btn {
       @include btn-gray;
-    margin-right: 10px;
-    text-transform: none;
+      padding: 3px 10px;
+      background-color: $background-color;
+    }
   }
 }
 

app/assets/stylesheets/pages/projects.scss

@@ -266,18 +266,6 @@
     @media (max-width: $screen-md-max) {
       top: 0;
     }
-
-    .access-request-button {
-      position: absolute;
-      right: 0;
-      bottom: 61px;
-
-      @media (max-width: $screen-md-max) {
-        position: relative;
-        bottom: 0;
-        margin-right: 10px;
-      }
-    }
   }
 
   @media (max-width: $screen-md-max) {

app/controllers/admin/application_settings_controller.rb

@@ -117,6 +117,7 @@ class Admin::ApplicationSettingsController < Admin::ApplicationController
       :elasticsearch_search,
       :elasticsearch_host,
       :elasticsearch_port,
+      :repository_storage,
       restricted_visibility_levels: [],
       import_sources: [],
       disabled_oauth_sign_in_sources: []

app/controllers/admin/runner_projects_controller.rb

@@ -4,8 +4,6 @@ class Admin::RunnerProjectsController < Admin::ApplicationController
   def create
     @runner = Ci::Runner.find(params[:runner_project][:runner_id])
 
-    return head(403) if @runner.is_shared? || @runner.locked?
-
     runner_project = @runner.assign_to(@project, current_user)
 
     if runner_project.persisted?

app/controllers/admin/system_info_controller.rb

 class Admin::SystemInfoController < Admin::ApplicationController
+  EXCLUDED_MOUNT_OPTIONS = [
+    'nobrowse',
+    'read-only',
+    'ro'
+  ]
+
+  EXCLUDED_MOUNT_TYPES = [
+    'autofs',
+    'binfmt_misc',
+    'cgroup',
+    'debugfs',
+    'devfs',
+    'devpts',
+    'devtmpfs',
+    'efivarfs',
+    'fuse.gvfsd-fuse',
+    'fuseblk',
+    'fusectl',
+    'hugetlbfs',
+    'mqueue',
+    'proc',
+    'pstore',
+    'securityfs',
+    'sysfs',
+    'tmpfs',
+    'tracefs',
+    'vfat'
+  ]
+
   def show
     system_info = Vmstat.snapshot
+    mounts = Sys::Filesystem.mounts
+
+    @disks = []
+    mounts.each do |mount|
+      mount_options = mount.options.split(',')
+
+      next if (EXCLUDED_MOUNT_OPTIONS & mount_options).any?
+      next if (EXCLUDED_MOUNT_TYPES & [mount.mount_type]).any?
+
+      begin
+        disk = Sys::Filesystem.stat(mount.mount_point)
+        @disks.push({
+          bytes_total: disk.bytes_total,
+          bytes_used:  disk.bytes_used,
+          disk_name:   mount.name,
+          mount_path:  disk.path
+        })
+      rescue Sys::Filesystem::Error
+      end
+    end
 
     @cpus = system_info.cpus.length
 
     @mem_used = system_info.memory.active_bytes
     @mem_total = system_info.memory.total_bytes
-
-    @disk_used = system_info.disks[0].used_bytes
-    @disk_total = system_info.disks[0].total_bytes
   end
 end

app/controllers/import/github_controller.rb

 class Import::GithubController < Import::BaseController
   before_action :verify_github_import_enabled
-  before_action :github_auth, except: :callback
+  before_action :github_auth, only: [:status, :jobs, :create]
 
   rescue_from Octokit::Unauthorized, with: :github_unauthorized
 
+  helper_method :logged_in_with_github?
+
+  def new
+    if logged_in_with_github?
+      go_to_github_for_permissions
+    elsif session[:github_access_token]
+      redirect_to status_import_github_url
+    end
+  end
+
   def callback
     session[:github_access_token] = client.get_token(params[:code])
     redirect_to status_import_github_url
   end
 
+  def personal_access_token
+    session[:github_access_token] = params[:personal_access_token]
+    redirect_to status_import_github_url
+  end
+
   def status
     @repos = client.repos
     @already_added_projects = current_user.created_projects.where(import_type: "github")
@@ -57,10 +72,14 @@ class Import::GithubController < Import::BaseController
   end
 
   def github_unauthorized
-    go_to_github_for_permissions
+    session[:github_access_token] = nil
+    redirect_to new_import_github_url,
+      alert: 'Access denied to your GitHub account.'
   end
 
-  private
+  def logged_in_with_github?
+    current_user.identities.exists?(provider: 'github')
+  end
 
   def access_params
     { github_access_token: session[:github_access_token] }

app/controllers/projects/runner_projects_controller.rb

@@ -6,8 +6,7 @@ class Projects::RunnerProjectsController < Projects::ApplicationController
   def create
     @runner = Ci::Runner.find(params[:runner_project][:runner_id])
 
-    return head(403) if @runner.is_shared? || @runner.locked?
-    return head(403) unless current_user.ci_authorized_runners.include?(@runner)
+    return head(403) unless can?(current_user, :assign_runner, @runner)
 
     path = runners_path(project)
     runner_project = @runner.assign_to(project, current_user)

app/controllers/projects_controller.rb

@@ -4,7 +4,8 @@ class ProjectsController < Projects::ApplicationController
   before_action :authenticate_user!, except: [:show, :activity, :refs]
   before_action :project, except: [:new, :create]
   before_action :repository, except: [:new, :create]
-  before_action :assign_ref_vars, :tree, only: [:show], if: :repo_exists?
+  before_action :assign_ref_vars, only: [:show], if: :repo_exists?
+  before_action :tree, only: [:show], if: :project_view_files?
 
   # Authorize
   before_action :authorize_admin_project!, only: [:edit, :update, :housekeeping, :download_export, :export, :remove_export, :generate_new_export]
@@ -315,6 +316,10 @@ class ProjectsController < Projects::ApplicationController
     project.repository_exists? && !project.empty_repo?
   end
 
+  def project_view_files?
+    current_user && current_user.project_view == 'files'
+  end
+
   # Override extract_ref from ExtractsPath, which returns the branch and file path
   # for the blob/tree, which in this case is just the root of the default branch.
   # This way we avoid to access the repository.ref_names.

app/finders/todos_finder.rb

@@ -25,6 +25,7 @@ class TodosFinder
   def execute
     items = current_user.todos
     items = by_action_id(items)
+    items = by_action(items)
     items = by_author(items)
     items = by_project(items)
     items = by_state(items)
@@ -43,6 +44,18 @@ class TodosFinder
     params[:action_id]
   end
 
+  def to_action_id
+    Todo::ACTION_NAMES.key(action.to_sym)
+  end
+
+  def action?
+    action.present? && to_action_id
+  end
+
+  def action
+    params[:action]
+  end
+
   def author?
     params[:author_id].present?
   end
@@ -96,6 +109,14 @@ class TodosFinder
     params[:type]
   end
 
+  def by_action(items)
+    if action?
+      items = items.where(action: to_action_id)
+    end
+
+    items
+  end
+
   def by_action_id(items)
     if action_id?
       items = items.where(action: action_id)

app/helpers/application_settings_helper.rb

@@ -82,4 +82,12 @@ module ApplicationSettingsHelper
       end
     end
   end
+
+  def repository_storage_options_for_select
+    options = Gitlab.config.repositories.storages.map do |name, path|
+      ["#{name} - #{path}", name]
+    end
+
+    options_for_select(options, @application_setting.repository_storage)
+  end
 end

app/helpers/projects_helper.rb

@@ -344,9 +344,9 @@ module ProjectsHelper
     end
   end
 
-  def sanitize_repo_path(message)
+  def sanitize_repo_path(project, message)
     return '' unless message.present?
 
-    message.strip.gsub(Gitlab.config.gitlab_shell.repos_path.chomp('/'), "[REPOS PATH]")
+    message.strip.gsub(project.repository_storage_path.chomp('/'), "[REPOS PATH]")
   end
 end

app/models/ability.rb

 class Ability
   class << self
+    # rubocop: disable Metrics/CyclomaticComplexity
     def allowed(user, subject)
       return anonymous_abilities(user, subject) if user.nil?
       return [] unless user.is_a?(User)
@@ -20,6 +21,7 @@ class Ability
         when ProjectMember then project_member_abilities(user, subject)
         when User then user_abilities
         when ExternalIssue, Deployment, Environment then project_abilities(user, subject.project)
+        when Ci::Runner then runner_abilities(user, subject)
         else []
         end.concat(global_abilities(user))
 
@@ -535,6 +537,18 @@ class Ability
       rules
     end
 
+    def runner_abilities(user, runner)
+      if user.is_admin?
+        [:assign_runner]
+      elsif runner.is_shared? || runner.locked?
+        []
+      elsif user.ci_authorized_runners.include?(runner)
+        [:assign_runner]
+      else
+        []
+      end
+    end
+
     def user_abilities
       [:read_user]
     end

app/models/application_setting.rb

@@ -63,6 +63,10 @@ class ApplicationSetting < ActiveRecord::Base
             presence: { message: "can't be blank when indexing is enabled" },
             if: :elasticsearch_indexing?
   
+  validates :repository_storage,
+    presence: true,
+    inclusion: { in: ->(_object) { Gitlab.config.repositories.storages.keys } }
+
   validates_each :restricted_visibility_levels do |record, attr, value|
     unless value.nil?
       value.each do |level|
@@ -144,6 +148,7 @@ class ApplicationSetting < ActiveRecord::Base
       container_registry_token_expire_delay: 5,
       elasticsearch_host: ENV['ELASTIC_HOST'] || 'localhost',
       elasticsearch_port: ENV['ELASTIC_PORT'] || '9200'
+      repository_storage: 'default',
     )
   end
 

app/models/ci/build.rb

@@ -90,7 +90,7 @@ module Ci
     end
 
     def retryable?
-      project.builds_enabled? && commands.present?
+      project.builds_enabled? && commands.present? && complete?
     end
 
     def retried?

app/models/issue.rb

@@ -25,6 +25,8 @@ class Issue < ActiveRecord::Base
   belongs_to :project
   belongs_to :moved_to, class_name: 'Issue'
 
+  has_many :events, as: :target, dependent: :destroy
+
   validates :project, presence: true
 
   scope :cared, ->(user) { where(assignee_id: user) }

app/models/merge_request.rb

@@ -15,6 +15,8 @@ class MergeRequest < ActiveRecord::Base
   has_many :approvals, dependent: :destroy
   has_many :approvers, as: :target, dependent: :destroy
 
+  has_many :events, as: :target, dependent: :destroy
+
   serialize :merge_params, Hash
 
   after_create :create_merge_request_diff, unless: :importing

app/models/milestone.rb

@@ -18,6 +18,7 @@ class Milestone < ActiveRecord::Base
   has_many :labels, -> { distinct.reorder('labels.title') },  through: :issues
   has_many :merge_requests
   has_many :participants, -> { distinct.reorder('users.name') }, through: :issues, source: :assignee
+  has_many :events, as: :target, dependent: :destroy
 
   scope :active, -> { with_state(:active) }
   scope :closed, -> { with_state(:closed) }

app/models/namespace.rb

@@ -21,8 +21,10 @@ class Namespace < ActiveRecord::Base
 
   delegate :name, to: :owner, allow_nil: true, prefix: true
 
-  after_create :ensure_dir_exist
   after_update :move_dir, if: :path_changed?
+
+  # Save the storage paths before the projects are destroyed to use them on after destroy
+  before_destroy(prepend: true) { @old_repository_storage_paths = repository_storage_paths }
   after_destroy :rm_dir
 
   scope :root, -> { where('type IS NULL') }
@@ -87,33 +89,12 @@ class Namespace < ActiveRecord::Base
     owner_name
   end
 
-  def ensure_dir_exist
-    gitlab_shell.add_namespace(path)
-  end
-
-  def rm_dir
-    # Move namespace directory into trash.
-    # We will remove it later async
-    new_path = "#{path}+#{id}+deleted"
-
-    if gitlab_shell.mv_namespace(path, new_path)
-      message = "Namespace directory \"#{path}\" moved to \"#{new_path}\""
-      Gitlab::AppLogger.info message
-
-      # Remove namespace directroy async with delay so
-      # GitLab has time to remove all projects first
-      GitlabShellWorker.perform_in(5.minutes, :rm_namespace, new_path)
-    end
-  end
-
   def move_dir
-    # Ensure old directory exists before moving it
-    gitlab_shell.add_namespace(path_was)
-
     if any_project_has_container_registry_tags?
       raise Exception.new('Namespace cannot be moved, because at least one project has tags in container registry')
     end
 
+<<<<<<< HEAD
     if gitlab_shell.mv_namespace(path_was, path)
       Gitlab::UploadsTransfer.new.rename_namespace(path_was, path)
       Gitlab::PagesTransfer.new.rename_namespace(path_was, path)
@@ -128,11 +109,32 @@ class Namespace < ActiveRecord::Base
         # Returning false does not rollback after_* transaction but gives
         # us information about failing some of tasks
         false
-      end
-    else
+=======
+    # Move the namespace directory in all storages paths used by member projects
+    repository_storage_paths.each do |repository_storage_path|
+      # Ensure old directory exists before moving it
+      gitlab_shell.add_namespace(repository_storage_path, path_was)
+
+      unless gitlab_shell.mv_namespace(repository_storage_path, path_was, path)
         # if we cannot move namespace directory we should rollback
         # db changes in order to prevent out of sync between db and fs
         raise Exception.new('namespace directory cannot be moved')
+>>>>>>> 557ca2b31ff503b36a4b65af2641fcd0f5682d5b
+      end
+    end
+
+    Gitlab::UploadsTransfer.new.rename_namespace(path_was, path)
+
+    # If repositories moved successfully we need to
+    # send update instructions to users.
+    # However we cannot allow rollback since we moved namespace dir
+    # So we basically we mute exceptions in next actions
+    begin
+      send_update_instructions
+    rescue
+      # Returning false does not rollback after_* transaction but gives
+      # us information about failing some of tasks
+      false
     end
   end
 
@@ -153,4 +155,33 @@ class Namespace < ActiveRecord::Base
   def find_fork_of(project)
     projects.joins(:forked_project_link).find_by('forked_project_links.forked_from_project_id = ?', project.id)
   end
+
+  private
+
+  def repository_storage_paths
+    # We need to get the storage paths for all the projects, even the ones that are
+    # pending delete. Unscoping also get rids of the default order, which causes
+    # problems with SELECT DISTINCT.
+    Project.unscoped do
+      projects.select('distinct(repository_storage)').to_a.map(&:repository_storage_path)
+    end
+  end
+
+  def rm_dir
+    # Remove the namespace directory in all storages paths used by member projects
+    @old_repository_storage_paths.each do |repository_storage_path|
+      # Move namespace directory into trash.
+      # We will remove it later async
+      new_path = "#{path}+#{id}+deleted"
+
+      if gitlab_shell.mv_namespace(repository_storage_path, path, new_path)
+        message = "Namespace directory \"#{path}\" moved to \"#{new_path}\""
+        Gitlab::AppLogger.info message
+
+        # Remove namespace directroy async with delay so
+        # GitLab has time to remove all projects first
+        GitlabShellWorker.perform_in(5.minutes, :rm_namespace, repository_storage_path, new_path)
+      end
+    end
+  end
 end

app/models/note.rb

@@ -22,6 +22,7 @@ class Note < ActiveRecord::Base
   belongs_to :updated_by, class_name: "User"
 
   has_many :todos, dependent: :destroy
+  has_many :events, as: :target, dependent: :destroy
 
   delegate :gfm_reference, :local_reference, to: :noteable
   delegate :name, to: :project, prefix: true

app/models/project.rb

@@ -25,8 +25,12 @@ class Project < ActiveRecord::Base
   default_value_for :wiki_enabled, gitlab_config_features.wiki
   default_value_for :snippets_enabled, gitlab_config_features.snippets
   default_value_for :container_registry_enabled, gitlab_config_features.container_registry
+  default_value_for(:repository_storage) { current_application_settings.repository_storage }
   default_value_for(:shared_runners_enabled) { current_application_settings.shared_runners_enabled }
 
+  after_create :ensure_dir_exist
+  after_save :ensure_dir_exist, if: :namespace_id_changed?
+
   # set last_activity_at to the same as created_at
   after_create :set_last_activity_at
   def set_last_activity_at
@@ -175,6 +179,9 @@ class Project < ActiveRecord::Base
   validate :visibility_level_allowed_by_group
   validate :visibility_level_allowed_as_fork
   validate :check_wiki_path_conflict
+  validates :repository_storage,
+    presence: true,
+    inclusion: { in: ->(_object) { Gitlab.config.repositories.storages.keys } }
 
   add_authentication_token_field :runners_token
   before_save :ensure_runners_token
@@ -409,6 +416,10 @@ class Project < ActiveRecord::Base
     end
   end
 
+  def repository_storage_path
+    Gitlab.config.repositories.storages[repository_storage]
+  end
+
   def team
     @team ||= ProjectTeam.new(self)
   end
@@ -956,12 +967,12 @@ class Project < ActiveRecord::Base
       raise Exception.new('Project cannot be renamed, because tags are present in its container registry')
     end
 
-    if gitlab_shell.mv_repository(old_path_with_namespace, new_path_with_namespace)
+    if gitlab_shell.mv_repository(repository_storage_path, old_path_with_namespace, new_path_with_namespace)
       # If repository moved successfully we need to send update instructions to users.
       # However we cannot allow rollback since we moved repository
       # So we basically we mute exceptions in next actions
       begin
-        gitlab_shell.mv_repository("#{old_path_with_namespace}.wiki", "#{new_path_with_namespace}.wiki")
+        gitlab_shell.mv_repository(repository_storage_path, "#{old_path_with_namespace}.wiki", "#{new_path_with_namespace}.wiki")
         send_move_instructions(old_path_with_namespace)
         reset_events_cache
 
@@ -1111,7 +1122,7 @@ class Project < ActiveRecord::Base
   def create_repository
     # Forked import is handled asynchronously
     unless forked?
-      if gitlab_shell.add_repository(path_with_namespace)
+      if gitlab_shell.add_repository(repository_storage_path, path_with_namespace)
         repository.after_create
         true
       else
@@ -1375,4 +1386,8 @@ class Project < ActiveRecord::Base
     _, status = Gitlab::Popen.popen(%W(find #{export_path} -not -path #{export_path} -delete))
     status.zero?
   end
+
+  def ensure_dir_exist
+    gitlab_shell.add_namespace(repository_storage_path, namespace.path)
+  end
 end

app/models/project_wiki.rb

@@ -172,7 +172,7 @@ class ProjectWiki
   private
 
   def init_repo(path_with_namespace)
-    gitlab_shell.add_repository(path_with_namespace)
+    gitlab_shell.add_repository(project.repository_storage_path, path_with_namespace)
   end
 
   def commit_details(action, message = nil, title = nil)
@@ -186,7 +186,7 @@ class ProjectWiki
   end
 
   def path_to_repo
-    @path_to_repo ||= File.join(Gitlab.config.gitlab_shell.repos_path, "#{path_with_namespace}.git")
+    @path_to_repo ||= File.join(project.repository_storage_path, "#{path_with_namespace}.git")
   end
 
   def update_project_activity

app/models/repository.rb

@@ -47,7 +47,7 @@ class Repository
   # Return absolute path to repository
   def path_to_repo
     @path_to_repo ||= File.expand_path(
-      File.join(Gitlab.config.gitlab_shell.repos_path, path_with_namespace + ".git")
+      File.join(@project.repository_storage_path, path_with_namespace + ".git")
     )
   end
 
@@ -298,24 +298,26 @@ class Repository
     end
   end
 
+  # Keys for data that can be affected for any commit push.
   def cache_keys
-    %i(size branch_names tag_names branch_count tag_count commit_count
+    %i(size commit_count
        readme version contribution_guide changelog
        license_blob license_key gitignore)
   end
 
+  # Keys for data on branch/tag operations.
+  def cache_keys_for_branches_and_tags
+    %i(branch_names tag_names branch_count tag_count)
+  end
+
   def build_cache
-    cache_keys.each do |key|
+    (cache_keys + cache_keys_for_branches_and_tags).each do |key|
       unless cache.exist?(key)
         send(key)
       end
     end
   end
 
-  def expire_gitignore
-    cache.expire(:gitignore)
-  end
-
   def expire_tags_cache
     cache.expire(:tag_names)
     @tags = nil
@@ -338,8 +340,6 @@ class Repository
     # This ensures this particular cache is flushed after the first commit to a
     # new repository.
     expire_emptiness_caches if empty?
-    expire_branch_count_cache
-    expire_tag_count_cache
   end
 
   def expire_branch_cache(branch_name = nil)

app/models/todo.rb

@@ -4,6 +4,13 @@ class Todo < ActiveRecord::Base
   BUILD_FAILED = 3
   MARKED       = 4
 
+  ACTION_NAMES = {
+    ASSIGNED => :assigned,
+    MENTIONED => :mentioned,
+    BUILD_FAILED => :build_failed,
+    MARKED => :marked
+  }
+
   belongs_to :author, class_name: "User"
   belongs_to :note
   belongs_to :project
@@ -34,6 +41,10 @@ class Todo < ActiveRecord::Base
     action == BUILD_FAILED
   end
 
+  def action_name
+    ACTION_NAMES[action]
+  end
+
   def body
     if note.present?
       note.note

app/services/projects/destroy_service.rb

@@ -72,13 +72,13 @@ module Projects
       return true if params[:skip_repo] == true
 
       # There is a possibility project does not have repository or wiki
-      return true unless gitlab_shell.exists?(path + '.git')
+      return true unless gitlab_shell.exists?(project.repository_storage_path, path + '.git')
 
       new_path = removal_path(path)
 
-      if gitlab_shell.mv_repository(path, new_path)
+      if gitlab_shell.mv_repository(project.repository_storage_path, path, new_path)
         log_info("Repository \"#{path}\" moved to \"#{new_path}\"")
-        GitlabShellWorker.perform_in(5.minutes, :remove_repository, new_path)
+        GitlabShellWorker.perform_in(5.minutes, :remove_repository, project.repository_storage_path, new_path)
       else
         false
       end

app/services/projects/housekeeping_service.rb

@@ -24,7 +24,7 @@ module Projects
     def execute
       raise LeaseTaken unless try_obtain_lease
 
-      GitlabShellOneShotWorker.perform_async(:gc, @project.path_with_namespace)
+      GitlabShellOneShotWorker.perform_async(:gc, @project.repository_storage_path, @project.path_with_namespace)
     ensure
       Gitlab::Metrics.measure(:reset_pushes_since_gc) do
         @project.update_column(:pushes_since_gc, 0)

app/services/projects/import_service.rb

@@ -42,7 +42,7 @@ module Projects
 
     def import_repository
       begin
-        gitlab_shell.import_repository(project.path_with_namespace, project.import_url)
+        gitlab_shell.import_repository(project.repository_storage_path, project.path_with_namespace, project.import_url)
       rescue Gitlab::Shell::Error => e
         raise Error,  "Error importing repository #{project.import_url} into #{project.path_with_namespace} - #{e.message}"
       end

app/services/projects/transfer_service.rb

@@ -50,12 +50,12 @@ module Projects
         project.send_move_instructions(old_path)
 
         # Move main repository
-        unless gitlab_shell.mv_repository(old_path, new_path)
+        unless gitlab_shell.mv_repository(project.repository_storage_path, old_path, new_path)
           raise TransferError.new('Cannot move project')
         end
 
         # Move wiki repo also if present
-        gitlab_shell.mv_repository("#{old_path}.wiki", "#{new_path}.wiki")
+        gitlab_shell.mv_repository(project.repository_storage_path, "#{old_path}.wiki", "#{new_path}.wiki")
 
         # clear project cached events
         project.reset_events_cache

app/views/admin/application_settings/_form.html.haml

@@ -322,6 +322,15 @@
       .col-sm-10
         = f.text_field :sentry_dsn, class: 'form-control'
 
+  %fieldset
+    %legend Repository Storage
+    .form-group
+      = f.label :repository_storage, 'Storage path for new projects', class: 'control-label col-sm-2'
+      .col-sm-10
+        = f.select :repository_storage, repository_storage_options_for_select, {}, class: 'form-control'
+        .help-block
+          You can manage the repository storage paths in your gitlab.yml configuration file
+
   %fieldset
     %legend Repository Checks
     .form-group

app/views/admin/background_jobs/show.html.haml

@@ -2,7 +2,7 @@
 - page_title "Background Jobs"
 = render 'admin/background_jobs/head'
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   %h3.page-title Background Jobs
   %p.light GitLab uses #{link_to "sidekiq", "http://sidekiq.org/"} library for async job processing
 

app/views/admin/builds/index.html.haml

 - @no_container = true
 = render "admin/dashboard/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
 
   .top-area
     %ul.nav-links

app/views/admin/dashboard/index.html.haml

 - @no_container = true
 = render "admin/dashboard/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .admin-dashboard.prepend-top-default
     .row
       .col-md-4

app/views/admin/groups/index.html.haml

@@ -2,7 +2,7 @@
 - page_title "Groups"
 = render "admin/dashboard/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   %h3.page-title
     Groups (#{number_with_delimiter(@groups.total_count)})
 
@@ -39,7 +39,6 @@
       = link_to 'New Group', new_admin_group_path, class: "btn btn-new"
 
   %ul.content-list
-    - @groups.each do |group|
-      = render 'group', group: group
+    = render @groups
 
   = paginate @groups, theme: "gitlab"

app/views/admin/health_check/show.html.haml

@@ -2,7 +2,7 @@
 - page_title "Health Check"
 = render 'admin/background_jobs/head'
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   %h3.page-title
     Health Check
   .bs-callout.clearfix

app/views/admin/logs/show.html.haml

@@ -5,7 +5,7 @@
              Gitlab::RepositoryCheckLogger]
 = render 'admin/background_jobs/head'
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   %ul.nav-links.log-tabs
     - loggers.each do |klass|
       %li{ class: (klass == Gitlab::GitLogger ? 'active' : '') }

app/views/admin/projects/index.html.haml

@@ -3,7 +3,7 @@
 = render 'shared/show_aside'
 = render "admin/dashboard/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .row.prepend-top-default
     %aside.col-md-3
       .panel.admin-filter

app/views/admin/runners/_runner.html.haml

@@ -4,6 +4,8 @@
       %span.label.label-success shared
     - else
       %span.label.label-info specific
+    - if runner.locked?
+      %span.label.label-warning locked
     - unless runner.active?
       %span.label.label-danger paused
 

app/views/admin/runners/index.html.haml

 - @no_container = true
 = render "admin/dashboard/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
 
   %p.prepend-top-default
     %span
@@ -39,6 +39,9 @@
         %li
           %span.label.label-info specific
           \- run builds from assigned projects
+        %li
+          %span.label.label-warning locked
+          \- runner cannot be assigned to other projects
         %li
           %span.label.label-danger paused
           \- runner will not receive any new builds

app/views/admin/system_info/show.html.haml

@@ -2,7 +2,7 @@
 - page_title "System Info"
 = render 'admin/background_jobs/head'
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .prepend-top-default
   .row
     .col-sm-4
@@ -17,6 +17,9 @@
           %h1= "#{number_to_human_size(@mem_used)} / #{number_to_human_size(@mem_total)}"
     .col-sm-4
       .light-well
-        %h4 Disk
+        %h4 Disks
         .data
-          %h1= "#{number_to_human_size(@disk_used)} / #{number_to_human_size(@disk_total)}"
+          - @disks.each do |disk|
+            %h1= "#{number_to_human_size(disk[:bytes_used])} / #{number_to_human_size(disk[:bytes_total])}"
+            %p= "#{disk[:disk_name]}"
+            %p= "#{disk[:mount_path]}"

app/views/admin/users/index.html.haml

@@ -3,7 +3,7 @@
 = render 'shared/show_aside'
 = render "admin/dashboard/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .admin-filter
     %ul.nav-links
       %li{class: "#{'active' unless params[:filter]}"}

app/views/groups/show.html.haml

@@ -5,7 +5,7 @@
     = auto_discovery_link_tag(:atom, group_url(@group, format: :atom, private_token: current_user.private_token), title: "#{@group.name} activity")
 
 .cover-block.groups-cover-block
-  %div{ class: (container_class) }
+  %div{ class: container_class }
     = link_to group_icon(@group), target: '_blank' do
       = image_tag group_icon(@group), class: "avatar group-avatar s70"
     .group-info
@@ -15,18 +15,15 @@
           %span.visibility-icon.has-tooltip{ data: { container: 'body' }, title: visibility_icon_description(@group) }
             = visibility_level_icon(@group.visibility_level, fw: false)
 
-        %span.hidden-xs
-          = render 'shared/notifications/button', notification_setting: @notification_setting
-
+        .group-right-buttons.btn-group
           - if current_user
-          .pull-right
-            = render 'shared/members/access_request_buttons', source: @group
+            .pull-left.append-right-10= render 'shared/members/access_request_buttons', source: @group
+          = render 'shared/notifications/button', notification_setting: @notification_setting
 
       - if @group.description.present?
         .cover-desc.description
           = markdown(@group.description, pipeline: :description)
 
-
 %div{ class: container_class }
   .top-area
     %ul.nav-links

app/views/import/github/new.html.haml

+- page_title "GitHub Import"
+- header_title "Projects", root_path
+
+%h3.page-title
+  = icon 'github', text: 'Import Projects from GitHub'
+
+- if github_import_configured?
+  %p
+    To import a GitHub project, you first need to authorize GitLab to access
+    the list of your GitHub repositories:
+
+  = link_to 'List Your GitHub Repositories', status_import_github_path, class: 'btn btn-success'
+
+  %hr
+
+%p
+  - if github_import_configured?
+    Alternatively,
+  - else
+    To import a GitHub project,
+  you can use a
+  = succeed '.' do
+    = link_to 'Personal Access Token', 'https://github.com/settings/tokens'
+  When you create your Personal Access Token,
+  you will need to select the <code>repo</code> scope, so we can display a
+  list of your public and private repositories which are available for import.
+
+= form_tag personal_access_token_import_github_path, method: :post, class: 'form-inline' do
+  .form-group
+    = text_field_tag :personal_access_token, '', class: 'form-control', placeholder: "Personal Access Token", size: 40
+    = submit_tag 'List Your GitHub Repositories', class: 'btn btn-success'
+
+- unless github_import_configured?
+  %hr
+  %p
+    Note:
+    - if current_user.admin?
+      As an administrator you may like to configure
+    - else
+      Consider asking your GitLab administrator to configure
+    = link_to 'GitHub integration', help_page_path("integration", "github")
+    which will allow login via GitHub and allow importing projects without
+    generating a Personal Access Token.

app/views/layouts/_init_auto_complete.html.haml

@@ -3,4 +3,5 @@
 - if @noteable
   :javascript
     GitLab.GfmAutoComplete.dataSource = "#{autocomplete_sources_namespace_project_path(project.namespace, project, type: @noteable.class, type_id: params[:id])}"
+    GitLab.GfmAutoComplete.cachedData = undefined;
     GitLab.GfmAutoComplete.setup();

app/views/projects/_github_import_modal.html.haml

-%div#github_import_modal.modal
-  .modal-dialog
-    .modal-content
-      .modal-header
-        %a.close{href: "#", "data-dismiss" => "modal"} ×
-        %h3 Import projects from GitHub
-      .modal-body
-        To enable importing projects from GitHub,
-        - if current_user.admin?
-          as administrator you need to configure
-        - else
-          ask your Gitlab administrator to configure
-        == #{link_to 'OAuth integration', help_page_path("integration", "github")}.

app/views/projects/_home_panel.html.haml

 - empty_repo = @project.empty_repo?
 .project-home-panel.cover-block.clearfix{:class => ("empty-project" if empty_repo)}
-  %div{ class: (container_class) }
+  %div{ class: container_class }
     .row
       .project-image-container
         = project_icon(@project, alt: '', class: 'project-avatar avatar s70')
@@ -38,7 +38,8 @@
 
       .project-repo-buttons.btn-group.project-right-buttons
         - if current_user
-          = render 'shared/members/access_request_buttons', source: @project
+          .pull-left.append-right-10= render 'shared/members/access_request_buttons', source: @project
+
         = render "projects/buttons/update_mirror"
         = render "projects/buttons/download"
         = render 'projects/buttons/dropdown'

app/views/projects/_last_push.html.haml

 - if event = last_push_event
   - if show_last_push_widget?(event)
     .row-content-block.top-block.clear-block.hidden-xs
-      %div{ class: (container_class) }
+      %div{ class: container_class }
         .event-last-push
           .event-last-push-text
             %span You pushed to

app/views/projects/blob/show.html.haml

@@ -2,7 +2,7 @@
 - page_title @blob.path, @ref
 = render "projects/commits/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   = render 'projects/last_push'
 
   %div#tree-holder.tree-holder

app/views/projects/branches/index.html.haml

@@ -2,7 +2,7 @@
 - page_title "Branches"
 = render "projects/commits/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .top-area
     .nav-text
       Protected branches can be managed in project settings
@@ -30,7 +30,7 @@
 
   = render 'projects/commits/mirror_status'
 
-  - unless @branches.empty?
+  - if @branches.any?
     %ul.content-list.all-branches
       - @branches.each do |branch|
         = render "projects/branches/branch", branch: branch

app/views/projects/builds/index.html.haml

@@ -2,7 +2,7 @@
 - page_title "Builds"
 = render "projects/pipelines/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .top-area
     %ul.nav-links
       %li{class: ('active' if @scope.nil?)}

app/views/projects/builds/show.html.haml

@@ -67,4 +67,4 @@
 = render "sidebar"
 
 :javascript
-  new CiBuild("#{namespace_project_build_url(@project.namespace, @project, @build)}", "#{@build.status}", "#{trace_with_state[:state]}")
+  new CiBuild("#{namespace_project_build_url(@project.namespace, @project, @build, :json)}", "#{@build.status}", "#{trace_with_state[:state]}")

app/views/projects/commits/show.html.haml

@@ -7,7 +7,7 @@
 
 = render "head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .row-content-block.second-block.content-component-block
     .tree-ref-holder
       = render 'shared/ref_switcher', destination: 'commits'

app/views/projects/compare/index.html.haml

@@ -2,7 +2,7 @@
 - page_title "Compare"
 = render "projects/commits/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .sub-header-block
     Compare branches, tags or commit ranges.
     %br

app/views/projects/compare/show.html.haml

@@ -2,7 +2,7 @@
 - page_title "#{params[:from]}...#{params[:to]}"
 = render "projects/commits/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .sub-header-block.no-bottom-space
     = render "form"
 

app/views/projects/environments/index.html.haml

@@ -2,7 +2,7 @@
 - page_title "Environments"
 = render "projects/pipelines/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   - if can?(current_user, :create_environment, @project) && !@environments.blank?
     .top-area
       .nav-controls

app/views/projects/environments/show.html.haml

@@ -2,7 +2,7 @@
 - page_title "Environments"
 = render "projects/pipelines/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .top-area
     .col-md-9
       %h3.page-title= @environment.name.titleize

app/views/projects/graphs/ci.html.haml

@@ -2,7 +2,7 @@
 - page_title "Continuous Integration", "Graphs"
 = render 'head'
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .sub-header-block
     .oneline
       A collection of graphs for Continuous Integration

app/views/projects/graphs/commits.html.haml

@@ -2,7 +2,7 @@
 - page_title "Commits", "Graphs"
 = render 'head'
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .sub-header-block
     .tree-ref-holder
       = render 'shared/ref_switcher', destination: 'graphs_commits'

app/views/projects/graphs/languages.html.haml

@@ -2,7 +2,7 @@
 - page_title "Languages", "Graphs"
 = render 'head'
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .sub-header-block
     .oneline
       Programming languages used in this repository

app/views/projects/graphs/show.html.haml

@@ -2,7 +2,7 @@
 - page_title "Contributors", "Graphs"
 = render 'head'
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .sub-header-block
     .tree-ref-holder
       = render 'shared/ref_switcher', destination: 'graphs'

app/views/projects/imports/new.html.haml

@@ -10,7 +10,7 @@
     .panel-body
       %pre
         :preserve
-          #{sanitize_repo_path(@project.import_error)}
+          #{sanitize_repo_path(@project, @project.import_error)}
 
 = form_for @project, url: namespace_project_import_path(@project.namespace, @project), method: :post, html: { class: 'form-horizontal' } do |f|
   = render "shared/import_form", f: f

app/views/projects/issues/index.html.haml

@@ -6,7 +6,7 @@
   - if current_user
     = auto_discovery_link_tag(:atom, namespace_project_issues_url(@project.namespace, @project, :atom, private_token: current_user.private_token), title: "#{@project.name} issues")
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .top-area
     = render 'shared/issuable/nav', type: :issues
     .nav-controls

app/views/projects/labels/index.html.haml

@@ -3,7 +3,7 @@
 - hide_class = ''
 = render "projects/issues/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .top-area.adjust
     .nav-text
       Labels can be applied to issues and merge requests. Star a label to make it a priority label. Order the prioritized labels to change their relative priority, by dragging.

app/views/projects/merge_requests/index.html.haml

@@ -3,7 +3,7 @@
 = render "projects/issues/head"
 = render 'projects/last_push'
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .top-area
     = render 'shared/issuable/nav', type: :merge_requests
     .nav-controls

app/views/projects/milestones/index.html.haml

@@ -2,7 +2,7 @@
 - page_title "Milestones"
 = render "projects/issues/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .top-area
     = render 'shared/milestones_filter'
 

app/views/projects/network/_head.html.haml

 - @no_container = true
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .row-content-block.second-block.content-component-block
     .tree-ref-holder
       = render partial: 'shared/ref_switcher', locals: {destination: 'graph'}

app/views/projects/network/show.html.haml

@@ -4,7 +4,7 @@
   = page_specific_javascript_tag('network/application.js')
 = render "projects/commits/head"
 = render "head"
-%div{ class: (container_class) }
+%div{ class: container_class }
   .project-network
     .controls
       = form_tag namespace_project_network_path(@project.namespace, @project, @id), method: :get, class: 'form-inline network-form' do |f|

app/views/projects/new.html.haml

@@ -23,6 +23,7 @@
                   .input-group-addon
                     = root_url
                   = f.select :namespace_id, namespaces_options(params[:namespace_id] || :current_user, display_path: true), {}, {class: 'select2 js-select-namespace', tabindex: 1}
+
                 - else
                   .input-group-addon.static-namespace
                     #{root_url}#{current_user.username}/
@@ -44,15 +45,8 @@
               .col-sm-12.import-buttons
                 %div
                   - if github_import_enabled?
-                    - if github_import_configured?
-                      = link_to status_import_github_path, class: 'btn import_github' do
-                        %i.fa.fa-github
-                        GitHub
-                    - else
-                      = link_to '#', class: 'how_to_import_link btn import_github' do
-                        %i.fa.fa-github
-                        GitHub
-                      = render 'github_import_modal'
+                    = link_to new_import_github_path, class: 'btn import_github' do
+                      = icon 'github', text: 'GitHub'
                 %div
                   - if bitbucket_import_enabled?
                     - if bitbucket_import_configured?

app/views/projects/pipelines/index.html.haml

@@ -2,7 +2,7 @@
 - page_title "Pipelines"
 = render "projects/pipelines/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .top-area
     %ul.nav-links
       %li{class: ('active' if @scope.nil?)}

app/views/projects/show.html.haml

@@ -13,7 +13,7 @@
 = render "home_panel"
 
 .project-stats.row-content-block.second-block
-  %div{ class: (container_class) }
+  %div{ class: container_class }
     %ul.nav
       %li
         = link_to project_files_path(@project) do

app/views/projects/tags/index.html.haml

@@ -2,7 +2,7 @@
 - page_title "Tags"
 = render "projects/commits/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .top-area
     .nav-text
       Tags give the ability to mark specific points in history as being important
@@ -27,7 +27,7 @@
   = render 'projects/commits/mirror_status'
 
   .tags
-    - unless @tags.empty?
+    - if @tags.any?
       %ul.content-list
         = render partial: 'tag', collection: @tags
 

app/views/projects/tree/show.html.haml

@@ -7,7 +7,7 @@
 = render 'projects/last_push'
 = render "projects/commits/head"
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .tree-controls
     = render 'projects/find_file_link'
     - if can? current_user, :download_code, @project

app/views/projects/wikis/_new.html.haml

 - @no_container = true
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   %div#modal-new-wiki.modal
     .modal-dialog
       .modal-content

app/views/projects/wikis/edit.html.haml

@@ -2,7 +2,7 @@
 - page_title "Edit", @page.title.capitalize, "Wiki"
 = render 'nav'
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .top-area
     .nav-text
       %strong

app/views/projects/wikis/git_access.html.haml

@@ -2,7 +2,7 @@
 - page_title "Git Access", "Wiki"
 
 = render 'nav'
-%div{ class: (container_class) }
+%div{ class: container_class }
   .sub-header-block
     %span.oneline
       Git access for

app/views/projects/wikis/history.html.haml

 - page_title "History", @page.title.capitalize, "Wiki"
 = render 'nav'
-%div{ class: (container_class) }
+%div{ class: container_class }
   .top-area
     .nav-text
       %strong

app/views/projects/wikis/pages.html.haml

@@ -3,7 +3,7 @@
 
 = render 'nav'
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   %ul.content-list
     - @wiki_pages.each do |wiki_page|
       %li

app/views/projects/wikis/show.html.haml

@@ -2,7 +2,7 @@
 - page_title @page.title.capitalize, "Wiki"
 = render 'nav'
 
-%div{ class: (container_class) }
+%div{ class: container_class }
   .top-area
     .nav-text
       %strong= @page.title.capitalize

app/views/shared/members/_access_request_buttons.html.haml

@@ -7,8 +7,8 @@
       = link_to 'Withdraw Access Request', polymorphic_path([:leave, source, :members]),
                 method: :delete,
                 data: { confirm: remove_member_message(member) },
-                class: 'btn access-request-button hidden-xs'
+                class: 'btn'
   - else
     = link_to 'Request Access', polymorphic_path([:request_access, source, :members]),
               method: :post,
-              class: 'btn access-request-button hidden-xs'
+              class: 'btn'

app/workers/post_receive.rb

@@ -5,10 +5,10 @@ class PostReceive
   sidekiq_options queue: :post_receive
 
   def perform(repo_path, identifier, changes)
-    if repo_path.start_with?(Gitlab.config.gitlab_shell.repos_path.to_s)
-      repo_path.gsub!(Gitlab.config.gitlab_shell.repos_path.to_s, "")
+    if path = Gitlab.config.repositories.storages.find { |p| repo_path.start_with?(p[1].to_s) }
+      repo_path.gsub!(path[1].to_s, "")
     else
-      log("Check gitlab.yml config for correct gitlab_shell.repos_path variable. \"#{Gitlab.config.gitlab_shell.repos_path}\" does not match \"#{repo_path}\"")
+      log("Check gitlab.yml config for correct repositories.storages values. No repository storage path matches \"#{repo_path}\"")
     end
 
     post_received = Gitlab::GitPostReceive.new(repo_path, identifier, changes)

app/workers/repository_fork_worker.rb

@@ -12,7 +12,7 @@ class RepositoryForkWorker
       return
     end
 
-    result = gitlab_shell.fork_repository(source_path, target_path)
+    result = gitlab_shell.fork_repository(project.repository_storage_path, source_path, target_path)
     unless result
       logger.error("Unable to fork project #{project_id} for repository #{source_path} -> #{target_path}")
       project.mark_import_as_failed('The project could not be forked.')

config/dependency_decisions.yml

@@ -181,3 +181,9 @@
     :why: Equivalent to LGPLv2
     :versions: []
     :when: 2016-06-07 17:14:10.907682000 Z
+- - :whitelist
+  - Artistic 2.0
+  - :who: Josh Frye
+    :why: Disk/mount information display on Admin pages
+    :versions: []
+    :when: 2016-06-29 16:32:45.432113000 Z

config/environments/development.rb

@@ -42,4 +42,7 @@ Rails.application.configure do
   config.action_mailer.preview_path = 'spec/mailers/previews'
 
   config.eager_load = false
+
+  # Do not log asset requests
+  config.assets.quiet = true
 end

config/gitlab.teatro.yml

@@ -47,11 +47,13 @@ production: &base
   backup:
     path: "tmp/backups"   # Relative paths are relative to Rails.root (default: tmp/backups/)
 
+  repositories:
+    storages: # REPO PATHS MUST NOT BE A SYMLINK!!!
+      default: /apps/repositories/
+
   gitlab_shell:
     path: /apps/gitlab-shell/
 
-    # REPOS_PATH MUST NOT BE A SYMLINK!!!
-    repos_path: /apps/repositories/
     hooks_path: /apps/gitlab-shell/hooks/
 
     upload_pack: true

config/gitlab.yml.example

@@ -528,6 +528,13 @@ production: &base
   satellites:
     path: /home/git/gitlab-satellites/
 
+  ## Repositories settings
+  repositories:
+    # Paths where repositories can be stored. Give the canonicalized absolute pathname.
+    # NOTE: REPOS PATHS MUST NOT CONTAIN ANY SYMLINK!!!
+    storages: # You must have at least a `default` storage path.
+      default: /home/git/repositories/
+
   ## Backup settings
   backup:
     path: "tmp/backups"   # Relative paths are relative to Rails.root (default: tmp/backups/)
@@ -552,9 +559,6 @@ production: &base
   ## GitLab Shell settings
   gitlab_shell:
     path: /home/git/gitlab-shell/
-
-    # REPOS_PATH MUST NOT BE A SYMLINK!!!
-    repos_path: /home/git/repositories/
     hooks_path: /home/git/gitlab-shell/hooks/
 
     # File that contains the secret key for verifying access for gitlab-shell.
@@ -633,11 +637,13 @@ test:
     # user: YOUR_USERNAME
   satellites:
     path: tmp/tests/gitlab-satellites/
+  repositories:
+    storages:
+      default: tmp/tests/repositories/
   backup:
     path: tmp/tests/backups
   gitlab_shell:
     path: tmp/tests/gitlab-shell/
-    repos_path: tmp/tests/repositories/
     hooks_path: tmp/tests/gitlab-shell/hooks/
   issues_tracker:
     redmine:

config/initializers/1_settings.rb

@@ -382,7 +382,6 @@ Settings.gitlab_shell['hooks_path']   ||= Settings.gitlab['user_home'] + '/gitla
 Settings.gitlab_shell['secret_file'] ||= Rails.root.join('.gitlab_shell_secret')
 Settings.gitlab_shell['receive_pack']   = true if Settings.gitlab_shell['receive_pack'].nil?
 Settings.gitlab_shell['upload_pack']    = true if Settings.gitlab_shell['upload_pack'].nil?
-Settings.gitlab_shell['repos_path']   ||= Settings.gitlab['user_home'] + '/repositories/'
 Settings.gitlab_shell['ssh_host']     ||= Settings.gitlab.ssh_host
 Settings.gitlab_shell['ssh_port']     ||= 22
 Settings.gitlab_shell['ssh_user']     ||= Settings.gitlab.user
@@ -390,6 +389,14 @@ Settings.gitlab_shell['owner_group']  ||= Settings.gitlab.user
 Settings.gitlab_shell['ssh_path_prefix'] ||= Settings.send(:build_gitlab_shell_ssh_path_prefix)
 Settings.gitlab_shell['git_annex_enabled'] ||= false
 
+#
+# Repositories
+#
+Settings['repositories'] ||= Settingslogic.new({})
+Settings.repositories['storages'] ||= {}
+# Setting gitlab_shell.repos_path is DEPRECATED and WILL BE REMOVED in version 9.0
+Settings.repositories.storages['default'] ||= Settings.gitlab_shell['repos_path'] || Settings.gitlab['user_home'] + '/repositories/'
+
 #
 # Backup
 #

config/initializers/6_validations.rb

+def storage_name_valid?(name)
+  !!(name =~ /\A[a-zA-Z0-9\-_]+\z/)
+end
+
+def find_parent_path(name, path)
+  Gitlab.config.repositories.storages.detect do |n, p|
+    name != n && path.chomp('/').start_with?(p.chomp('/'))
+  end
+end
+
+def error(message)
+  raise "#{message}. Please fix this in your gitlab.yml before starting GitLab."
+end
+
+error('No repository storage path defined') if Gitlab.config.repositories.storages.empty?
+
+Gitlab.config.repositories.storages.each do |name, path|
+  error("\"#{name}\" is not a valid storage name") unless storage_name_valid?(name)
+
+  parent_name, _parent_path = find_parent_path(name, path)
+  if parent_name
+    error("#{name} is a nested path of #{parent_name}. Nested paths are not supported for repository storages")
+  end
+end

config/initializers/gitlab_shell_secret_token.rb

-# Be sure to restart your server when you modify this file.
-
-require 'securerandom'
-
-# Your secret key for verifying the gitlab_shell.
-
-
-secret_file = Gitlab.config.gitlab_shell.secret_file
-
-unless File.exist? secret_file
-  # Generate a new token of 16 random hexadecimal characters and store it in secret_file.
-  token = SecureRandom.hex(16)
-  File.write(secret_file, token)
-end
-
-link_path = File.join(Gitlab.config.gitlab_shell.path, '.gitlab_shell_secret')
-if File.exist?(Gitlab.config.gitlab_shell.path) && !File.exist?(link_path)
-  FileUtils.symlink(secret_file, link_path)
-end
+Gitlab::Shell.new.generate_and_link_secret_token

config/initializers/metrics.rb

@@ -132,6 +132,9 @@ if Gitlab::Metrics.enabled?
     config.instrument_instance_methods(API::Helpers)
 
     config.instrument_instance_methods(RepositoryCheck::SingleRepositoryWorker)
+
+    config.instrument_instance_methods(Rouge::Plugins::Redcarpet)
+    config.instrument_instance_methods(Rouge::Formatters::HTMLGitlab)
   end
 
   GC::Profiler.enable

config/initializers/rack_attack.rb.example

@@ -10,7 +10,8 @@ paths_to_be_protected = [
   "#{Rails.application.config.relative_url_root}/api/#{API::API.version}/session",
   "#{Rails.application.config.relative_url_root}/users",
   "#{Rails.application.config.relative_url_root}/users/confirmation",
-  "#{Rails.application.config.relative_url_root}/unsubscribes/"
+  "#{Rails.application.config.relative_url_root}/unsubscribes/",
+  "#{Rails.application.config.relative_url_root}/import/github/personal_access_token"
 
 ]
 

config/initializers/trusted_proxies.rb

+# Override Rack::Request to make use of the same list of trusted_proxies
+# as the ActionDispatch::Request object. This is necessary for libraries
+# like rack_attack where they don't use ActionDispatch, and we want them
+# to block/throttle requests on private networks.
+# Rack Attack specific issue: https://github.com/kickstarter/rack-attack/issues/145 
+module Rack
+  class Request
+    def trusted_proxy?(ip)
+      Rails.application.config.action_dispatch.trusted_proxies.any? { |proxy| proxy === ip }
+    end
+  end
+end
+
 Rails.application.config.action_dispatch.trusted_proxies = (
   [ '127.0.0.1', '::1' ] + Array(Gitlab.config.gitlab.trusted_proxies)
 ).map { |proxy| IPAddr.new(proxy) }

config/routes.rb

@@ -147,6 +147,7 @@ Rails.application.routes.draw do
   #
   namespace :import do
     resource :github, only: [:create, :new], controller: :github do
+      post :personal_access_token
       get :status
       get :callback
       get :jobs

db/migrate/20160608195742_add_repository_storage_to_projects.rb

+class AddRepositoryStorageToProjects < ActiveRecord::Migration
+  include Gitlab::Database::MigrationHelpers
+  disable_ddl_transaction!
+
+  def up
+    add_column_with_default(:projects, :repository_storage, :string, default: 'default')
+  end
+
+  def down
+    remove_column(:projects, :repository_storage)
+  end
+end