Merge remote-tracking branch 'origin/master' into ce-to-ee

Signed-off-by: Rémy Coutable <remy@rymai.me>

.gitlab-ci.yml

@@ -445,7 +445,7 @@ ee_compat_check:
   retry: 0
   artifacts:
     name: "${CI_JOB_NAME}_${CI_COMIT_REF_NAME}_${CI_COMMIT_SHA}"
-    when: on_failure
+    when: always
     expire_in: 10d
     paths:
       - ee_compat_check/patches/*.patch

CHANGELOG-EE.md

 Please view this file on the master branch, on stable branches it's out of date.
 
+## 10.2.0 (2017-11-22)
+
+### Fixed (17 changes)
+
+- Geo - Does not move projects backed by hashed storage when handling renamed events. !3066
+- Geo: Don't sync disabled project wikis. !3109
+- Reconfigure the Geo tracking database pool size when running as Sidekiq. !3181
+- Geo - Ensures that leases were returned. !3241
+- Fix (un)approver names not being shown in plaintext emails. !3266
+- Add post-migration to drain all Geo related redis queues. !3289
+- Prevent the Geo log cursor from running on primary nodes. !3411
+- Reduce the number of Elasticsearch client instances that are created. !3432
+- Fix generated clone URLs for wikis on Geo secondaries. !3448
+- Remove duplicate delete button in epic.
+- Fix: Failed to rebase MR from forked repo.
+- Fix: Geo API bug. Statistic is not collected when prometheus is disabled.
+- Geo - Ensure that repository deletions in a primary node are correctly deleted in a secondary node.
+- Geo: Fix handling of nil values on advanced section in admin screen.
+- Redirect to existing group boards using old URL if there is no subgroup called 'boards'.
+- Geo - Allow Sidekiq to retry failed jobs to rename project repositories.
+- Geo: Ensure database is connected before attempting to check for secondary status.
+
+### Changed (4 changes)
+
+- Add project actions in Audit events. !3160
+- Add group actions in Audit events. !3176
+- Geo: Don't retry repositories or files until everything has been backfilled. !3182
+- Improve Codeclimate UI.
+
+### Performance (1 change)
+
+- Reduce the quiet times between scheduler runs on Geo secondaries. !3185
+
+### Added (19 changes, 1 of them is from the community)
+
+- Make the maximum capacity of Geo backfill operations configurable. !3107
+- Mirrors can now hard fail, keeping them from being retried until a project admin takes action. !3117
+- View/edit epic at group level. !3126
+- Add worker to prune the Geo Event Log. !3172
+- julian7 Add required_groups option to SAML config, to restrict access to GitLab to specific SAML groups. !3223 (Balazs Nagy)
+- Geo: Expire and resync attachments from renamed projects in secondary nodes when using legacy storage. !3259
+- On Secondary read-only Geo Nodes now a flash banner is shown on all pages. !3260
+- Make GeoLogCursor Highly Available. !3305
+- Allow Geo repository sync over HTTPS. !3341
+- Allow persisting board configuration in order to automatically filter issues.
+- Improve error handling.
+- Add epics list and add epics to nav sidebar.
+- Introduce EEU lincese with epics as the first feature.
+- Add ability to create new epics.
+- Add sidebar for epic.
+- Add delete epic button.
+- Allow admins to globally disable all remote mirrors from application settings page.
+- Add support for logging Prometheus metrics for Geo.
+- Use PostgreSQL FDW for Geo downloads.
+
+### Other (2 changes, 1 of them is from the community)
+
+- Suppress MergeableSelector warning candidates in EE-only files. !3225 (Takuya Noguchi)
+- Enhance the documentation for gitlab-ctl replicate-geo-database. !3268
+
+
 ## 10.1.4 (2017-11-14)
 
 - No changes.

CHANGELOG.md

@@ -2,6 +2,194 @@
 documentation](doc/development/changelog.md) for instructions on adding your own
 entry.
 
+## 10.2.0 (2017-11-22)
+
+### Security (4 changes)
+
+- Upgrade Ruby to 2.3.5 to include security patches. !15099
+- Prevent OAuth phishing attack by presenting detailed wording about app to user during authorization.
+- Convert private tokens to Personal Access Tokens with sudo scope.
+- Remove private tokens from web interface and API.
+
+### Removed (5 changes)
+
+- Remove help text from group issues page and group merge requests page. !14963
+- Remove overzealous tooltips in projects page tabs. !15017
+- Stop merge requests from fetching their refs when the data is already available. !15129
+- Remove update merge request worker tagging.
+- Remove Session API now that private tokens are removed from user API endpoints.
+
+### Fixed (75 changes, 18 of them are from the community)
+
+- Fix 404 errors in API caused when the branch name had a dot. !14462 (gvieira37)
+- Remove unnecessary alt-texts from pipeline emails. !14602 (gernberg)
+- Renders 404 in commits controller if no commits are found for a given path. !14610 (Guilherme Vieira)
+- Cleanup data-page attribute after each Karma test. !14742
+- Removed extra border radius from .file-editor and .file-holder when editing a file. !14803 (Rachel Pipkin)
+- Add support for markdown preview to group milestones. !14806 (Vitaliy @blackst0ne Klachkov)
+- Fixed 'Removed source branch' checkbox in merge widget being ignored. !14832
+- Fix unnecessary ajax requests in admin broadcast message form. !14853
+- Make NamespaceSelect change URL when filtering. !14888
+- Get true failure from evalulate_script by checking for element beforehand. !14898
+- Fix SAML error 500 when no groups are defined for user. !14913
+- Fix 500 errors caused by empty diffs in some discussions. !14945 (Alexander Popov)
+- Fix the atom feed for group events. !14974
+- Hides pipeline duration in commit box when it is zero (nil). !14979 (gvieira37)
+- Add new diff discussions on MR diffs tab in "realtime". !14981
+- Returns a ssh url for go-get=1. !14990 (gvieira37)
+- Case insensitive search for branches. !14995 (George Andrinopoulos)
+- Fixes 404 error to 'Issues assigned to me' and 'Issues I've created' when issues are disabled. !15021 (Jacopo Beschi @jacopo-beschi)
+- Update the groups API documentation. !15024 (Robert Schilling)
+- Validate username/pw for Jiraservice, require them in the API. !15025 (Robert Schilling)
+- Update Merge Request polling so there is only one request at a time. !15032
+- Use project select dropdown not only as a combobutton. !15043
+- Remove create MR button from issues when MRs are disabled. !15071 (George Andrinopoulos)
+- Tighten up whitelisting of certain Geo routes. !15082
+- Allow to disable the Performance Bar. !15084
+- Refresh open Issue and Merge Request project counter caches when re-opening. !15085 (Rob Ede @robjtede)
+- Fix markdown form tabs toggling preview mode from double clicking write mode button. !15119
+- Fix cancel button not working while uploading on the new issue page. !15137
+- Fix webhooks recent deliveries. !15146 (Alexander Randa (@randaalex))
+- Fix issues with forked projects of which the source was deleted. !15150
+- Fix GPG signature popup info in Safari and Firefox. !15228
+- Fix GFM reference links for closed milestones. !15234 (Vitaliy @blackst0ne Klachkov)
+- When deleting merged branches, ignore protected tags. !15252
+- Revert a regression on runners sorting (!15134). !15341 (Takuya Noguchi)
+- Don't use JS to delete memberships from projects and groups. !15344
+- Don't try to create fork network memberships for forks with a missing source. !15366
+- Fix gitlab:backup rake for hashed storage based repositories. !15400
+- Fix issue where clicking a GPG verification badge would scroll to the top of the page. !15407
+- Update container repository path reference and allow using double underscore. !15417
+- Fix crash when navigating to second page of the group dashbaord when there are projects and groups on the first page. !15456
+- Fix flash errors showing up on a non configured prometheus integration. !35652
+- Fix arguments Import/Export error importing project merge requests.
+- Moves mini graph of pipeline to the end of sentence in MR widget. Cleans HTML and tests.
+- Fix user autocomplete in subgroups.
+- Fixed user profile activity tab being off-screen on mobile.
+- Fix diff parser so it tolerates to diff special markers in the content.
+- Fix a migration that adds merge_requests_ff_only_enabled column to MR table.
+- Don't create build failed todos when the job is automatically retried.
+- Render 404 when polling commit notes without having permissions.
+- Show error message when fast-forward merge is not possible.
+- Prevents position update for image diff notes.
+- Mobile-friendly table on Admin Runners. (Takuya Noguchi)
+- Decreases z-index of select2 to a lower number of our navigation bar.
+- Fix broken Members link when relative URL root paths are used.
+- Avoid regenerating the ref path for the environment.
+- Memoize GitLab logger to reduce open file descriptors.
+- Fix hashed storage with project transfers to another namespace.
+- Fix bad type checking to prevent 0 count badge to be shown.
+- Fix problem with issuable header wrapping when content is too long.
+- Move retry button in job page to sidebar.
+- Formats bytes to human reabale number in registry table.
+- Fix commit pipeline showing wrong status.
+- Include link to issue in reopen message for Slack and Mattermost notifications.
+- Fix double border UI bug on pipelines/environments table and pagination.
+- Remove native title tooltip in pipeline jobs dropdown in Safari.
+- Fix namespacing for MergeWhenPipelineSucceedsService in MR API.
+- Prevent error when authorizing an admin-created OAauth application without a set owner.
+- Always return full avatar URL for private/internal groups/projects when asset host is set.
+- Make sure group and project creation is blocked for new users that are external by default.
+- Make sure NotesActions#noteable returns a Noteable in the update action.
+- Reallow project paths ending in periods.
+- Only set Auto-Submitted header once for emails on push.
+- Fix overlap of right-sidebar and main content when creating a Wiki page.
+- Enables scroll to bottom once user has scrolled back to bottom in job log.
+- Fix timezone bug in Pikaday and upgrade Pikaday version.
+
+### Changed (21 changes, 7 of them are from the community)
+
+- Added possibility to enter past date in /spend command to log time in the past. !3044 (g3dinua, LockiStrike)
+- Add Prometheus equivalent of all InfluxDB metrics. !13891
+- Show collapsible project lists. !14055
+- Make Prometheus metrics endpoint return empty response when metrics are disabled. !14490
+- Support custom attributes on groups and projects. !14593 (Markus Koller)
+- Avoid fetching all branches for branch existence checks. !14778
+- Update participants and subscriptions button in issuable sidebar to be async. !14836
+- Replace WikiPage::CreateService calls with wiki_page factory in specs. !14850 (Jacopo Beschi @jacopo-beschi)
+- Add lazy option to UserAvatarImage. !14895
+- Add readme only option as project view. !14900
+- Todos spelled correctly on Todos list page. !15015
+- Support uml:: and captions in reStructuredText. !15120 (Markus Koller)
+- Add system hooks user_rename and group_rename. !15123
+- Change tags order in refs dropdown. !15235 (Vitaliy @blackst0ne Klachkov)
+- Change default cluster size to n1-default-2. !39649 (Fabio Busatto)
+- Change 'Sign Out' route from a DELETE to a GET. !39708 (Joe Marty)
+- Change background color of nav sidebar to match other gl sidebars.
+- Update i18n section in FE docs for marking and interpolation.
+- Add a count of changes to the merge requests API.
+- Improve GitLab Import rake task to work with Hashed Storage and Subgroups.
+- 14830 Move GitLab export option to top of import list when creating a new project.
+
+### Performance (14 changes)
+
+- Improve branch listing page performance. !14729
+- Improve DashboardController#activity.json performance. !14985
+- Add a latest_merge_request_diff_id column to merge_requests. !15035
+- Improve performance of the /projects/:id/repository/branches API endpoint. !15215
+- Ensure merge requests with lots of version don't time out when searching for pipelines.
+- Speed up issues list APIs.
+- Remove Filesystem check metrics that use too much CPU to handle requests.
+- Disable Unicorn sampling in Sidekiq since there are no Unicorn sockets to monitor.
+- Truncate tree to max 1,000 items and display notice to users.
+- Add Performance improvement as category on the changelog.
+- Cache commits fetched from the repository.
+- Cache the number of user SSH keys.
+- Optimise getting the pipeline status of commits.
+- Improve performance of commits list by fully using DB index when getting commit note counts.
+
+### Added (27 changes, 10 of them are from the community)
+
+- Add new push rule to enforce that only the author of a commit can push to the repository. !3086
+- Expose duration in Job entity. !13644 (Mehdi Lahmam (@mehlah))
+- Prevent git push when LFS objects are missing. !13837
+- Automatic configuration settings page. !13850 (Francisco Lopez)
+- Add API endpoints for Pages Domains. !13917 (Travis Miller)
+- Include the changes in issuable webhook payloads. !14308
+- Add Packagist project service. !14493 (Matt Coleman)
+- Add sort runners on admin runners. !14661 (Takuya Noguchi)
+- Repo Editor: Add option to start a new MR directly from comit section. !14665
+- Issue JWT token with registry:catalog:* scope when requested by GitLab admin. !14751 (Vratislav Kalenda)
+- Support show-all-refs for git over HTTP. !14834
+- Add loading button for new UX paradigm. !14883
+- Get Project Branch API shows an helpful error message on invalid refname. !14884 (Jacopo Beschi @jacopo-beschi)
+- Refactor have_http_status into have_gitlab_http_status. !14958 (Jacopo Beschi @jacopo-beschi)
+- Suggest to rename the remote for existing repository instructions. !14970 (helmo42)
+- Adds project_id to pipeline hook data. !15044 (Jacopo Beschi @jacopo-beschi)
+- Hashed Storage support for Attachments. !15068
+- Add metric tagging for sidekiq workers. !15111
+- Expose project visibility as CI variable - CI_PROJECT_VISIBILITY. !15193
+- Allow multiple queries in a single Prometheus graph to support additional environments (Canary, Staging, et al.). !15201
+- Allow promoting project milestones to group milestones.
+- Added submodule support in multi-file editor.
+- Add applications section to GKE clusters page to easily install Helm Tiller, Ingress.
+- Allow files to uploaded in the multi-file editor.
+- Add Ingress to available Cluster applications.
+- Adds typescript support.
+- Add sudo scope for OAuth and Personal Access Tokens to be used by admins to impersonate other users on the API.
+
+### Other (18 changes, 8 of them are from the community)
+
+- Decrease Perceived Complexity threshold to 14. !14231 (Maxim Rydkin)
+- Replace the 'features/explore/projects.feature' spinach test with an rspec analog. !14755 (Vitaliy @blackst0ne Klachkov)
+- While displaying a commit, do not show list of related branches if there are thousands of branches. !14812
+- Removed d3.js from the graph and users bundles and used the common_d3 bundle instead. !14826
+- Make contributors page translatable. !14915
+- Decrease ABC threshold to 54.28. !14920 (Maxim Rydkin)
+- Clarify system_hook triggers in documentation. !14957 (Joe Marty)
+- Free up some reserved group names. !15052
+- Bump carrierwave to 1.2.1. !15072 (Takuya Noguchi)
+- Enable NestingDepth (level 6) on scss-lint. !15073 (Takuya Noguchi)
+- Enable BorderZero rule in scss-lint. !15168 (Takuya Noguchi)
+- Internationalized tags page. !38589
+- Moves placeholders components into shared folder with documentation. Makes them easier to reuse in MR and Snippets comments.
+- Reorganize welcome page for new users.
+- Refactor GroupLinksController. (15121)
+- Remove filter icon from search bar.
+- Use title as placeholder instead of issue title for reusability.
+- Add Gitaly metrics to the performance bar.
+
+
 ## 10.1.4 (2017-11-14)
 
 ### Fixed (4 changes)

app/controllers/admin/application_controller.rb

@@ -3,23 +3,9 @@
 # Automatically sets the layout and ensures an administrator is logged in
 class Admin::ApplicationController < ApplicationController
   before_action :authenticate_admin!
-  before_action :display_read_only_information
   layout 'admin'
 
   def authenticate_admin!
     render_404 unless current_user.admin?
   end
-
-  def display_read_only_information
-    return unless Gitlab::Database.read_only?
-
-    flash.now[:notice] = read_only_message
-  end
-
-  private
-
-  # Overridden in EE
-  def read_only_message
-    _('You are on a read-only GitLab instance.')
-  end
 end

app/models/note.rb

@@ -114,6 +114,7 @@ class Note < ActiveRecord::Base
     includes(:author, :noteable, :updated_by,
              project: [:project_members, { group: [:group_members] }])
   end
+  scope :with_metadata, -> { includes(:system_note_metadata) }
 
   after_initialize :ensure_discussion_id
   before_validation :nullify_blank_type, :nullify_blank_line_code
@@ -177,7 +178,13 @@ class Note < ActiveRecord::Base
   end
 
   def cross_reference?
-    system? && matches_cross_reference_regex?
+    return unless system?
+
+    if force_cross_reference_regex_check?
+      matches_cross_reference_regex?
+    else
+      SystemNoteService.cross_reference?(note)
+    end
   end
 
   def diff_note?
@@ -390,4 +397,10 @@ class Note < ActiveRecord::Base
   def set_discussion_id
     self.discussion_id ||= discussion_class.discussion_id(self)
   end
+
+  def force_cross_reference_regex_check?
+    return unless system?
+
+    SystemNoteMetadata::TYPES_WITH_CROSS_REFERENCES.include?(system_note_metadata&.action)
+  end
 end

app/models/system_note_metadata.rb

 class SystemNoteMetadata < ActiveRecord::Base
+  # These notes's action text might contain a reference that is external.
+  # We should always force a deep validation upon references that are found
+  # in this note type.
+  # Other notes can always be safely shown as all its references are
+  # in the same project (i.e. with the same permissions)
+  TYPES_WITH_CROSS_REFERENCES = %w[
+    commit cross_reference
+    close duplicate
+    relate unrelate
+  ].freeze
+
   ICON_TYPES = %w[
     commit description merge confidential visible label assignee cross_reference
     title time_tracking branch milestone discussion task moved

app/services/system_note_service.rb

@@ -637,6 +637,10 @@ module SystemNoteService
     create_note(NoteSummary.new(issuable, issuable.project, author, body, action: action))
   end
 
+  def cross_reference?(note_text)
+    note_text =~ /\A#{cross_reference_note_prefix}/i
+  end
+
   private
 
   def notes_for_mentioner(mentioner, noteable, notes)

changelogs/unreleased-ee/1144-allow-admins-to-disable-push-mirrors.yml

----
-title: Allow admins to globally disable all remote mirrors from application settings
-  page.
-merge_request:
-author:
-type: added

changelogs/unreleased-ee/1371-audit-project-events.yml

----
-title: Add project actions in Audit events
-merge_request: 3160
-author:
-type: changed

changelogs/unreleased-ee/1372-audit-group-events.yml

----
-title: Add group actions in Audit events
-merge_request: 3176
-author:
-type: changed

changelogs/unreleased-ee/2518-persisted-issue-boards-filter-be.yml

----
-title: Allow persisting board configuration in order to automatically filter issues
-merge_request:
-author:
-type: added

changelogs/unreleased-ee/2524-geo-secondary-help-users-not-waste-time-on-impossible-operations.yml

----
-title: On Secondary read-only Geo Nodes now a flash banner is shown on all pages
-merge_request: 3260
-author:
-type: added

changelogs/unreleased-ee/2548-improve-codeclimate.yml

----
-title: Improve Codeclimate UI
-merge_request:
-author:
-type: changed

changelogs/unreleased-ee/2831-hard-fail-mirror-when-retry-count-number-exceeds-the-limit.yml

----
-title: Mirrors can now hard fail, keeping them from being retried until a project
-  admin takes action.
-merge_request: 3117
-author:
-type: added

changelogs/unreleased-ee/3119-improve-error-recovery.yml

----
-title: Improve error handling
-merge_request:
-author:
-type: added

changelogs/unreleased-ee/3143-saml_required_groups.yml

----
-title: julian7 Add required_groups option to SAML config, to restrict access to GitLab
-  to specific SAML groups.
-merge_request: 3223
-author: Balazs Nagy
-type: added

changelogs/unreleased-ee/3341-geo-http-repo-sync.yml

----
-title: Allow Geo repository sync over HTTPS
-merge_request: 3341
-author:
-type: added

changelogs/unreleased-ee/3508-hashed-storage-sends-a-geo-repository-renamed-event.yml

----
-title: Geo - Does not move projects backed by hashed storage when handling renamed events
-merge_request: 3066
-author:
-type: fixed

changelogs/unreleased-ee/3532-configurable-geo-max-capacity.yml

----
-title: Make the maximum capacity of Geo backfill operations configurable
-merge_request: 3107
-author:
-type: added

changelogs/unreleased-ee/3550-show-epic.yml

----
-title: View/edit epic at group level
-merge_request: 3126
-author:
-type: added

changelogs/unreleased-ee/3553-epics-list.yml

----
-title: Add epics list and add epics to nav sidebar
-merge_request:
-author:
-type: added

changelogs/unreleased-ee/3568-restarting-sidekiq-does-not-cause-leases-to-be-returned.yml

----
-title: Geo - Ensures that leases were returned.
-merge_request: 3241
-author:
-type: fixed

changelogs/unreleased-ee/3569-geo-backfill-unconditional-wiki.yml

----
-title: 'Geo: Don''t sync disabled project wikis'
-merge_request: 3109
-author:
-type: fixed

changelogs/unreleased-ee/3650-persist-elasticsearch-client.yml

----
-title: Reduce the number of Elasticsearch client instances that are created
-merge_request: 3432
-author:
-type: fixed

changelogs/unreleased-ee/3674-attachments-legacy-storage-rename.yml

----
-title: 'Geo: Expire and resync attachments from renamed projects in secondary nodes
-  when using legacy storage'
-merge_request: 3259
-author:
-type: added

changelogs/unreleased-ee/3731-eeu-license.yml

----
-title: Introduce EEU lincese with epics as the first feature
-merge_request:
-author:
-type: added

changelogs/unreleased-ee/3809-geo-sidekiq-pool.yml

----
-title: Reconfigure the Geo tracking database pool size when running as Sidekiq
-merge_request: 3181
-author:
-type: fixed

changelogs/unreleased-ee/3810-geo-disable-retries.yml

----
-title: 'Geo: Don''t retry repositories or files until everything has been backfilled'
-merge_request: 3182
-author:
-type: changed

changelogs/unreleased-ee/3813-reduce-geo-scheduler-quiet-times.yml

----
-title: Reduce the quiet times between scheduler runs on Geo secondaries
-merge_request: 3185
-author:
-type: performance

changelogs/unreleased-ee/3877-expand-replicate-geo-database-docs.yml

----
-title: Enhance the documentation for gitlab-ctl replicate-geo-database
-merge_request: 3268
-author:
-type: other

changelogs/unreleased-ee/3879-suppress-mergeable-selector-in-ee-only-files.yml

----
-title: Suppress MergeableSelector warning candidates in EE-only files
-merge_request: 3225
-author: Takuya Noguchi
-type: other

changelogs/unreleased-ee/3924-approver-emails-text.yml

----
-title: Fix (un)approver names not being shown in plaintext emails
-merge_request: 3266
-author:
-type: fixed

changelogs/unreleased-ee/3937-new-epic.yml

----
-title: Add ability to create new epics
-merge_request:
-author:
-type: added

changelogs/unreleased-ee/39977-git-timeout-for-geo-doc.yml

+---
+title: Document a failure mode for large repositories in Geo
+merge_request: 3500
+author:
+type: other

changelogs/unreleased-ee/4049-log-cursor-runs-on-primary.yml

----
-title: Prevent the Geo log cursor from running on primary nodes
-merge_request: 3411
-author:
-type: fixed

changelogs/unreleased-ee/4054-fix-geo-wiki-clone-instructions.yml

----
-title: Fix generated clone URLs for wikis on Geo secondaries
-merge_request: 3448
-author:
-type: fixed

changelogs/unreleased-ee/4055-geo-api-bugs.yml

----
-title: 'Fix: Geo API bug. Statistic is not collected when prometheus is disabled'
-merge_request:
-author:
-type: fixed

changelogs/unreleased/27654-retry-button.yml → changelogs/unreleased-ee/4114-epic-list-fix.yml

 ---
-title: Move retry button in job page to sidebar
+title: Don't user issuable_sort cookie for epics collection
 merge_request:
 author:
 type: fixed

changelogs/unreleased/fix-subgroup-autocomplete.yml → changelogs/unreleased-ee/4115-enable-scoped-boards-for-early-adopters.yml

 ---
-title: Fix user autocomplete in subgroups
+title: Enable scoped boards for Early Adopters
 merge_request:
 author:
 type: fixed

changelogs/unreleased-ee/add-epic-sidebar.yml

----
-title: Add sidebar for epic
-merge_request:
-author:
-type: added

changelogs/unreleased-ee/da-ensure-repository-deletions-are-communicated-to-secondary.yml

----
-title: Geo - Ensure that repository deletions in a primary node are correctly deleted
-  in a secondary node
-merge_request:
-author:
-type: fixed

changelogs/unreleased-ee/dba-fix-nil-status-count.yml

----
-title: 'Geo: Fix handling of nil values on advanced section in admin screen'
-merge_request:
-author:
-type: fixed

changelogs/unreleased-ee/delete-epic.yml

----
-title: Add delete epic button
-merge_request:
-author:
-type: added

changelogs/unreleased-ee/feature-ha-geologcursor.yml

----
-title: Make GeoLogCursor Highly Available
-merge_request: 3305
-author:
-type: added

changelogs/unreleased-ee/fix-group-boards-redirect.yml

----
-title: Redirect to existing group boards using old URL if there is no subgroup called
-  'boards'
-merge_request:
-author:
-type: fixed

changelogs/unreleased-ee/refactor-geo-move-repository-service.yml

----
-title: Geo - Allow Sidekiq to retry failed jobs to rename project repositories
-merge_request:
-author:
-type: fixed

changelogs/unreleased-ee/remove-duplicate-delete-btn.yml

----
-title: Remove duplicate delete button in epic
-merge_request:
-author:
-type: fixed

changelogs/unreleased-ee/sh-fix-geo-secondary-check.yml

----
-title: 'Geo: Ensure database is connected before attempting to check for secondary
-  status'
-merge_request:
-author:
-type: fixed

changelogs/unreleased-ee/sh-geo-add-prom-metrics.yml

----
-title: Add support for logging Prometheus metrics for Geo
-merge_request: !3187
-author:
-type: added

changelogs/unreleased-ee/sh-rails-fdw-support.yml

----
-title: Use PostgreSQL FDW for Geo downloads
-merge_request:
-author:
-type: added

changelogs/unreleased-ee/tc-geo-drain-queues.yml

----
-title: Add post-migration to drain all Geo related redis queues
-merge_request: 3289
-author:
-type: fixed

changelogs/unreleased-ee/tc-geo-prune-event-log.yml

----
-title: Add worker to prune the Geo Event Log
-merge_request: 3172
-author:
-type: added

changelogs/unreleased-ee/tc-geo-remove-double-banner.yml

+---
+title: Remove duplicate read-only flash message on admin pages
+merge_request: 3495
+author:
+type: fixed

changelogs/unreleased/.yml

----
-title: Remove update merge request worker tagging.
-merge_request:
-author:
-type: removed

changelogs/unreleased/1312-time-spent-at.yml

----
-title: Added possibility to enter past date in /spend command to log time in the past
-merge_request: 3044
-author: g3dinua, LockiStrike
-type: changed

changelogs/unreleased/14970-suggest-rename-remote.yml

----
-title: Suggest to rename the remote for existing repository instructions
-merge_request: 14970
-author: helmo42
-type: added

changelogs/unreleased/20666-404-error-issue-assigned-with-issues-disabled.yml

----
-title: Fixes 404 error to 'Issues assigned to me' and 'Issues I've created' when issues
-  are disabled
-merge_request: 15021
-author: Jacopo Beschi @jacopo-beschi
-type: fixed

changelogs/unreleased/23000-pages-api.yml

----
-title: Add API endpoints for Pages Domains
-merge_request: 13917
-author: Travis Miller
-type: added

changelogs/unreleased/23206-load-participants-async.yml

----
-title: Update participants and subscriptions button in issuable sidebar to be async
-merge_request: 14836
-author:
-type: changed

changelogs/unreleased/26763-grant-registry-auth-scope-to-admins.yml

----
-title: Issue JWT token with registry:catalog:* scope when requested by GitLab admin
-merge_request: 14751
-author: Vratislav Kalenda
-type: added

changelogs/unreleased/27375-dashboard-activity-performance.yml

----
-title: Improve DashboardController#activity.json performance
-merge_request: 14985
-author:
-type: performance

changelogs/unreleased/28202_decrease_abc_threshold_step5.yml

----
-title: Decrease ABC threshold to 54.28
-merge_request: 14920
-author: Maxim Rydkin
-type: other

changelogs/unreleased/30140-restore-readme-only-preference.yml

----
-title: Add readme only option as project view
-merge_request: 14900
-author:
-type: changed

changelogs/unreleased/31358_decrease_perceived_complexity_threshold_step3.yml

----
-title: Decrease Perceived Complexity threshold to 14
-merge_request: 14231
-author: Maxim Rydkin
-type: other

changelogs/unreleased/31454-missing-project-id-pipeline-hook-data.yml

----
-title: Adds project_id to pipeline hook data
-merge_request: 15044
-author: Jacopo Beschi @jacopo-beschi
-type: added

changelogs/unreleased/32318-filter-icon.yml

----
-title: Remove filter icon from search bar
-merge_request:
-author:
-type: other

changelogs/unreleased/3274-geo-route-whitelisting.yml

----
-title: Tighten up whitelisting of certain Geo routes
-merge_request: 15082
-author:
-type: fixed

changelogs/unreleased/34284-add-changes-to-issuable-webhook-data.yml

----
-title: Include the changes in issuable webhook payloads
-merge_request: 14308
-author:
-type: added

changelogs/unreleased/34768-fix-issuable-header-wrapping.yml

----
-title: Fix problem with issuable header wrapping when content is too long
-merge_request:
-author:
-type: fixed

changelogs/unreleased/34841-todos.yml

----
-title: Fix bad type checking to prevent 0 count badge to be shown
-merge_request:
-author:
-type: fixed

changelogs/unreleased/34897-delete-branch-after-merge.yml

----
-title: Fixed 'Removed source branch' checkbox in merge widget being ignored.
-merge_request: 14832
-author:
-type: fixed

changelogs/unreleased/35199-case-insensitive-branches-search.yml

----
-title: Case insensitive search for branches
-merge_request: 14995
-author: George Andrinopoulos
-type: fixed

changelogs/unreleased/35644-refactor-have-http-status-into-have-gitlab-http-status.yml

----
-title: Refactor have_http_status into have_gitlab_http_status
-merge_request: 14958
-author: Jacopo Beschi @jacopo-beschi
-type: added

changelogs/unreleased/35652-prometheus-service-page-shows-error.yml

----
-title: Fix flash errors showing up on a non configured prometheus integration
-merge_request: 35652
-author:
-type: fixed

changelogs/unreleased/35914-merge-request-update-worker-is-slow.yml

----
-title: Add metric tagging for sidekiq workers
-merge_request: 15111
-author:
-type: added

changelogs/unreleased/3615-improve-welcome-screen.yml

----
-title: Reorganize welcome page for new users
-merge_request:
-author:
-type: other

changelogs/unreleased/36160-zindex.yml

----
-title: Decreases z-index of select2 to a lower number of our navigation bar
-merge_request:
-author:
-type: fixed

changelogs/unreleased/36629-35958-add-cluster-application-section.yml

----
-title: Add applications section to GKE clusters page to easily install Helm Tiller,
-  Ingress
-merge_request:
-author:
-type: added

changelogs/unreleased/3674-hashed-storage-attachments.yml

----
-title: Hashed Storage support for Attachments
-merge_request: 15068
-author:
-type: added

changelogs/unreleased/37032-get-project-branch-invalid-name-message.yml

----
-title: Get Project Branch API shows an helpful error message on invalid refname
-merge_request: 14884
-author: Jacopo Beschi @jacopo-beschi
-type: added

changelogs/unreleased/37442-api-branches-id-repository-branches-is-calling-gitaly-n-1-times-per-request.yml

----
-title: Improve performance of the /projects/:id/repository/branches API endpoint
-merge_request: 15215
-author:
-type: performance

changelogs/unreleased/37473-expose-project-visibility-as-ci-variable.yml

----
-title: Expose project visibility as CI variable - CI_PROJECT_VISIBILITY
-merge_request: 15193
-author:
-type: added

changelogs/unreleased/37571-replace-wikipage-createservice-with-factory.yml

----
-title: Replace WikiPage::CreateService calls with wiki_page factory in specs
-merge_request: 14850
-author: Jacopo Beschi @jacopo-beschi
-type: changed

changelogs/unreleased/37631-add-a-merge_request_diff_id-column-to-merge_requests.yml

----
-title: Add a latest_merge_request_diff_id column to merge_requests
-merge_request: 15035
-author:
-type: performance

changelogs/unreleased/37660-match-sidebar-colors.yml

----
-title: Change background color of nav sidebar to match other gl sidebars
-merge_request:
-author:
-type: changed

changelogs/unreleased/37824-many-branches-lock-server.yml

----
-title: While displaying a commit, do not show list of related branches if there are
-  thousands of branches
-merge_request: 14812
-author:
-type: other

changelogs/unreleased/37978-extra-border-radius-while-editing-a-file.yml

----
-title: Removed extra border radius from .file-editor and .file-holder when editing
-  a file
-merge_request: 14803
-author: Rachel Pipkin
-type: fixed

changelogs/unreleased/38178-fl-mr-notes-components.yml

----
-title: Moves placeholders components into shared folder with documentation. Makes
-  them easier to reuse in MR and Snippets comments
-merge_request:
-author:
-type: other

changelogs/unreleased/38236-remove-build-failed-todo-if-it-has-been-auto-retried.yml

----
-title: Don't create build failed todos when the job is automatically retried
-merge_request:
-author:
-type: fixed

changelogs/unreleased/38247-hide-create-mr-button-in-issue-show.yml

----
-title: Remove create MR button from issues when MRs are disabled
-merge_request: 15071
-author: George Andrinopoulos
-type: fixed

changelogs/unreleased/38394-smarter-interval.yml

----
-title: Update Merge Request polling so there is only one request at a time
-merge_request: 15032
-author:
-type: fixed

changelogs/unreleased/38395-mr-widget-ci.yml

----
-title: Moves mini graph of pipeline to the end of sentence in MR widget. Cleans HTML
-  and tests
-merge_request:
-author:
-type: fixed

changelogs/unreleased/38589-internationalize-tags-page.yml

----
-title: Internationalized tags page
-merge_request: 38589
-author:
-type: other

changelogs/unreleased/38677-render-new-discussions-on-diff-tab.yml

----
-title: Add new diff discussions on MR diffs tab in "realtime"
-merge_request: 14981
-author:
-type: fixed

changelogs/unreleased/38720-sort-admin-runners.yml

----
-title: Add sort runners on admin runners
-merge_request: 14661
-author: Takuya Noguchi
-type: added

changelogs/unreleased/38871-cleanup-data-page-attribute-after-karma-test.yml

----
-title: Cleanup data-page attribute after each Karma test
-merge_request: 14742
-author:
-type: fixed

changelogs/unreleased/38986-due-date.yml

----
-title: Fix timezone bug in Pikaday and upgrade Pikaday version
-merge_request:
-author:
-type: fixed

changelogs/unreleased/39033-d3-js-is-being-included-in-the-user_profile-and-graphs_show-bundles.yml

----
-title: Removed d3.js from the graph and users bundles and used the common_d3 bundle
-  instead
-merge_request: 14826
-author:
-type: other

changelogs/unreleased/39035-move-gitlab-export-to-top-import-list.yml

----
-title: 14830 Move GitLab export option to top of import list when creating a new project
-merge_request:
-author:
-type: changed

changelogs/unreleased/39109-reenable-scroll-job.yml

----
-title: Enables scroll to bottom once user has scrolled back to bottom in job log
-merge_request:
-author:
-type: fixed

changelogs/unreleased/39297-remove-help-text-group-lists.yml

----
-title: Remove help text from group issues page and group merge requests page
-merge_request: 14963
-author:
-type: removed

changelogs/unreleased/39417-todos-spelled-correctly-on-todos-list-page.yml

----
-title: Todos spelled correctly on Todos list page
-merge_request: 15015
-author:
-type: changed

changelogs/unreleased/39419-remove-overzealous-tooltips.yml

----
-title: Remove overzealous tooltips in projects page tabs
-merge_request: 15017
-author:
-type: removed

changelogs/unreleased/39509-fix-wiki-create-sidebar-overlap.yml

----
-title: Fix overlap of right-sidebar and main content when creating a Wiki page
-merge_request:
-author:
-type: fixed

changelogs/unreleased/39570-performance-bar-appears-enabled-even-though-it-won-t-show-up.yml

----
-title: Allow to disable the Performance Bar
-merge_request: 15084
-author:
-type: fixed

changelogs/unreleased/39580-bump-carrierwave-to-1-2-1.yml

----
-title: Bump carrierwave to 1.2.1
-merge_request: 15072
-author: Takuya Noguchi
-type: other

changelogs/unreleased/39582-nestingdepth-6.yml

----
-title: Enable NestingDepth (level 6) on scss-lint
-merge_request: 15073
-author: Takuya Noguchi
-type: other

changelogs/unreleased/39583-reopen-issue-count-cache.yml

----
-title: Refresh open Issue and Merge Request project counter caches when re-opening.
-merge_request: 15085
-author: Rob Ede @robjtede
-type: fixed

changelogs/unreleased/39593-emails-on-push-are-sent-to-only-the-first-recipient-when-using-aws-ses.yml

----
-title: Only set Auto-Submitted header once for emails on push
-merge_request:
-author:
-type: fixed

changelogs/unreleased/39619-cancel-merge-when-pipeline-succeeds-from-the-api-fails.yml

----
-title: Fix namespacing for MergeWhenPipelineSucceedsService in MR API
-merge_request:
-author:
-type: fixed

changelogs/unreleased/39649-change-default-size-for-gke-cluster-creation.yml

----
-title: Change default cluster size to n1-default-2
-merge_request: 39649
-author: Fabio Busatto
-type: changed

changelogs/unreleased/39668-tooltip-safari.yml

----
-title: Remove native title tooltip in pipeline jobs dropdown in Safari
-merge_request:
-author:
-type: fixed

changelogs/unreleased/39757-border-zero-of-scss-lint.yml

----
-title: Enable BorderZero rule in scss-lint
-merge_request: 15168
-author: Takuya Noguchi
-type: other

changelogs/unreleased/39776-remove-responsive-table-bottom-border.yml

----
-title: Fix double border UI bug on pipelines/environments table and pagination
-merge_request:
-author:
-type: fixed

changelogs/unreleased/39791-when-reopening-an-issue-the-mattermost-notification-has-no-context-to-the-issue.yml

----
-title: Include link to issue in reopen message for Slack and Mattermost notifications
-merge_request:
-author:
-type: fixed

changelogs/unreleased/39878-commit-pipeline-reads-wrong-key.yml

----
-title: Fix commit pipeline showing wrong status
-merge_request:
-author:
-type: fixed

changelogs/unreleased/40068-runner-sorting-regression.yml

----
-title: Revert a regression on runners sorting (!15134)
-merge_request: 15341
-author: Takuya Noguchi
-type: fixed

changelogs/unreleased/add-changes-count-to-merge-requests-api.yml

----
-title: Add a count of changes to the merge requests API
-merge_request:
-author:
-type: changed

changelogs/unreleased/add-ingress-to-cluster-applications.yml

----
-title: Add Ingress to available Cluster applications
-merge_request:
-author:
-type: added

changelogs/unreleased/add-lazy-option-to-user-avatar-image-component.yml

----
-title: Add lazy option to UserAvatarImage
-merge_request: 14895
-author:
-type: changed

changelogs/unreleased/add-packagist-project-service.yml

----
-title: Add Packagist project service
-merge_request: 14493
-author: Matt Coleman
-type: added

changelogs/unreleased/add-shared-vue-loading-button.yml

----
-title: Add loading button for new UX paradigm
-merge_request: 14883
-author:
-type: added

changelogs/unreleased/add_commit_author_check_feature.yml

----
-title: Add new push rule to enforce that only the author of a commit can push to the repository
-merge_request: 3086
-author:
-type: added

changelogs/unreleased/an-use-branch-exists-over-branch-names-include.yml

----
-title: Avoid fetching all branches for branch existence checks
-merge_request: 14778
-author:
-type: changed

changelogs/unreleased/api-configure-jira.yml

----
-title: Validate username/pw for Jiraservice, require them in the API
-merge_request: 15025
-author: Robert Schilling
-type: fixed

changelogs/unreleased/api-doc-group-statistics.yml

----
-title: Update the groups API documentation
-merge_request: 15024
-author: Robert Schilling
-type: fixed

changelogs/unreleased/backport-workhorse-show-all-refs.yml

----
-title: Support show-all-refs for git over HTTP
-merge_request: 14834
-author:
-type: added

changelogs/unreleased/bugfix_banzai_closed_milestones.yml

----
-title: Fix GFM reference links for closed milestones
-merge_request: 15234
-author: Vitaliy @blackst0ne Klachkov
-type: fixed

changelogs/unreleased/bvl-fix-group-atom-feed.yml

----
-title: Fix the atom feed for group events
-merge_request: 14974
-author:
-type: fixed

changelogs/unreleased/bvl-free-paths.yml

----
-title: Free up some reserved group names
-merge_request: 15052
-author:
-type: other

changelogs/unreleased/bvl-group-trees.yml

----
-title: Show collapsible project lists
-merge_request: 14055
-author:
-type: changed

changelogs/unreleased/bvl-refresh-member-listing-on-removal.yml

----
-title: Don't use JS to delete memberships from projects and groups
-merge_request: 15344
-author:
-type: fixed

changelogs/unreleased/dm-add-sudo-scope.yml

----
-title: Add sudo scope for OAuth and Personal Access Tokens to be used by admins to
-  impersonate other users on the API
-merge_request:
-author:
-type: added

changelogs/unreleased/dm-avatarable-with-asset-host.yml

----
-title: Always return full avatar URL for private/internal groups/projects when asset
-  host is set
-merge_request:
-author:
-type: fixed

changelogs/unreleased/dm-convert-private-tokens.yml

----
-title: Convert private tokens to Personal Access Tokens with sudo scope
-merge_request:
-author:
-type: security

changelogs/unreleased/dm-notes-for-commit-id.yml

----
-title: Improve performance of commits list by fully using DB index when getting commit
-  note counts
-merge_request:
-author:
-type: performance

changelogs/unreleased/dm-reallow-project-path-ending-in-period.yml

----
-title: Reallow project paths ending in periods
-merge_request:
-author:
-type: fixed

changelogs/unreleased/dm-remove-private-token-from-interface.yml

----
-title: Remove private tokens from web interface and API
-merge_request:
-author:
-type: security

changelogs/unreleased/dm-remove-private-token.yml

----
-title: Remove Session API now that private tokens are removed from user API endpoints
-merge_request:
-author:
-type: removed

changelogs/unreleased/enable-scss-lint-mergeable-selector.yml

----
-title: Enable MergeableSelector in scss-lint
-merge_request: 12810
-author: Takuya Noguchi

changelogs/unreleased/es-module-broadcast_message.yml

----
-title: Fix unnecessary ajax requests in admin broadcast message form
-merge_request: 14853
-author:
-type: fixed

changelogs/unreleased/expose-job-duration.yml

----
-title: Expose duration in Job entity
-merge_request: 13644
-author: Mehdi Lahmam (@mehlah)
-type: added

changelogs/unreleased/feature-change-signout-route.yml

----
-title: Change 'Sign Out' route from a DELETE to a GET
-merge_request: 39708
-author: Joe Marty
-type: changed

changelogs/unreleased/feature-custom-attributes-on-projects-and-groups.yml

----
-title: Support custom attributes on groups and projects
-merge_request: 14593
-author: Markus Koller
-type: changed

changelogs/unreleased/feature-hashed-storage-repo-import.yml

----
-title: Improve GitLab Import rake task to work with Hashed Storage and Subgroups
-merge_request:
-author:
-type: changed

changelogs/unreleased/feature-plantuml-restructured-text-captions.yml

----
-title: 'Support uml:: and captions in reStructuredText'
-merge_request: 15120
-author: Markus Koller
-type: changed

changelogs/unreleased/feature-reliable-rspec-with-eval-script.yml

----
-title: Get true failure from evalulate_script by checking for element beforehand
-merge_request: 14898
-author:
-type: fixed

changelogs/unreleased/feature-ssh_host_fingerprint.yml

----
-title: Automatic configuration settings page
-merge_request: 13850
-author: Francisco Lopez
-type: added

changelogs/unreleased/feature_change_sort_refs.yml

----
-title: Change tags order in refs dropdown
-merge_request: 15235
-author: Vitaliy @blackst0ne Klachkov
-type: changed

changelogs/unreleased/fix-500-on-old-merge-requests.yml

----
-title: Fix 500 errors caused by empty diffs in some discussions
-merge_request: 14945
-author: Alexander Popov
-type: fixed

changelogs/unreleased/fix-502-mrs-with-lots-of-versions.yml

----
-title: Ensure merge requests with lots of version don't time out when searching for
-  pipelines
-merge_request:
-author:
-type: performance

changelogs/unreleased/fix-issues-api-list-performance.yml

----
-title: Speed up issues list APIs
-merge_request:
-author:
-type: performance

changelogs/unreleased/fix-md-form-tabs-double-click-toggle.yml

----
-title: Fix markdown form tabs toggling preview mode from double clicking write mode
-  button
-merge_request: 15119
-author:
-type: fixed

changelogs/unreleased/fix-project-select-js-without-button.yml

----
-title: Use project select dropdown not only as a combobutton
-merge_request: 15043
-author:
-type: fixed

changelogs/unreleased/fix-system-hook-docs.yml

----
-title: Clarify system_hook triggers in documentation
-merge_request: 14957
-author: Joe Marty
-type: other

changelogs/unreleased/fix-user-tab-activity-mobile.yml

----
-title: Fixed user profile activity tab being off-screen on mobile
-merge_request:
-author:
-type: fixed

changelogs/unreleased/go-get-ssh.yml

----
-title: Returns a ssh url for go-get=1
-merge_request: 14990
-author: gvieira37
-type: fixed

changelogs/unreleased/hide-pipeline-zero-duration.yml

----
-title: Hides pipeline duration in commit box when it is zero (nil)
-merge_request: 14979
-author: gvieira37
-type: fixed

changelogs/unreleased/issue-36484.yml

----
-title: Remove unnecessary alt-texts from pipeline emails
-merge_request: 14602
-author: gernberg
-type: fixed

changelogs/unreleased/issue_38777.yml

----
-title: Allow promoting project milestones to group milestones
-merge_request:
-author:
-type: added

changelogs/unreleased/jej-fs-prevent-push-when-missing-objects.yml

----
-title: Prevent git push when LFS objects are missing
-merge_request: 13837
-author:
-type: added

changelogs/unreleased/jivl-mobile-friendly-table-runners.yml

----
-title: Mobile-friendly table on Admin Runners
-merge_request:
-author: Takuya Noguchi
-type: fixed

changelogs/unreleased/move_markdown_preview_to_concern.yml

----
-title: Add support for markdown preview to group milestones
-merge_request: 14806
-author: Vitaliy @blackst0ne Klachkov
-type: fixed

changelogs/unreleased/multi-file-editor-submodules.yml

----
-title: Added submodule support in multi-file editor
-merge_request:
-author:
-type: added

changelogs/unreleased/multiple-query-prometheus-graphs.yml

----
-title: Allow multiple queries in a single Prometheus graph to support additional environments
-  (Canary, Staging, et al.)
-merge_request: 15201
-author:
-type: added

changelogs/unreleased/new-mr-repo-editor.yml

----
-title: 'Repo Editor: Add option to start a new MR directly from comit section'
-merge_request: 14665
-author:
-type: added

changelogs/unreleased/not-found-in-commits.yml

----
-title: Renders 404 in commits controller if no commits are found for a given path
-merge_request: 14610
-author: Guilherme Vieira
-type: fixed

changelogs/unreleased/pawel-metrics-to-prometheus-33643.yml

----
-title: Add Prometheus equivalent of all InfluxDB metrics
-merge_request: 13891
-author:
-type: changed

changelogs/unreleased/pawel-show_empty_page_when_prometheus_metrics_are_disabled-35639.yml

----
-title: Make Prometheus metrics endpoint return empty response when metrics are disabled
-merge_request: 14490
-author:
-type: changed

changelogs/unreleased/ph-multi-file-upload-file.yml

----
-title: Allow files to uploaded in the multi-file editor
-merge_request:
-author:
-type: added

changelogs/unreleased/refactor-group_links_controller.yml

----
-title: Refactor GroupLinksController
-merge_request:
-author: 15121
-type: other

changelogs/unreleased/remove-ensure-ref-fetched-from-controllers.yml

----
-title: Stop merge requests from fetching their refs when the data is already available.
-merge_request: 15129
-author:
-type: removed

changelogs/unreleased/replace_explore_projects-feature.yml

----
-title: Replace the 'features/explore/projects.feature' spinach test with an rspec analog
-merge_request: 14755
-author: Vitaliy @blackst0ne Klachkov
-type: other

changelogs/unreleased/sh-disable-unicorn-sampling-sidekiq.yml

----
-title: Disable Unicorn sampling in Sidekiq since there are no Unicorn sockets to monitor
-merge_request:
-author:
-type: performance

changelogs/unreleased/sh-fix-broken-redirection-relative-url-root.yml

----
-title: Fix broken Members link when relative URL root paths are used
-merge_request:
-author:
-type: fixed

changelogs/unreleased/sh-memoize-logger.yml

----
-title: Memoize GitLab logger to reduce open file descriptors
-merge_request:
-author:
-type: fixed

changelogs/unreleased/sha-handling.yml

----
-title: Fix 404 errors in API caused when the branch name had a dot
-merge_request: 14462
-author: gvieira37
-type: fixed

changelogs/unreleased/tc-delete-merged-protected-tags-fix.yml

----
-title: When deleting merged branches, ignore protected tags
-merge_request: 15252
-author:
-type: fixed

changelogs/unreleased/tc-saml-fix-false-empty.yml

----
-title: Fix SAML error 500 when no groups are defined for user
-merge_request: 14913
-author:
-type: fixed

changelogs/unreleased/update-fe-i18n-guide.yml

----
-title: Update i18n section in FE docs for marking and interpolation
-merge_request:
-author:
-type: changed

changelogs/unreleased/use-git-branch-merged.yml

----
-title: Improve branch listing page performance
-merge_request: 14729
-author:
-type: performance

changelogs/unreleased/use-title.yml

----
-title: Use title as placeholder instead of issue title for reusability
-merge_request:
-author:
-type: other

changelogs/unreleased/winh-admin-projects-namespace-filter.yml

----
-title: Make NamespaceSelect change URL when filtering
-merge_request: 14888
-author:
-type: fixed

changelogs/unreleased/winh-i18n-contributors-page.yml

----
-title: Make contributors page translatable
-merge_request: 14915
-author:
-type: other

changelogs/unreleased/winh-namespace-rename-hooks.yml

----
-title: Add system hooks user_rename and group_rename
-merge_request: 15123
-author:
-type: changed

changelogs/unreleased/zj-add-performance-changelog-cat.yml

----
-title: Add Performance improvement as category on the changelog
-merge_request:
-author:
-type: performance

changelogs/unreleased/zj-commit-cache.yml

----
-title: Cache commits fetched from the repository
-merge_request:
-author:
-type: performance

changelogs/unreleased/zj-peek-gitaly.yml

----
-title: Add Gitaly metrics to the performance bar
-merge_request:
-author:
-type: other

changelogs/unreleased/zj-ruby-2-3-5.yml

----
-title: Upgrade Ruby to 2.3.5 to include security patches
-merge_request: 15099
-author:
-type: security

doc/administration/high_availability/alpha_database.md

 # Configuring a Database for GitLab HA
 
-There are multiple ways in which you can achieve Database High Availability
-for use with GitLab:
-
-* Use bundled services and configuration provided by the Omnibus GitLab package.
-This option is available with [Enterprise Edition Premium](https://about.gitlab.com/gitlab-ee/) license.
-* Use a cloud hosted solution
-* Install and manage the database and other components yourself
-
-> Important notes:
-- Please read [database requirements document](https://docs.gitlab.com/ee/install/requirements.html#database) for more information on supported databases.
-- This document will focus only on configuration supported with [GitLab Enterprise Edition Premium](https://about.gitlab.com/gitlab-ee/), using the Omnibus GitLab package.
-- If you are a Community Edition or Enterprise Edition Starter user, consider using a cloud hosted solution.
-- This document will not cover installations from source.
-
->
-- If HA setup is not what you were looking for,  see the [database configuration document](http://docs.gitlab.com/omnibus/settings/database.html)
-for the Omnibus GitLab packages.
-
-## Overview
-
->
-Please read this document fully before attempting to configure PostgreSQL HA
-for GitLab.
-
-The recommended configuration for a PostgreSQL HA requires:
-
-- A minimum of three database nodes
-  - Each node will run the following services:
-    - `PostgreSQL` - The database itself
-    - `repmgrd` - A service to monitor, and handle failover in case of a failure
-    - `Consul` agent - Used for service discovery, to alert other nodes when failover occurs
-- A minimum of three `Consul` server nodes
-- A minimum of one `pgbouncer` service node
-
-You also need to take into consideration the underlying network topology,
-making sure you have redundant connectivity between all Database and GitLab instances,
-otherwise the networks will become a single point of failure.
-
-## Required information
-
-Before proceeding with configuration, you will need to collect all the necessary
-information.
-
-### Network information
-
-PostgreSQL does not listen on any network interface by default. It needs to know
-which IP address to listen on in order to be accessible to other services.
-Similarly, PostgreSQL access is controlled based on the network source.
-
-This is why you will need:
-
-> IP address of each nodes network interface
-- This can be set to `0.0.0.0` to listen on all interfaces. It cannot
-  be set to the loopack address `127.0.0.1`
-
-> Network Address
-- This can be in subnet (i.e. `192.168.0.0/255.255.255.0`) or CIDR (i.e.
-  `192.168.0.0/24`) form.
-
-### User information
-
-Various services require different configuration to secure
-the communication as well as information required for running the service.
-Bellow you will find details on each service and the minimum required
-information you need to provide.
-
-#### Consul information
-
-When using default setup, minimum configuration requires:
-
-- `CONSUL_USERNAME`. Defaults to `gitlab-consul`
-- `CONSUL_DATABASE_PASSWORD`. Password for the database user.
-- `CONSUL_PASSWORD_HASH`. This is a hash generated out of consul username/password pair.
-Can be generated with:
-    ```sh
-    echo -n 'CONSUL_DATABASE_PASSWORDCONSUL_USERNAME' | md5sum
-    ```
-- `CONSUL_SERVER_NODES`.  The IP addresses or DNS records of the Consul server nodes.
-
-Few notes on the service itself:
-
-- The service runs under a system account, by default `gitlab-consul`.
-  - If you are using a different username, you will have to specify it. We
-will refer to it with `CONSUL_USERNAME`,
-- There will be a database user created with read only access to the repmgr
-database
-- Passwords will be stored in the following locations:
-  - `/etc/gitlab/gitlab.rb`: hashed
-  - `/var/opt/gitlab/pgbouncer/pg_auth`: hashed
-  - `/var/opt/gitlab/gitlab-consul/.pgpass`: plaintext
-
-#### PostgreSQL information
-
-When configuring PostgreSQL, we will set `max_wal_senders` to one more than
-the number of database nodes in the cluster.
-This is used to prevent replication from using up all of the
-available database connections.
-
-> Note:
-- In this document we are assuming 3 database nodes, which makes this configuration:
-
-```
-postgresql['max_wal_senders'] = 4
-```
-
-As previously mentioned, you'll have to prepare the network subnets that will
-be allowed to authenticate with the database.
-You'll also need to supply the IP addresses or DNS records of Consul
-server nodes.
-
-We will need the following password information for the application's database user:
-
-- `POSTGRESQL_USERNAME`. Defaults to `gitlab`
-- `POSTGRESQL_USER_PASSWORD`. The password for the database user
-- `POSTGRESQL_PASSWORD_HASH`. This is a hash generated out of the username/password pair.
-Can be generated with:
-    ```sh
-    echo -n 'POSTGRESQL_USER_PASSWORDPOSTGRESQL_USERNAME' | md5sum
-    ```
-
-#### Pgbouncer information
-
-When using default setup, minimum configuration requires:
-
-- `PGBOUNCER_USERNAME`. Defaults to `pgbouncer`
-- `PGBOUNCER_PASSWORD`. This is a password for pgbouncer service.
-- `PGBOUNCER_PASSWORD_HASH`. This is a hash generated out of pgbouncer username/password pair.
-Can be generated with:
-    ```sh
-    echo -n 'PGBOUNCER_PASSWORDPGBOUNCER_USERNAME' | md5sum
-    ```
-- `PGBOUNCER_NODE`, is the IP address or a FQDN of the node running Pgbouncer.
-
-Few notes on the service itself:
-
-- The service runs as the same system account as the database
-  - In the package, this is by default `gitlab-psql`
-- If you use a non-default user account for Pgbouncer service (by default `pgbouncer`), you will have to specify this username. We will refer to this requirement with `PGBOUNCER_USERNAME`.
-- The service will have a regular database user account generated for it
-  - This defaults to `repmgr`
-- Passwords will be stored in the following locations:
-  - `/etc/gitlab/gitlab.rb`: hashed, and in plain text
-  - `/var/opt/gitlab/pgbouncer/pg_auth`: hashed
-
-#### Repmgr information
-
-When using default setup, you will only have to prepare the network subnets that will
-be allowed to authenticate with the service.
-
-Few notes on the service itself:
-
-- The service runs under the same system account as the database
-  -  In the package, this is by default `gitlab-psql`
-- The service will have a superuser database user account generated for it
-  - This defaults to `gitlab_repmgr`
-
-## Installing Omnibus GitLab
-
-First, make sure to [download/install](https://about.gitlab.com/installation)
-GitLab Omnibus **on each node**.
-
-Make sure you install the necessary dependencies from step 1,
-add GitLab package repository from step 2.
-When installing the GitLab package, do not supply `EXTERNAL_URL` value.
-
-## Initial node configuration
-
-Each node needs to be configured to run only the services it needs.
-
-### Configuring the Consul nodes
-
-On each Consul node perform the following:
-
-1. Make sure you collect all required information before executing the next step.
-See `START user configuration` section in the next step for required information.
-1. Edit `/etc/gitlab/gitlab.rb`:
-
-    ```ruby
-    # Disable all components except Consul
-    bootstrap['enable'] = false
-    gitlab_rails['auto_migrate'] = false
-    gitaly['enable'] = false
-    gitlab_workhorse['enable'] = false
-    mailroom['enable'] = false
-    nginx['enable'] = false
-    postgresql['enable'] = false
-    redis['enable'] = false
-    sidekiq['enable'] = false
-    prometheus['enable'] = false
-    unicorn['enable'] = false
-
-    consul['enable'] = true
-    # START user configuration
-    # Replace placeholders:
-    #
-    # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
-    # with the addresses gathered for CONSUL_SERVER_NODES
-    consul['configuration'] = {
-      server: true,
-      retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z)
-    }
-    #
-    # END user configuration
-    ```
-
-1. [Reconfigure GitLab] for the changes to take effect.
-
-After this is completed on each Consul server node, proceed further.
-
-### Configuring the Database nodes
-
-On each database node perform the following:
-
-1. Make sure you collect all required information before executing the next step.
-See `START user configuration` section in the next step for required information.
-1. Edit `/etc/gitlab/gitlab.rb`:
-
-    ```ruby
-    # Disable all components except PostgreSQL and Repmgr and Consul
-    bootstrap['enable'] = false
-    gitaly['enable'] = false
-    mailroom['enable'] = false
-    nginx['enable'] = false
-    unicorn['enable'] = false
-    sidekiq['enable'] = false
-    redis['enable'] = false
-    gitlab_workhorse['enable'] = false
-    prometheus_monitoring['enable'] = false
-
-    repmgr['enable'] = true
-    postgresql['enable'] = true
-    consul['enable'] = true
-
-    # PostgreSQL configuration
-    postgresql['listen_address'] = '0.0.0.0'
-    postgresql['hot_standby'] = 'on'
-    postgresql['wal_level'] = 'replica'
-    postgresql['shared_preload_libraries'] = 'repmgr_funcs'
-
-    # Disable automatic database migrations
-    gitlab_rails['auto_migrate'] = false
-
-    # Configure the consul agent
-    consul['services'] = %w(postgresql)
-
-    # START user configuration
-    # Please set the real values as explained in Required Information section
-    #
-    # Replace PGBOUNCER_PASSWORD_HASH with a generated md5 value
-    postgresql['pgbouncer_user_password'] = 'PGBOUNCER_PASSWORD_HASH'
-    postgresql['sql_user_password'] = 'POSTGRESQL_PASSWORD_HASH'
-    # Replace X with value of number of db nodes + 1
-    postgresql['max_wal_senders'] = X
-
-    # Replace XXX.XXX.XXX.XXX/YY with Network Address
-    postgresql['trust_auth_cidr_addresses'] = %w(XXX.XXX.XXX.XXX/YY)
-    repmgr['trust_auth_cidr_addresses'] = %w(XXX.XXX.XXX.XXX/YY)
-
-    # Replace placeholders:
-    #
-    # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
-    # with the addresses gathered for CONSUL_SERVER_NODES
-    consul['configuration'] = {
-      retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z)
-    }
-    #
-    # END user configuration
-    ```
-
-1. [Reconfigure GitLab] for the changes to take effect.
-
-> Please note:
-- If you want your database to listen on a specific interface, change the config:
-`postgresql['listen_address'] = '0.0.0.0'`
-- If your Pgbouncer service runs under a different user account,
-you also need to specify: `postgresql['pgbouncer_user'] = PGBOUNCER_USERNAME` in
-your configuration
-`
-
-### Configuring the Pgbouncer node
-
-1. Edit `/etc/gitlab/gitlab.rb`:
-
-    ```ruby
-    # Disable all components except Pgbouncer and Consul agent
-    bootstrap['enable'] = false
-    gitaly['enable'] = false
-    mailroom['enable'] = false
-    nginx['enable'] = false
-    redis['enable'] = false
-    prometheus['enable'] = false
-    postgresql['enable'] = false
-    unicorn['enable'] = false
-    sidekiq['enable'] = false
-    gitlab_workhorse['enable'] = false
-    gitlab_rails['auto_migrate'] = false
-
-    pgbouncer['enable'] = true
-    consul['enable'] = true
-
-    # Configure Pgbouncer
-    pgbouncer['admin_users'] = %w(pgbouncer gitlab-consul)
-
-    # Configure Consul agent
-    consul['watchers'] = %w(postgresql)
-
-    # START user configuration
-    # Please set the real values as explained in Required Information section
-    # Replace CONSUL_PASSWORD_HASH with with a generated md5 value
-    # Replace PGBOUNCER_PASSWORD_HASH with with a generated md5 value
-    pgbouncer['users'] = {
-      'gitlab-consul': {
-        password: 'CONSUL_PASSWORD_HASH'
-      },
-      'pgbouncer': {
-        password: 'PGBOUNCER_PASSWORD_HASH'
-      }
-    }
-    # Replace placeholders:
-    #
-    # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
-    # with the addresses gathered for CONSUL_SERVER_NODES
-    consul['configuration'] = {
-      retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z)
-    }
-    #
-    # END user configuration
-    ```
-
-1. [Reconfigure GitLab] for the changes to take effect.
-
-### Configuring the Application nodes
-
-These will be the nodes running the `gitlab-rails` service. You may have other
-attributes set, but the following need to be set.
-
-1. Edit `/etc/gitlab/gitlab.rb`:
-
-    ```ruby
-    # Disable PostgreSQL on the application node
-    postgresql['enable'] = false
-
-    gitlab_rails['db_host'] = 'PGBOUNCER_NODE'
-    gitlab_rails['db_port'] = 6432
-    gitlab_rails['db_password'] = 'POSTGRESQL_USER_PASSWORD'
-    gitlab_rails['auto_migrate'] = false
-    ```
-
-1. [Reconfigure GitLab] for the changes to take effect.
-
-## Node post-configuration
-
-After reconfigure successfully runs, the following steps must be completed to
-get the cluster up and running.
-
-### Consul nodes post-configuration
-
-Verify the nodes are all communicating:
-
-```sh
-/opt/gitlab/embedded/bin/consul members
-```
-
-The output should be similar to:
-
-```
-Node                 Address               Status  Type    Build  Protocol  DC
-CONSUL_NODE_ONE      XXX.XXX.XXX.YYY:8301  alive   server  0.9.2  2         gitlab_consul
-CONSUL_NODE_TWO      XXX.XXX.XXX.YYY:8301  alive   server  0.9.2  2         gitlab_consul
-CONSUL_NODE_THREE    XXX.XXX.XXX.YYY:8301  alive   server  0.9.2  2         gitlab_consul
-DATABASE_NODE_ONE    XXX.XXX.XXX.YYY:8301  alive   client  0.9.2  2         gitlab_consul
-DATABASE_NODE_TWO    XXX.XXX.XXX.YYY:8301  alive   client  0.9.2  2         gitlab_consul
-DATABASE_NODE_THREE  XXX.XXX.XXX.YYY:8301  alive   client  0.9.2  2         gitlab_consul
-PGBOUNCER_NODE       XXX.XXX.XXX.YYY:8301  alive   client  0.9.0  2         gitlab_consul
-```
-
-### Database nodes post-configuration
-
-#### Primary node
-
-Select one node as a primary node.
-
-1. Open a database prompt:
-
-    ```sh
-    gitlab-psql -d gitlabhq_production
-    ```
-
-1. Enable the `pg_trgm` extension:
-
-    ```sh
-    CREATE EXTENSION pg_trgm;
-    ```
-
-1. Exit the database prompt by typing `\q` and Enter.
-1. Verify the cluster is initialized with one node:
-
-     ```sh
-     gitlab-ctl repmgr cluster show
-     ```
-
-     The output should be similar to the following:
-
-     ```
-     Role      | Name     | Upstream | Connection String
-     ----------+----------|----------|----------------------------------------
-     * master  | HOSTNAME |          | host=HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr
-     ```
-1. Note down the value in the `Name` column. We will refer to it in the next section
-as `MASTER_NODE_NAME`.
-
-#### Secondary nodes
-
-1. Setup the repmgr standby:
-
-    ```sh
-    gitlab-ctl repmgr standby setup MASTER_NODE_NAME
-    ```
-    Do note that this will remove the existing data on the node. The command
-    has a wait time.
-
-1. Verify the node now appears in the cluster:
-
-     ```sh
-     gitlab-ctl repmgr cluster show
-     ```
-
-     The output should be similar to the following:
-
-     ```
-     Role      | Name    | Upstream  | Connection String
-     ----------+---------|-----------|------------------------------------------------
-     * master  | MASTER  |           | host=MASTER_NODE_NAME user=gitlab_repmgr dbname=gitlab_repmgr
-       standby | STANDBY | MASTER    | host=STANDBY_HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr
-     ```
-
-Repeat the above steps on all secondary nodes.
-
-### Pgbouncer node post-configuration
-
-1. Create a `.pgpass` file user for the `CONSUL_USER` account to be able to
-   reload pgbouncer. Confirm `PGBOUNCER_PASSWORD` twice when asked:
-
-     ```sh
-     gitlab-ctl write-pgpass --host 127.0.0.1 --database pgbouncer --user pgbouncer --hostuser gitlab-consul
-     ```
-
-1. Ensure the node is talking to the current master:
-
-     ```sh
-     gitlab-ctl pgb-console # You will be prompted for PGBOUNCER_PASSWORD
-     ```
-
-     Then run:
-
-     ```sh
-     show databases ; show clients ;
-     ```
-
-     The output should be similar to the following:
-
-     ```
-             name         |  host       | port |      database       | force_user | pool_size | reserve_pool | pool_mode | max_connections | current_connections
-     ---------------------+-------------+------+---------------------+------------+-----------+--------------+-----------+-----------------+---------------------
-      gitlabhq_production | MASTER_HOST | 5432 | gitlabhq_production |            |        20 |            0 |           |               0 |                   0
-      pgbouncer           |             | 6432 | pgbouncer           | pgbouncer  |         2 |            0 | statement |               0 |                   0
-     (2 rows)
-
-      type |   user    |      database       |  state  |   addr         | port  | local_addr | local_port |    connect_time     |    request_time     |    ptr    | link | remote_pid | tls
-     ------+-----------+---------------------+---------+----------------+-------+------------+------------+---------------------+---------------------+-----------+------+------------+-----
-      C    | (nouser)  | gitlabhq_production | waiting | IP_OF_APP_NODE | 56512 | 127.0.0.1  |       6432 | 2017-08-21 18:08:51 | 2017-08-21 18:08:51 | 0x22b3700 |      |          0 |
-      C    | pgbouncer | pgbouncer           | active  | 127.0.0.1      | 56846 | 127.0.0.1  |       6432 | 2017-08-21 18:09:59 | 2017-08-21 18:10:48 | 0x22b3880 |      |          0 |
-     (2 rows)
-     ```
-
-### Application node post-configuration
-
-Ensure that all migrations ran:
-
-```sh
-gitlab-rake gitlab:db:configure
-```
-
-## Ensure GitLab is running
-
-At this point, your GitLab instance should be up and running. Verify you are
-able to login, and create issues and merge requests.
-
-## Failover procedure
-
-By default, if the master database fails, `repmgrd` should promote one of the
-standby nodes to master automatically, and consul will update pgbouncer with
-the new master.
-
-If you need to failover manually, you have two options:
-
-**Shutdown the current master database**
-
-Run:
-
-```sh
-gitlab-ctl stop postgresql
-```
-
-The automated failover process will see this and failover to one of the
-standby nodes.
-
-**Or perform a manual failover**
-
-1. Ensure the old master node is not still active.
-1. Login to the server that should become the new master and run:
-
-    ```sh
-    gitlab-ctl repmgr standby promote
-    ```
-
-1. If there are any other standby servers in the cluster, have them follow
-   the new master server:
-
-    ```sh
-    gitlab-ctl repmgr standby follow NEW_MASTER
-    ```
-
-## Restore procedure
-
-If a node fails, it can be removed from the cluster, or added back as a standby
-after it has been restored to service.
-
-- If you want to remove the node from the cluster, on any other node in the
-  cluster, run:
-
-    ```sh
-    gitlab-ctl repmgr standby unregister --node=X
-    ```
-
-    where X is be the value of node in `repmgr.conf` on the old server.
-
-- To add the node as a standby server:
-
-    ```sh
-    gitlab-ctl repmgr standby follow NEW_MASTER
-    gitlab-ctl restart repmgrd
-    ```
-
-    CAUTION: **Warning:** When the server is brought back online, and before
-    you switch it to a standby node, repmgr will report that there are two masters.
-    If there are any clients that are still attempting to write to the old master,
-    this will cause a split, and the old master will need to be resynced from
-    scratch by performing a `standby setup NEW_MASTER`.
-
-## Alternate configurations
-
-### Database authorization
-
-By default, we give any host on the database network the permission to perform
-repmgr operations using PostgreSQL's `trust` method. If you do not want this
-level of trust, there are alternatives.
-
-You can trust only the specific nodes that will be database clusters, or you
-can require md5 authentication.
-
-#### Trust specific addresses
-
-If you know the IP address, or FQDN of all database and pgbouncer nodes in the
-cluster, you can trust only those nodes.
-
-In `/etc/gitlab/gitlab.rb` on all of the database nodes, set
-`repmgr['trust_auth_cidr_addresses']` to an array of strings containing all of
-the addresses.
-
-If setting to a node's FQDN, they must have a corresponding PTR record in DNS.
-If setting to a node's IP address, specify it as `XXX.XXX.XXX.XXX/32`.
-
-For example:
-
-```ruby
-repmgr['trust_auth_cidr_addresses'] = %w(192.168.1.44/32 db2.example.com)
-```
-
-#### MD5 Authentication
-
-If you are running on an untrusted network, repmgr can use md5 authentication
-with a [.pgpass file](https://www.postgresql.org/docs/9.6/static/libpq-pgpass.html)
-to authenticate.
-
-You can specify by IP address, FQDN, or by subnet, using the same format as in
-the previous section:
-
-1. On the current master node, create a password for the `gitlab` and
-   `gitlab_repmgr` user:
-
-    ```sh
-    gitlab-psql -d template1
-    template1=# \password gitlab_repmgr
-    Enter password: ****
-    Confirm password: ****
-    template1=# \password gitlab
-    ```
-
-1. On each database node:
-
-  1. Edit `/etc/gitlab/gitlab.rb`:
-    1. Ensure `repmgr['trust_auth_cidr_addresses']` is **not** set
-    1. Set `postgresql['md5_auth_cidr_addresses']` to the desired value
-    1. Set `postgresql['sql_replication_user'] = 'gitlab_repmgr'`
-    1. Reconfigure with `gitlab-ctl reconfigure`
-    1. Restart postgresql with `gitlab-ctl restart postgresql`
-
-  1. Create a `.pgpass` file. Enter the `gitlab_repmgr` password twice to
-     when asked:
-
-        ```sh
-        gitlab-ctl write-pgpass --user gitlab_repmgr --hostuser gitlab-psql --database '*'
-        ```
-
-1. On each pgbouncer node, edit `/etc/gitlab/gitlab.rb`:
-  1. Ensure `gitlab_rails['db_password']` is set to the plaintext password for
-     the `gitlab` database user
-  1. [Reconfigure GitLab] for the changes to take effect
-
-## Architecture
-
-![PG HA Architecture](pg_ha_architecture.png)
-
-Database nodes run two services besides PostgreSQL
-1. Repmgrd -- monitors the cluster and handles failover in case of an issue with the master
-
-   The failover consists of
-   * Selecting a new master for the cluster
-   * Promoting the new node to master
-   * Instructing remaining servers to follow the new master node
-
-   On failure, the old master node is automatically evicted from the cluster, and should be rejoined manually once recovered.
-
-1. Consul -- Monitors the status of each node in the database cluster, and tracks its health in a service definiton on the consul cluster.
-
-Alongside pgbouncer, there is a consul agent that watches the status of the PostgreSQL service. If that status changes, consul runs a script which updates the configuration and reloads pgbouncer
-
-## Troubleshooting
-
-### Consul and PostgreSQL changes not taking effect.
-
-Due to the potential impacts, `gitlab-ctl reconfigure` only reloads Consul and PostgreSQL, it will not restart the services. However, not all changes can be activated by reloading.
-
-To restart either service, run `gitlab-ctl restart SERVICE`
-
-For PostgreSQL, it is usually safe to restart the master node by default. Automatic failover defaults to a 1 minute timeout. Provided the database returns before then, nothing else needs to be done. To be safe, you can stop `repmgrd` on the standby nodes first with `gitlab-ctl stop repmgrd`, then start afterwards with `gitlab-ctl start repmgrd`.
-
-On the consul server nodes, it is important to restart the consul service in a controlled fashion. Read our [consul documentation](consul.md#restarting-the-server-cluster) for instructions on how to restart the service.
-
-### Issues with other components
-
-If you're running into an issue with a component not outlined here, be sure to check the troubleshooting section of their specific documentation page.
-
-- [Consul](consul.md#troubleshooting)
-- [PostgreSQL](http://docs.gitlab.com/omnibus/settings/database.html#troubleshooting)
-- [GitLab application](gitlab.md#troubleshooting)
-
----
-
-Read more on high-availability configuration:
-
-1. [Configure Redis](redis.md)
-1. [Configure NFS](nfs.md)
-1. [Configure the GitLab application servers](gitlab.md)
-1. [Configure the load balancers](load_balancer.md)
-1. [Manage the bundled Consul cluster](consul.md)
-
-[reconfigure GitLab]: ../restart_gitlab.md#omnibus-gitlab-reconfigure
+This documentation has been moved to the main [database documentation](database.md#configure_using_omnibus_for_high_availability).

doc/administration/high_availability/database.md

@@ -5,6 +5,15 @@ yourself, or you can use GitLab Omnibus packages to help. GitLab recommends
 PostgreSQL. This is the database that will be installed if you use the
 Omnibus package to manage your database.
 
+> Important notes:
+- This document will focus only on configuration supported with [GitLab Enterprise Edition Premium](https://about.gitlab.com/gitlab-ee/), using the Omnibus GitLab package.
+- If you are a Community Edition or Enterprise Edition Starter user, consider using a cloud hosted solution.
+- This document will not cover installations from source.
+
+>
+- If HA setup is not what you were looking for,  see the [database configuration document](http://docs.gitlab.com/omnibus/settings/database.html)
+for the Omnibus GitLab packages.
+
 ## Configure your own database server
 
 If you're hosting GitLab on a cloud provider, you can optionally use a
@@ -20,103 +29,691 @@ If you use a cloud-managed service, or provide your own PostgreSQL:
 1. Configure the GitLab application servers with the appropriate details.
    This step is covered in [Configuring GitLab for HA](gitlab.md).
 
-## Configure using Omnibus
+## Configure using Omnibus for High Availability
+
+>
+Please read this document fully before attempting to configure PostgreSQL HA
+for GitLab.
+
+>
+This configuration is GA in EE 10.2.
+
+The recommended configuration for a PostgreSQL HA requires:
+
+- A minimum of three database nodes
+  - Each node will run the following services:
+    - `PostgreSQL` - The database itself
+    - `repmgrd` - A service to monitor, and handle failover in case of a failure
+    - `Consul` agent - Used for service discovery, to alert other nodes when failover occurs
+- A minimum of three `Consul` server nodes
+- A minimum of one `pgbouncer` service node
+
+You also need to take into consideration the underlying network topology,
+making sure you have redundant connectivity between all Database and GitLab instances,
+otherwise the networks will become a single point of failure.
+
+### Architecture
+
+![PG HA Architecture](pg_ha_architecture.png)
+
+Database nodes run two services besides PostgreSQL
+1. Repmgrd -- monitors the cluster and handles failover in case of an issue with the master
+
+   The failover consists of
+   * Selecting a new master for the cluster
+   * Promoting the new node to master
+   * Instructing remaining servers to follow the new master node
+
+   On failure, the old master node is automatically evicted from the cluster, and should be rejoined manually once recovered.
+
+1. Consul -- Monitors the status of each node in the database cluster, and tracks its health in a service definiton on the consul cluster.
 
-**Note**: We're working on a new version that will help automate the setup of a PostgreSQL cluster.
-You can use the [alpha version of the document](alpha_database.md) to try it out now.
+Alongside pgbouncer, there is a consul agent that watches the status of the PostgreSQL service. If that status changes, consul runs a script which updates the configuration and reloads pgbouncer
 
-1. Download/install GitLab Omnibus using **steps 1 and 2** from
-   [GitLab downloads](https://about.gitlab.com/downloads). Do not complete other
-   steps on the download page.
-1. Create/edit `/etc/gitlab/gitlab.rb` and use the following configuration.
-   Be sure to change the `external_url` to match your eventual GitLab front-end
-   URL. If there is a directive listed below that you do not see in the configuration, be sure to add it.
+### Required information
+
+Before proceeding with configuration, you will need to collect all the necessary
+information.
+
+#### Network information
+
+PostgreSQL does not listen on any network interface by default. It needs to know
+which IP address to listen on in order to be accessible to other services.
+Similarly, PostgreSQL access is controlled based on the network source.
+
+This is why you will need:
+
+> IP address of each nodes network interface
+- This can be set to `0.0.0.0` to listen on all interfaces. It cannot
+  be set to the loopack address `127.0.0.1`
+
+> Network Address
+- This can be in subnet (i.e. `192.168.0.0/255.255.255.0`) or CIDR (i.e.
+  `192.168.0.0/24`) form.
+
+#### User information
+
+Various services require different configuration to secure
+the communication as well as information required for running the service.
+Bellow you will find details on each service and the minimum required
+information you need to provide.
+
+##### Consul information
+
+When using default setup, minimum configuration requires:
+
+- `CONSUL_USERNAME`. Defaults to `gitlab-consul`
+- `CONSUL_DATABASE_PASSWORD`. Password for the database user.
+- `CONSUL_PASSWORD_HASH`. This is a hash generated out of consul username/password pair.
+Can be generated with:
+    ```sh
+    echo -n 'CONSUL_DATABASE_PASSWORDCONSUL_USERNAME' | md5sum
+    ```
+- `CONSUL_SERVER_NODES`.  The IP addresses or DNS records of the Consul server nodes.
+
+Few notes on the service itself:
+
+- The service runs under a system account, by default `gitlab-consul`.
+  - If you are using a different username, you will have to specify it. We
+will refer to it with `CONSUL_USERNAME`,
+- There will be a database user created with read only access to the repmgr
+database
+- Passwords will be stored in the following locations:
+  - `/etc/gitlab/gitlab.rb`: hashed
+  - `/var/opt/gitlab/pgbouncer/pg_auth`: hashed
+  - `/var/opt/gitlab/gitlab-consul/.pgpass`: plaintext
+
+##### PostgreSQL information
+
+When configuring PostgreSQL, we will set `max_wal_senders` to one more than
+the number of database nodes in the cluster.
+This is used to prevent replication from using up all of the
+available database connections.
+
+> Note:
+- In this document we are assuming 3 database nodes, which makes this configuration:
+
+```
+postgresql['max_wal_senders'] = 4
+```
+
+As previously mentioned, you'll have to prepare the network subnets that will
+be allowed to authenticate with the database.
+You'll also need to supply the IP addresses or DNS records of Consul
+server nodes.
+
+We will need the following password information for the application's database user:
+
+- `POSTGRESQL_USERNAME`. Defaults to `gitlab`
+- `POSTGRESQL_USER_PASSWORD`. The password for the database user
+- `POSTGRESQL_PASSWORD_HASH`. This is a hash generated out of the username/password pair.
+Can be generated with:
+    ```sh
+    echo -n 'POSTGRESQL_USER_PASSWORDPOSTGRESQL_USERNAME' | md5sum
+    ```
+
+##### Pgbouncer information
+
+When using default setup, minimum configuration requires:
+
+- `PGBOUNCER_USERNAME`. Defaults to `pgbouncer`
+- `PGBOUNCER_PASSWORD`. This is a password for pgbouncer service.
+- `PGBOUNCER_PASSWORD_HASH`. This is a hash generated out of pgbouncer username/password pair.
+Can be generated with:
+    ```sh
+    echo -n 'PGBOUNCER_PASSWORDPGBOUNCER_USERNAME' | md5sum
+    ```
+- `PGBOUNCER_NODE`, is the IP address or a FQDN of the node running Pgbouncer.
+
+Few notes on the service itself:
+
+- The service runs as the same system account as the database
+  - In the package, this is by default `gitlab-psql`
+- If you use a non-default user account for Pgbouncer service (by default `pgbouncer`), you will have to specify this username. We will refer to this requirement with `PGBOUNCER_USERNAME`.
+- The service will have a regular database user account generated for it
+  - This defaults to `repmgr`
+- Passwords will be stored in the following locations:
+  - `/etc/gitlab/gitlab.rb`: hashed, and in plain text
+  - `/var/opt/gitlab/pgbouncer/pg_auth`: hashed
+
+##### Repmgr information
+
+When using default setup, you will only have to prepare the network subnets that will
+be allowed to authenticate with the service.
+
+Few notes on the service itself:
+
+- The service runs under the same system account as the database
+  -  In the package, this is by default `gitlab-psql`
+- The service will have a superuser database user account generated for it
+  - This defaults to `gitlab_repmgr`
+
+### Installing Omnibus GitLab
+
+First, make sure to [download/install](https://about.gitlab.com/installation)
+GitLab Omnibus **on each node**.
+
+Make sure you install the necessary dependencies from step 1,
+add GitLab package repository from step 2.
+When installing the GitLab package, do not supply `EXTERNAL_URL` value.
+
+### Initial node configuration
+
+Each node needs to be configured to run only the services it needs.
+
+#### Configuring the Consul nodes
+
+On each Consul node perform the following:
+
+1. Make sure you collect [`CONSUL_SERVER_NODES`](#consul_information) before executing the next step.
+
+1. Edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section:
 
     ```ruby
-    external_url 'https://gitlab.example.com'
+    # Disable all components except Consul
+    bootstrap['enable'] = false
+    gitlab_rails['auto_migrate'] = false
+    gitaly['enable'] = false
+    gitlab_workhorse['enable'] = false
+    mailroom['enable'] = false
+    nginx['enable'] = false
+    postgresql['enable'] = false
+    redis['enable'] = false
+    sidekiq['enable'] = false
+    prometheus['enable'] = false
+    unicorn['enable'] = false
 
-    # Disable all components except PostgreSQL
-    postgresql['enable'] = true
+    consul['enable'] = true
+    # START user configuration
+    # Replace placeholders:
+    #
+    # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
+    # with the addresses gathered for CONSUL_SERVER_NODES
+    consul['configuration'] = {
+      server: true,
+      retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z)
+    }
+    #
+    # END user configuration
+    ```
+
+1. [Reconfigure GitLab] for the changes to take effect.
+
+After this is completed on each Consul server node, proceed further.
+
+#### Configuring the Database nodes
+
+On each database node perform the following:
+
+1. Make sure you collect [`CONSUL_SERVER_NODES`](#consul_information), [`PGBOUNCER_PASSWORD_HASH`](#pgbouncer_information), [`POSTGRESQL_PASSWORD_HASH`](#postgresql_information), [`Number of db nodes`](#postgresql_information), and [`Network Address`](#network_address) before executing the next step.
+
+1. Edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section:
+
+    ```ruby
+    # Disable all components except PostgreSQL and Repmgr and Consul
     bootstrap['enable'] = false
+    gitaly['enable'] = false
+    mailroom['enable'] = false
     nginx['enable'] = false
     unicorn['enable'] = false
     sidekiq['enable'] = false
     redis['enable'] = false
-    prometheus['enable'] = false
-    gitaly['enable'] = false
     gitlab_workhorse['enable'] = false
-    mailroom['enable'] = false
+    prometheus_monitoring['enable'] = false
+
+    repmgr['enable'] = true
+    postgresql['enable'] = true
+    consul['enable'] = true
 
     # PostgreSQL configuration
-    gitlab_rails['db_password'] = 'DB password'
-    postgresql['md5_auth_cidr_addresses'] = ['0.0.0.0/0']
     postgresql['listen_address'] = '0.0.0.0'
+    postgresql['hot_standby'] = 'on'
+    postgresql['wal_level'] = 'replica'
+    postgresql['shared_preload_libraries'] = 'repmgr_funcs'
 
     # Disable automatic database migrations
     gitlab_rails['auto_migrate'] = false
+
+    # Configure the consul agent
+    consul['services'] = %w(postgresql)
+
+    # START user configuration
+    # Please set the real values as explained in Required Information section
+    #
+    # Replace PGBOUNCER_PASSWORD_HASH with a generated md5 value
+    postgresql['pgbouncer_user_password'] = 'PGBOUNCER_PASSWORD_HASH'
+    # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value
+    postgresql['sql_user_password'] = 'POSTGRESQL_PASSWORD_HASH'
+    # Replace X with value of number of db nodes + 1
+    postgresql['max_wal_senders'] = X
+
+    # Replace XXX.XXX.XXX.XXX/YY with Network Address
+    postgresql['trust_auth_cidr_addresses'] = %w(XXX.XXX.XXX.XXX/YY)
+    repmgr['trust_auth_cidr_addresses'] = %w(XXX.XXX.XXX.XXX/YY)
+
+    # Replace placeholders:
+    #
+    # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
+    # with the addresses gathered for CONSUL_SERVER_NODES
+    consul['configuration'] = {
+      retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z)
+    }
+    #
+    # END user configuration
     ```
 
-1. Run `sudo gitlab-ctl reconfigure` to install and configure PostgreSQL.
+1. [Reconfigure GitLab] for the changes to take effect.
+
+> Please note:
+- If you want your database to listen on a specific interface, change the config:
+`postgresql['listen_address'] = '0.0.0.0'`
+- If your Pgbouncer service runs under a different user account,
+you also need to specify: `postgresql['pgbouncer_user'] = PGBOUNCER_USERNAME` in
+your configuration
+
+#### Configuring the Pgbouncer node
+
+1. Make sure you collect [`CONSUL_SERVER_NODES`](#consul_information), [`CONSUL_PASSWORD_HASH`](#consul_information), and [`PGBOUNCER_PASSWORD_HASH`](#pgbouncer_information) before executing the next step.
+
+1. Edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section:
+
+    ```ruby
+    # Disable all components except Pgbouncer and Consul agent
+    bootstrap['enable'] = false
+    gitaly['enable'] = false
+    mailroom['enable'] = false
+    nginx['enable'] = false
+    redis['enable'] = false
+    prometheus['enable'] = false
+    postgresql['enable'] = false
+    unicorn['enable'] = false
+    sidekiq['enable'] = false
+    gitlab_workhorse['enable'] = false
+    gitlab_rails['auto_migrate'] = false
+
+    pgbouncer['enable'] = true
+    consul['enable'] = true
+
+    # Configure Pgbouncer
+    pgbouncer['admin_users'] = %w(pgbouncer gitlab-consul)
+
+    # Configure Consul agent
+    consul['watchers'] = %w(postgresql)
+
+    # START user configuration
+    # Please set the real values as explained in Required Information section
+    # Replace CONSUL_PASSWORD_HASH with with a generated md5 value
+    # Replace PGBOUNCER_PASSWORD_HASH with with a generated md5 value
+    pgbouncer['users'] = {
+      'gitlab-consul': {
+        password: 'CONSUL_PASSWORD_HASH'
+      },
+      'pgbouncer': {
+        password: 'PGBOUNCER_PASSWORD_HASH'
+      }
+    }
+    # Replace placeholders:
+    #
+    # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
+    # with the addresses gathered for CONSUL_SERVER_NODES
+    consul['configuration'] = {
+      retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z)
+    }
+    #
+    # END user configuration
+    ```
+
+1. [Reconfigure GitLab] for the changes to take effect.
+
+#### Configuring the Application nodes
+
+These will be the nodes running the `gitlab-rails` service. You may have other
+attributes set, but the following need to be set.
+
+1. Edit `/etc/gitlab/gitlab.rb`:
+
+    ```ruby
+    # Disable PostgreSQL on the application node
+    postgresql['enable'] = false
+
+    gitlab_rails['db_host'] = 'PGBOUNCER_NODE'
+    gitlab_rails['db_port'] = 6432
+    gitlab_rails['db_password'] = 'POSTGRESQL_USER_PASSWORD'
+    gitlab_rails['auto_migrate'] = false
+    ```
+
+1. [Reconfigure GitLab] for the changes to take effect.
+
+### Node post-configuration
+
+After reconfigure successfully runs, the following steps must be completed to
+get the cluster up and running.
+
+#### Consul nodes post-configuration
+
+Verify the nodes are all communicating:
 
-    > **Note**: This `reconfigure` step will result in some errors.
-      That's OK - don't be alarmed.
+```sh
+/opt/gitlab/embedded/bin/consul members
+```
+
+The output should be similar to:
+
+```
+Node                 Address               Status  Type    Build  Protocol  DC
+CONSUL_NODE_ONE      XXX.XXX.XXX.YYY:8301  alive   server  0.9.2  2         gitlab_consul
+CONSUL_NODE_TWO      XXX.XXX.XXX.YYY:8301  alive   server  0.9.2  2         gitlab_consul
+CONSUL_NODE_THREE    XXX.XXX.XXX.YYY:8301  alive   server  0.9.2  2         gitlab_consul
+DATABASE_NODE_ONE    XXX.XXX.XXX.YYY:8301  alive   client  0.9.2  2         gitlab_consul
+DATABASE_NODE_TWO    XXX.XXX.XXX.YYY:8301  alive   client  0.9.2  2         gitlab_consul
+DATABASE_NODE_THREE  XXX.XXX.XXX.YYY:8301  alive   client  0.9.2  2         gitlab_consul
+PGBOUNCER_NODE       XXX.XXX.XXX.YYY:8301  alive   client  0.9.0  2         gitlab_consul
+```
+
+#### Database nodes post-configuration
+
+##### Primary node
+
+Select one node as a primary node.
 
 1. Open a database prompt:
 
+    ```sh
+    gitlab-psql -d gitlabhq_production
+    ```
+
+1. Enable the `pg_trgm` extension:
+
+    ```sh
+    CREATE EXTENSION pg_trgm;
     ```
-    su - gitlab-psql
-    /bin/bash
-    psql -h /var/opt/gitlab/postgresql -d template1
 
-    # Output:
+1. Exit the database prompt by typing `\q` and Enter.
 
-    psql (9.2.15)
-    Type "help" for help.
+1. Verify the cluster is initialized with one node:
 
-    template1=#
+     ```sh
+     gitlab-ctl repmgr cluster show
      ```
 
-1. Run the following command at the database prompt and you will be asked to
-   enter the new password for the PostgreSQL superuser.
+     The output should be similar to the following:
 
      ```
-    \password
+     Role      | Name     | Upstream | Connection String
+     ----------+----------|----------|----------------------------------------
+     * master  | HOSTNAME |          | host=HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr
+     ```
+1. Note down the value in the `Name` column. We will refer to it in the next section as `MASTER_NODE_NAME`.
+
+
+##### Secondary nodes
 
-    # Output:
+1. Setup the repmgr standby:
 
-    Enter new password:
-    Enter it again:
+    ```sh
+    gitlab-ctl repmgr standby setup MASTER_NODE_NAME
+    ```
+    Do note that this will remove the existing data on the node. The command
+    has a wait time.
+
+    The output should be similar to the following:
+
+    ```console
+    # gitlab-ctl repmgr standby setup MASTER_NODE_NAME
+    Doing this will delete the entire contents of /var/opt/gitlab/postgresql/data
+    If this is not what you want, hit Ctrl-C now to exit
+    To skip waiting, rerun with the -w option
+    Sleeping for 30 seconds
+    Stopping the database
+    Removing the data
+    Cloning the data
+    Starting the database
+    Registering the node with the cluster
+    ok: run: repmgrd: (pid 19068) 0s
     ```
 
-1. Similarly, set the password for the `gitlab` database user. Use the same
-   password that you specified in the `/etc/gitlab/gitlab.rb` file for
-   `gitlab_rails['db_password']`.
+1. Verify the node now appears in the cluster:
 
+     ```sh
+     gitlab-ctl repmgr cluster show
      ```
-    \password gitlab
 
-    # Output:
+     The output should be similar to the following:
 
-    Enter new password:
-    Enter it again:
      ```
-1. Exit from editing `template1` prompt by typing `\q` and Enter.
-1. Enable the `pg_trgm` extension within the `gitlabhq_production` database:
+     Role      | Name    | Upstream  | Connection String
+     ----------+---------|-----------|------------------------------------------------
+     * master  | MASTER  |           | host=MASTER_NODE_NAME user=gitlab_repmgr dbname=gitlab_repmgr
+       standby | STANDBY | MASTER    | host=STANDBY_HOSTNAME user=gitlab_repmgr dbname=gitlab_repmgr
+     ```
+
+Repeat the above steps on all secondary nodes.
+
+#### Pgbouncer node post-configuration
 
+1. Create a `.pgpass` file user for the `CONSUL_USER` account to be able to
+   reload pgbouncer. Confirm `PGBOUNCER_PASSWORD` twice when asked:
+
+     ```sh
+     gitlab-ctl write-pgpass --host 127.0.0.1 --database pgbouncer --user pgbouncer --hostuser gitlab-consul
      ```
-    gitlab-psql -d gitlabhq_production
 
-    CREATE EXTENSION pg_trgm;
+1. Ensure the node is talking to the current master:
 
-    # Output:
+     ```sh
+     gitlab-ctl pgb-console # You will be prompted for PGBOUNCER_PASSWORD
+     ```
+
+     Then run:
 
-    CREATE EXTENSION
+     ```sh
+     show databases ; show clients ;
      ```
-1. Exit the database prompt by typing `\q` and Enter.
-1. Exit the `gitlab-psql` user by running `exit` twice.
-1. Run `sudo gitlab-ctl reconfigure` a final time.
-1. Configure the GitLab application servers with the appropriate details.
-   This step is covered in [Configuring GitLab for HA](gitlab.md).
+
+     The output should be similar to the following:
+
+     ```
+             name         |  host       | port |      database       | force_user | pool_size | reserve_pool | pool_mode | max_connections | current_connections
+     ---------------------+-------------+------+---------------------+------------+-----------+--------------+-----------+-----------------+---------------------
+      gitlabhq_production | MASTER_HOST | 5432 | gitlabhq_production |            |        20 |            0 |           |               0 |                   0
+      pgbouncer           |             | 6432 | pgbouncer           | pgbouncer  |         2 |            0 | statement |               0 |                   0
+     (2 rows)
+
+      type |   user    |      database       |  state  |   addr         | port  | local_addr | local_port |    connect_time     |    request_time     |    ptr    | link | remote_pid | tls
+     ------+-----------+---------------------+---------+----------------+-------+------------+------------+---------------------+---------------------+-----------+------+------------+-----
+      C    | (nouser)  | gitlabhq_production | waiting | IP_OF_APP_NODE | 56512 | 127.0.0.1  |       6432 | 2017-08-21 18:08:51 | 2017-08-21 18:08:51 | 0x22b3700 |      |          0 |
+      C    | pgbouncer | pgbouncer           | active  | 127.0.0.1      | 56846 | 127.0.0.1  |       6432 | 2017-08-21 18:09:59 | 2017-08-21 18:10:48 | 0x22b3880 |      |          0 |
+     (2 rows)
+     ```
+
+#### Application node post-configuration
+
+Ensure that all migrations ran:
+
+```sh
+gitlab-rake gitlab:db:configure
+```
+
+#### Ensure GitLab is running
+
+At this point, your GitLab instance should be up and running. Verify you are
+able to login, and create issues and merge requests.  If you have troubles check the [Troubleshooting section](#troubleshooting).
+
+### Failover procedure
+
+By default, if the master database fails, `repmgrd` should promote one of the
+standby nodes to master automatically, and consul will update pgbouncer with
+the new master.
+
+If you need to failover manually, you have two options:
+
+**Shutdown the current master database**
+
+Run:
+
+```sh
+gitlab-ctl stop postgresql
+```
+
+The automated failover process will see this and failover to one of the
+standby nodes.
+
+**Or perform a manual failover**
+
+1. Ensure the old master node is not still active.
+1. Login to the server that should become the new master and run:
+
+    ```sh
+    gitlab-ctl repmgr standby promote
+    ```
+
+1. If there are any other standby servers in the cluster, have them follow
+   the new master server:
+
+    ```sh
+    gitlab-ctl repmgr standby follow NEW_MASTER
+    ```
+
+### Restore procedure
+
+If a node fails, it can be removed from the cluster, or added back as a standby
+after it has been restored to service.
+
+- If you want to remove the node from the cluster, on any other node in the
+  cluster, run:
+
+    ```sh
+    gitlab-ctl repmgr standby unregister --node=X
+    ```
+
+    where X is the value of node in `repmgr.conf` on the old server.
+
+    To find this, you can use:
+
+    ```sh
+    awk -F = '$1 == "node" { print $2 }' /var/opt/gitlab/postgresql/repmgr.conf
+    ```
+
+    It will output something like:
+
+    ```
+    959789412
+    ```
+
+    Then you will use this id to unregister the node:
+
+    ```sh
+    gitlab-ctl repmgr standby unregister --node=959789412
+    ```
+
+- To add the node as a standby server:
+
+    ```sh
+    gitlab-ctl repmgr standby follow NEW_MASTER
+    gitlab-ctl restart repmgrd
+    ```
+
+    CAUTION: **Warning:** When the server is brought back online, and before
+    you switch it to a standby node, repmgr will report that there are two masters.
+    If there are any clients that are still attempting to write to the old master,
+    this will cause a split, and the old master will need to be resynced from
+    scratch by performing a `standby setup NEW_MASTER`.
+
+### Alternate configurations
+
+#### Database authorization
+
+By default, we give any host on the database network the permission to perform
+repmgr operations using PostgreSQL's `trust` method. If you do not want this
+level of trust, there are alternatives.
+
+You can trust only the specific nodes that will be database clusters, or you
+can require md5 authentication.
+
+##### Trust specific addresses
+
+If you know the IP address, or FQDN of all database and pgbouncer nodes in the
+cluster, you can trust only those nodes.
+
+In `/etc/gitlab/gitlab.rb` on all of the database nodes, set
+`repmgr['trust_auth_cidr_addresses']` to an array of strings containing all of
+the addresses.
+
+If setting to a node's FQDN, they must have a corresponding PTR record in DNS.
+If setting to a node's IP address, specify it as `XXX.XXX.XXX.XXX/32`.
+
+For example:
+
+```ruby
+repmgr['trust_auth_cidr_addresses'] = %w(192.168.1.44/32 db2.example.com)
+```
+
+
+##### MD5 Authentication
+
+If you are running on an untrusted network, repmgr can use md5 authentication
+with a [.pgpass file](https://www.postgresql.org/docs/9.6/static/libpq-pgpass.html)
+to authenticate.
+
+You can specify by IP address, FQDN, or by subnet, using the same format as in
+the previous section:
+
+1. On the current master node, create a password for the `gitlab` and
+   `gitlab_repmgr` user:
+
+    ```sh
+    gitlab-psql -d template1
+    template1=# \password gitlab_repmgr
+    Enter password: ****
+    Confirm password: ****
+    template1=# \password gitlab
+    ```
+
+1. On each database node:
+
+  1. Edit `/etc/gitlab/gitlab.rb`:
+    1. Ensure `repmgr['trust_auth_cidr_addresses']` is **not** set
+    1. Set `postgresql['md5_auth_cidr_addresses']` to the desired value
+    1. Set `postgresql['sql_replication_user'] = 'gitlab_repmgr'`
+    1. Reconfigure with `gitlab-ctl reconfigure`
+    1. Restart postgresql with `gitlab-ctl restart postgresql`
+
+  1. Create a `.pgpass` file. Enter the `gitlab_repmgr` password twice to
+     when asked:
+
+        ```sh
+        gitlab-ctl write-pgpass --user gitlab_repmgr --hostuser gitlab-psql --database '*'
+        ```
+
+1. On each pgbouncer node, edit `/etc/gitlab/gitlab.rb`:
+  1. Ensure `gitlab_rails['db_password']` is set to the plaintext password for
+     the `gitlab` database user
+  1. [Reconfigure GitLab] for the changes to take effect
+
+### Troubleshooting
+
+#### Consul and PostgreSQL changes not taking effect.
+
+Due to the potential impacts, `gitlab-ctl reconfigure` only reloads Consul and PostgreSQL, it will not restart the services. However, not all changes can be activated by reloading.
+
+To restart either service, run `gitlab-ctl restart SERVICE`
+
+For PostgreSQL, it is usually safe to restart the master node by default. Automatic failover defaults to a 1 minute timeout. Provided the database returns before then, nothing else needs to be done. To be safe, you can stop `repmgrd` on the standby nodes first with `gitlab-ctl stop repmgrd`, then start afterwards with `gitlab-ctl start repmgrd`.
+
+On the consul server nodes, it is important to restart the consul service in a controlled fashion. Read our [consul documentation](consul.md#restarting-the-server-cluster) for instructions on how to restart the service.
+
+#### Issues with other components
+
+If you're running into an issue with a component not outlined here, be sure to check the troubleshooting section of their specific documentation page.
+
+- [Consul](consul.md#troubleshooting)
+- [PostgreSQL](http://docs.gitlab.com/omnibus/settings/database.html#troubleshooting)
+- [GitLab application](gitlab.md#troubleshooting)
+
+## Configure using Omnibus
+
+**Note**: We recommend that you follow the instructions here for a full [PostgreSQL cluster](#configure_using_omnibus_for_high_availability).
+If you are reading this section due to an old bookmark, you can find that old documentation [in the repository](https://gitlab.com/gitlab-org/gitlab-ce/blob/v10.1.4/doc/administration/high_availability/database.md#configure-using-omnibus).
 
 ---
 
@@ -126,3 +723,6 @@ Read more on high-availability configuration:
 1. [Configure NFS](nfs.md)
 1. [Configure the GitLab application servers](gitlab.md)
 1. [Configure the load balancers](load_balancer.md)
+1. [Manage the bundled Consul cluster](consul.md)
+
+[reconfigure GitLab]: ../restart_gitlab.md#omnibus-gitlab-reconfigure

doc/gitlab-geo/troubleshooting.md

@@ -65,6 +65,27 @@ where you have to fix (all commands and path locations are for Omnibus installs)
     secondary by that name. You may want to rerun the [replication
     process](database.md) on the secondary.
 
+- Very large repositories never successfully synchronize on the secondary.
+
+  - GitLab places a timeout on all repository clones, including project imports
+    and Geo synchronization operations. If a fresh `git clone` of a repository
+    on the primary takes more than a few minutes, you may be affected by this.
+    To increase the timeout, add the following line to `/etc/gitlab/gitlab.rb`
+    on the secondary:
+
+        ```ruby
+        gitlab_rails['gitlab_shell_git_timeout'] = 10800
+        ```
+
+        Then reconfigure GitLab:
+
+        ```
+        sudo gitlab-ctl reconfigure
+        ```
+
+    This will increase the timeout to three hours (10800 seconds). Choose a time
+    long enough to accomodate a full clone of your largest repositories.
+
 Visit the primary node's **Admin Area ➔ Geo Nodes** (`/admin/geo_nodes`) in
 your browser. We perform the following health checks on each secondary node
 to help identify if something is wrong:

ee/app/controllers/ee/admin/application_controller.rb

-module EE
-  module Admin
-    module ApplicationController
-      def read_only_message
-        raise NotImplementedError unless defined?(super)
-
-        return super unless Gitlab::Geo.secondary_with_primary?
-
-        link_to_primary_node = view_context.link_to('primary node', Gitlab::Geo.primary_node.url)
-        (_('You are on a read-only GitLab instance. If you want to make any changes, you must visit the %{link_to_primary_node}.') % { link_to_primary_node: link_to_primary_node }).html_safe
-      end
-    end
-  end
-end

ee/app/controllers/groups/epics_controller.rb

@@ -77,6 +77,10 @@ class Groups::EpicsController < Groups::ApplicationController
     @collection_type ||= 'Epic'
   end
 
+  # we don't support custom sorting for epics and therefore don't want to use the issuable_sort cookie
+  def set_sort_order_from_cookie
+  end
+
   def preload_for_collection
     @preload_for_collection ||= [:group, :author]
   end

ee/app/models/license.rb

@@ -84,6 +84,7 @@ class License < ActiveRecord::Base
     push_rules
     related_issues
     repository_mirrors
+    scoped_issue_board
     service_desk
     variable_environment_scope
   ].freeze

lib/api/notes.rb

@@ -33,7 +33,7 @@ module API
               # paginate() only works with a relation. This could lead to a
               # mismatch between the pagination headers info and the actual notes
               # array returned, but this is really a edge-case.
-              paginate(noteable.notes)
+              paginate(noteable.notes.with_metadata)
               .reject { |n| n.cross_reference_not_visible_for?(current_user) }
             present notes, with: Entities::Note
           else
@@ -50,7 +50,7 @@ module API
         end
         get ":id/#{noteables_str}/:noteable_id/notes/:note_id" do
           noteable = find_project_noteable(noteables_str, params[:noteable_id])
-          note = noteable.notes.find(params[:note_id])
+          note = noteable.notes.with_metadata.find(params[:note_id])
           can_read_note = can?(current_user, noteable_read_ability_name(noteable), noteable) && !note.cross_reference_not_visible_for?(current_user)
 
           if can_read_note

lib/gitlab/ee_compat_check.rb

@@ -31,16 +31,22 @@ module Gitlab
 
     def check
       ensure_patches_dir
-      generate_patch(ce_branch, ce_patch_full_path)
+      add_remote('canonical-ce', "#{DEFAULT_CE_PROJECT_URL}.git")
+      generate_patch(branch: ce_branch, patch_path: ce_patch_full_path, remote: 'canonical-ce')
 
       ensure_ee_repo
       Dir.chdir(ee_repo_dir) do
         step("In the #{ee_repo_dir} directory")
 
+        add_remote('canonical-ee', EE_REPO_URL)
+
         status = catch(:halt_check) do
           ce_branch_compat_check!
           delete_ee_branches_locally!
           ee_branch_presence_check!
+
+          step("Checking out #{ee_branch_found}", %W[git checkout -b #{ee_branch_found} canonical-ee/#{ee_branch_found}])
+          generate_patch(branch: ee_branch_found, patch_path: ee_patch_full_path, remote: 'canonical-ee')
           ee_branch_compat_check!
         end
 
@@ -56,6 +62,13 @@ module Gitlab
 
     private
 
+    def add_remote(name, url)
+      step(
+        "Adding the #{name} remote (#{url})",
+        %W[git remote add #{name} #{url}]
+      )
+    end
+
     def ensure_ee_repo
       if Dir.exist?(ee_repo_dir)
         step("#{ee_repo_dir} already exists")
@@ -71,14 +84,14 @@ module Gitlab
       FileUtils.mkdir_p(patches_dir)
     end
 
-    def generate_patch(branch, patch_path)
+    def generate_patch(branch:, patch_path:, remote:)
       FileUtils.rm(patch_path, force: true)
 
-      find_merge_base_with_master(branch: branch)
+      find_merge_base_with_master(branch: branch, master_remote: remote)
 
       step(
-        "Generating the patch against origin/master in #{patch_path}",
-        %w[git diff --binary origin/master...HEAD]
+        "Generating the patch against #{remote}/master in #{patch_path}",
+        %W[git diff --binary #{remote}/master...#{branch}]
       ) do |output, status|
         throw(:halt_check, :ko) unless status.zero?
 
@@ -89,21 +102,21 @@ module Gitlab
     end
 
     def ce_branch_compat_check!
-      if check_patch(ce_patch_full_path).zero?
+      if check_patch(ce_patch_full_path, remote: 'canonical-ce').zero?
         puts applies_cleanly_msg(ce_branch)
         throw(:halt_check)
       end
     end
 
     def ee_branch_presence_check!
-      _, status = step("Fetching origin/#{ee_branch_prefix}", %W[git fetch origin #{ee_branch_prefix}])
+      _, status = step("Fetching origin/#{ee_branch_prefix}", %W[git fetch canonical-ee #{ee_branch_prefix}])
 
       if status.zero?
         @ee_branch_found = ee_branch_prefix
         return
       end
 
-      _, status = step("Fetching origin/#{ee_branch_suffix}", %W[git fetch origin #{ee_branch_suffix}])
+      _, status = step("Fetching origin/#{ee_branch_suffix}", %W[git fetch canonical-ee #{ee_branch_suffix}])
 
       if status.zero?
         @ee_branch_found = ee_branch_suffix
@@ -116,11 +129,7 @@ module Gitlab
     end
 
     def ee_branch_compat_check!
-      step("Checking out origin/#{ee_branch_found}", %W[git checkout -b #{ee_branch_found} FETCH_HEAD])
-
-      generate_patch(ee_branch_found, ee_patch_full_path)
-
-      unless check_patch(ee_patch_full_path).zero?
+      unless check_patch(ee_patch_full_path, remote: 'canonical-ee').zero?
         puts
         puts ee_branch_doesnt_apply_cleanly_msg
 
@@ -131,10 +140,9 @@ module Gitlab
       puts applies_cleanly_msg(ee_branch_found)
     end
 
-    def check_patch(patch_path)
+    def check_patch(patch_path, remote:)
       step("Checking out master", %w[git checkout master])
-      step("Resetting to latest master", %w[git reset --hard origin/master])
-      step("Fetching CE/#{ce_branch}", %W[git fetch #{ce_repo_url} #{ce_branch}])
+      step("Resetting to latest master", %W[git reset --hard #{remote}/master])
       step(
         "Checking if #{patch_path} applies cleanly to EE/master",
         # Don't use --check here because it can result in a 0-exit status even
@@ -171,10 +179,10 @@ module Gitlab
       command(%W[git branch --delete --force #{ee_branch_suffix}])
     end
 
-    def merge_base_found?
+    def merge_base_found?(master_remote:, branch:)
       step(
-        "Finding merge base with master",
-        %w[git merge-base origin/master HEAD]
+        "Finding merge base with #{master_remote}/master",
+        %W[git merge-base #{master_remote}/master #{branch}]
       ) do |output, status|
         if status.zero?
           puts "Merge base was found: #{output}"
@@ -183,7 +191,7 @@ module Gitlab
       end
     end
 
-    def find_merge_base_with_master(branch:)
+    def find_merge_base_with_master(branch:, master_remote:)
       # Start with (Math.exp(3).to_i = 20) until (Math.exp(6).to_i = 403)
       # In total we go (20 + 54 + 148 + 403 = 625) commits deeper
       depth = 20
@@ -192,19 +200,19 @@ module Gitlab
           depth += Math.exp(factor).to_i
           # Repository is initially cloned with a depth of 20 so we need to fetch
           # deeper in the case the branch has more than 20 commits on top of master
-          fetch(branch: branch, depth: depth)
-          fetch(branch: 'master', depth: depth, remote: DEFAULT_CE_PROJECT_URL)
+          fetch(branch: branch, depth: depth, remote: 'origin')
+          fetch(branch: 'master', depth: depth, remote: master_remote)
 
-          merge_base_found?
+          merge_base_found?(master_remote: master_remote, branch: branch)
         end
 
-      raise "\n#{branch} is too far behind master, please rebase it!\n" unless success
+      raise "\n#{branch} is too far behind #{master_remote}/master, please rebase it!\n" unless success
     end
 
     def fetch(branch:, depth:, remote: 'origin')
       step(
         "Fetching deeper...",
-        %W[git fetch --depth=#{depth} --prune #{remote} +refs/heads/#{branch}:refs/remotes/origin/#{branch}]
+        %W[git fetch --depth=#{depth} --prune #{remote} +refs/heads/#{branch}:refs/remotes/#{remote}/#{branch}]
       ) do |output, status|
         raise "Fetch failed: #{output}" unless status.zero?
       end
@@ -304,8 +312,8 @@ module Gitlab
         1. Create a new branch from master and cherry-pick your CE commits
 
           # In the EE repo
-          $ git fetch origin
-          $ git checkout -b #{ee_branch_prefix} origin/master
+          $ git fetch #{EE_REPO_URL} master
+          $ git checkout -b #{ee_branch_prefix} FETCH_HEAD
           $ git fetch #{ce_repo_url} #{ce_branch}
           $ git cherry-pick SHA # Repeat for all the commits you want to pick
 
@@ -314,10 +322,9 @@ module Gitlab
         2. Apply your branch's patch to EE
 
           # In the EE repo
-          $ git fetch origin master
-          $ git checkout -b #{ee_branch_prefix} origin/master
-          $ wget #{patch_url}
-          $ git apply --3way #{ce_patch_name}
+          $ git fetch #{EE_REPO_URL} master
+          $ git checkout -b #{ee_branch_prefix} FETCH_HEAD
+          $ wget #{patch_url} && git apply --3way #{ce_patch_name}
 
           At this point you might have conflicts such as:
 

spec/models/note_spec.rb

@@ -231,6 +231,37 @@ describe Note do
     end
   end
 
+  describe '#cross_reference?' do
+    it 'falsey for user-generated notes' do
+      note = create(:note, system: false)
+
+      expect(note.cross_reference?).to be_falsy
+    end
+
+    context 'when the note might contain cross references' do
+      SystemNoteMetadata::TYPES_WITH_CROSS_REFERENCES.each do |type|
+        let(:note) { create(:note, :system) }
+        let!(:metadata) { create(:system_note_metadata, note: note, action: type) }
+
+        it 'delegates to the cross-reference regex' do
+          expect(note).to receive(:matches_cross_reference_regex?).and_return(false)
+
+          note.cross_reference?
+        end
+      end
+    end
+
+    context 'when the note cannot contain cross references' do
+      let(:commit_note) { build(:note, note: 'mentioned in 1312312313 something else.', system: true) }
+      let(:label_note) { build(:note, note: 'added ~2323232323', system: true) }
+
+      it 'scan for a `mentioned in` prefix' do
+        expect(commit_note.cross_reference?).to be_truthy
+        expect(label_note.cross_reference?).to be_falsy
+      end
+    end
+  end
+
   describe 'clear_blank_line_code!' do
     it 'clears a blank line code before validation' do
       note = build(:note, line_code: ' ')

spec/services/geo/repository_sync_service_spec.rb

@@ -178,7 +178,10 @@ describe Geo::RepositorySyncService do
 
         subject.execute
 
-        expect(File.directory?("#{project.repository.path}+failed-geo-sync")).to be false
+        # gitlab-shell always appends .git to the end of the repository, so
+        # we're relying on the fact that projects can't contain + in the name
+        deleted_dir = File.join(project.repository.storage_path, project.path) + "+failed-geo-sync.git"
+        expect(File.directory?(deleted_dir)).to be false
         expect(File.directory?(project.repository.path)).to be true
       end