Merge branch 'ce-to-ee-2018-10-09' into 'master'

CE upstream - 2018-10-09 13:02 UTC

Closes #7883

See merge request gitlab-org/gitlab-ee!7846

CONTRIBUTING.md

@@ -92,7 +92,7 @@ Please report suspected security vulnerabilities in private to
 Please do **NOT** create publicly viewable issues for suspected security
 vulnerabilities.
 
-## Code of conduct
+## Code of Conduct
 
 ### Our Pledge
 

app/controllers/projects/compare_controller.rb

@@ -16,6 +16,8 @@ class Projects::CompareController < Projects::ApplicationController
   before_action :define_diff_notes_disabled, only: [:show, :diff_for_path]
   before_action :define_commits, only: [:show, :diff_for_path, :signatures]
   before_action :merge_request, only: [:index, :show]
+  # Validation
+  before_action :validate_refs!
 
   def index
   end
@@ -63,6 +65,21 @@ class Projects::CompareController < Projects::ApplicationController
 
   private
 
+  def valid_ref?(ref_name)
+    return true unless ref_name.present?
+
+    Gitlab::GitRefValidator.validate(ref_name)
+  end
+
+  def validate_refs!
+    valid = [head_ref, start_ref].map { |ref| valid_ref?(ref) }
+
+    return if valid.all?
+
+    flash[:alert] = "Invalid branch name"
+    redirect_to project_compare_index_path(@project)
+  end
+
   def compare
     return @compare if defined?(@compare)
 

changelogs/unreleased/52408-pip-cache-dir-to-cache-python-dependencies.yml

+---
+title: Use the standard PIP_CACHE_DIR for Python dependency caching template
+merge_request: 22211
+author: Takuya Noguchi
+type: fixed

changelogs/unreleased/rails5-user-status-spec.yml

+---
+title: 'Rails5: fix user edit profile clear status spec'
+merge_request: 22169
+author: Jasper Maes
+type: other

changelogs/unreleased/sh-handle-invalid-comparison.yml

+---
+title: Reject invalid branch names in repository compare controller
+merge_request: 22186
+author:
+type: fixed

doc/ci/caching/index.md

@@ -253,7 +253,7 @@ image: python:latest
 # Change pip's cache directory to be inside the project directory since we can
 # only cache local items.
 variables:
-    PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache"
+    PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
 
 # Pip's cache doesn't store the python packages
 # https://pip.pypa.io/en/stable/reference/pip_install/#caching
@@ -262,7 +262,7 @@ variables:
 # them in a virtualenv and cache it as well.
 cache:
   paths:
-    - .cache/
+    - .cache/pip
     - venv/
 
 before_script:

doc/development/documentation/structure.md

@@ -21,21 +21,21 @@ Before getting started, read through the following docs:
 Every document should include the following content in the following sequence:
 
 - **Feature name**: defines an intuitive name for the feature that clearly
-states what it is and is consistent with any relevant UI text.
+  states what it is and is consistent with any relevant UI text.
 - **Feature overview** and description: describe what it is, what it does, and in what context it should be used.
 - **Use cases**: describes real use case scenarios for that feature.
 - **Requirements**: describes what software and/or configuration is required to be able to
-use the feature and, if applicable, prerequisite knowledge for being able to follow/implement the tutorial.
-For example, familiarity with GitLab CI/CD, an account on a third-party service, dependencies installed, etc.
-Link each one to its most relevant resource; i.e., where the reader can go to begin to fullfil that requirement.
-(Another doc page, a third party application's site, etc.)
+  use the feature and, if applicable, prerequisite knowledge for being able to follow/implement the tutorial.
+  For example, familiarity with GitLab CI/CD, an account on a third-party service, dependencies installed, etc.
+  Link each one to its most relevant resource; i.e., where the reader can go to begin to fullfil that requirement.
+  (Another doc page, a third party application's site, etc.)
 - **Instructions**: clearly describes the steps to use the feature, leaving no gaps.
 - **Troubleshooting** guide (recommended but not required): if you know beforehand what issues
-one might have when setting it up, or when something is changed, or on upgrading, it's
-important to describe those too. Think of things that may go wrong and include them in the
-docs. This is important to minimize requests for support, and to avoid doc comments with
-questions that you know someone might ask. Answering them beforehand only makes your
-document better and more approachable.
+  one might have when setting it up, or when something is changed, or on upgrading, it's
+  important to describe those too. Think of things that may go wrong and include them in the
+  docs. This is important to minimize requests for support, and to avoid doc comments with
+  questions that you know someone might ask. Answering them beforehand only makes your
+  document better and more approachable.
 
 For additional details, see the subsections below, as well as the [Documentation template for new docs](#Documentation-template-for-new-docs).
 
@@ -55,10 +55,11 @@ You should answer this question: what can you do with this feature/change? Use c
 are examples of how this feature or change can be used in real life.
 
 Examples:
-- CE and EE: [Issues](../user/project/issues/index.md#use-cases)
-- CE and EE: [Merge Requests](../user/project/merge_requests/index.md#overview)
-- EE-only: [Geo](https://docs.gitlab.com/ee/gitlab-geo/README.html#overview)
-- EE-only: [Jenkins integration](https://docs.gitlab.com/ee/integration/jenkins.md#overview)
+
+- CE and EE: [Issues](../../user/project/issues/index.md#use-cases)
+- CE and EE: [Merge Requests](../../user/project/merge_requests/index.md)
+- EE-only: [Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html)
+- EE-only: [Jenkins integration](https://docs.gitlab.com/ee/integration/jenkins.html)
 
 Note that if you don't have anything to add between the doc title (`<h1>`) and
 the header `## Overview`, you can omit the header, but keep the content of the
@@ -72,14 +73,14 @@ and for every **major** feature present in Community Edition.
 Your new document will be discoverable by the user only if:
 
 - Crosslinked from the higher-level index (e.g., Issue Boards docs
-should be linked from Issues; Prometheus docs should be linked from
-Monitoring; CI/CD tutorials should be linked from CI/CD examples).
+  should be linked from Issues; Prometheus docs should be linked from
+  Monitoring; CI/CD tutorials should be linked from CI/CD examples).
   - When referencing other GitLab products and features, link to their
-respective docs; when referencing third-party products or technologies,
-link out to their external sites, documentation, and resources.
+  respective docs; when referencing third-party products or technologies,
+  link out to their external sites, documentation, and resources.
 - The headings are clear. E.g., "App testing" is a bad heading, "Testing
-an application with GitLab CI/CD" is much better. Think of something
-someone will search for and use these keywords in the headings.
+  an application with GitLab CI/CD" is much better. Think of something
+  someone will search for and use these keywords in the headings.
 
 ## Documentation template for new docs
 
@@ -133,7 +134,7 @@ is simple and the document is short.
 - Be clear, concise, and stick to the goal of the doc: explain how to
 use that feature.
 - Use inclusive language and avoid jargons, as well as uncommon and
-fancy words. The docs should be clear and very easy to understand.
+fancy words. The docs should be clear and easy to understand.
 - Write in the 3rd person (use "we", "you", "us", "one", instead of "I" or "me").
 - Always provide internal and external reference links.
 - Always link the doc from its higher-level index.

doc/development/rolling_out_changes_using_feature_flags.md

@@ -151,3 +151,27 @@ most cases this will translate to a feature (with a feature flag) being shipped
 in RC1, followed by the feature flag being removed in RC2. This in turn means
 the feature will be stable by the time we publish a stable package around the
 22nd of the month.
+
+## Undefined feature flags default to "on"
+
+By default, the [`Project#feature_available?`][project-fa],
+[`Namespace#feature_available?`][namespace-fa] (EE), and
+[`License.feature_available?`][license-fa] (EE) methods will check if the
+specified feature is behind a feature flag. Unless the feature is explicitly
+disabled or limited to a percentage of users, the feature flag check will
+default to `true`.
+
+As an example, if you were to ship the backend half of a feature behind a flag,
+you'd want to explicitly disable that flag until the frontend half is also ready
+to be shipped. You can do this via ChatOps:
+
+```
+/chatops run feature set some_feature 0
+```
+
+Note that you can do this at any time, even before the merge request using the
+flag has been merged!
+
+[project-fa]: https://gitlab.com/gitlab-org/gitlab-ee/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/app/models/project_feature.rb#L63-68
+[namespace-fa]: https://gitlab.com/gitlab-org/gitlab-ee/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/ee/namespace.rb#L71-85
+[license-fa]: https://gitlab.com/gitlab-org/gitlab-ee/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/license.rb#L293-300

doc/user/markdown.md

@@ -113,8 +113,8 @@ GFM will autolink almost any URL you copy and paste into your text:
 * https://www.google.com
 * https://google.com/
 * ftp://ftp.us.debian.org/debian/
-* smb://foo/bar/baz
-* irc://irc.freenode.net/gitlab
+* <a href="smb://foo/bar/baz">smb://foo/bar/baz</a>
+* <a href="irc://irc.freenode.net/gitlab">irc://irc.freenode.net/gitlab</a>
 * http://localhost:3000
 
 ### Multiline Blockquote
@@ -139,17 +139,13 @@ you can quote that without having to manually prepend `>` to every line!
 >>>
 ```
 
->>>
-If you paste a message from somewhere else
-
-that
-
-spans
-
-multiple lines,
-
-you can quote that without having to manually prepend `>` to every line!
->>>
+<blockquote dir="auto">
+<p>If you paste a message from somewhere else</p>
+<p>that</p>
+<p>spans</p>
+<p>multiple lines,</p>
+<p>you can quote that without having to manually prepend <code>&gt;</code> to every line!</p>
+</blockquote>
 
 ### Code and Syntax Highlighting
 
@@ -270,15 +266,15 @@ https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#emoji
 	Ubuntu 18.04 (like many modern Linux distros) has this font installed by default.
 
 
-Sometimes you want to :monkey: around a bit and add some :star2: to your :speech_balloon:. Well we have a gift for you:
+Sometimes you want to <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/monkey.png" width="20px" height="20px"> around a bit and add some <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/star2.png" width="20px" height="20px"> to your <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/speech_balloon.png" width="20px" height="20px">. Well we have a gift for you:
 
-:zap: You can use emoji anywhere GFM is supported. :v:
+<img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/zap.png" width="20px" height="20px">You can use emoji anywhere GFM is supported. <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/v.png" width="20px" height="20px">
 
-You can use it to point out a :bug: or warn about :speak_no_evil: patches. And if someone improves your really :snail: code, send them some :birthday:. People will :heart: you for that.
+You can use it to point out a <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/bug.png" width="20px" height="20px"> or warn about <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/speak_no_evil.png" width="20px" height="20px"> patches. And if someone improves your really <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/snail.png" width="20px" height="20px"> code, send them some <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/birthday.png" width="20px" height="20px">. People will <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/heart.png" width="20px" height="20px"> you for that.
 
-If you are new to this, don't be :fearful:. You can easily join the emoji :family:. All you need to do is to look up one of the supported codes.
+If you are new to this, don't be <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/fearful.png" width="20px" height="20px">. You can easily join the emoji <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/family.png" width="20px" height="20px">. All you need to do is to look up one of the supported codes.
 
-Consult the [Emoji Cheat Sheet](https://www.emojicopy.com) for a list of all supported emoji codes. :thumbsup:
+Consult the [Emoji Cheat Sheet](https://www.emojicopy.com) for a list of all supported emoji codes. <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/thumbsup.png" width="20px" height="20px">
 
 Most emoji are natively supported on macOS, Windows, iOS, Android and will fallback to image-based emoji where there is lack of support.
 
@@ -287,7 +283,6 @@ On Linux, you can download [Noto Color Emoji](https://www.google.com/get/noto/he
 Ubuntu 18.04 (like many modern Linux distros) has this font installed by default.
 
 
-
 ### Special GitLab References
 
 GFM recognizes special references.
@@ -359,11 +354,7 @@ You can add task lists to issues, merge requests and comments. To create a task
     - [ ] Sub-task 3
 ```
 
-- [x] Completed task
-- [ ] Incomplete task
-    - [ ] Sub-task 1
-    - [x] Sub-task 2
-    - [ ] Sub-task 3
+![alt unordered-check-list-render-gfm](img/unordered_check_list_render_gfm.png)
 
 Tasks formatted as ordered lists are supported as well:
 
@@ -374,10 +365,7 @@ Tasks formatted as ordered lists are supported as well:
     1. [x] Sub-task 2
 ```
 
-1. [x] Completed task
-1. [ ] Incomplete task
-    1. [ ] Sub-task 1
-    1. [x] Sub-task 2
+![alt task-list-ordered-render-gfm](img/task_list_ordered_render_gfm.png)
 
 Task lists can only be created in descriptions, not in titles. Task item state can be managed by editing the description's Markdown or by toggling the rendered check boxes.
 
@@ -396,7 +384,10 @@ The valid video extensions are `.mp4`, `.m4v`, `.mov`, `.webm`, and `.ogv`.
 
 Here's a sample video:
 
-![Sample Video](img/markdown_video.mp4)
+<div class="video-container">
+   <video src="img/markdown_video.mp4" width="400" controls="true" data-setup="{}" data-title="Sample Video"></video>
+   <p><a href="img/markdown_video.mp4" target="_blank" rel="noopener noreferrer" title="Download 'Sample Video'">Sample Video</a></p>
+</div>
 
 ### Math
 
@@ -420,12 +411,11 @@ Example:
 
 Becomes:
 
-This math is inline $`a^2+b^2=c^2`$.
+This math is inline ![alt text](img/math_inline_sup_render_gfm.png).
 
 This is on a separate line
-```math
-a^2+b^2=c^2
-```
+
+<div align="center"><img src="./img/math_inline_sup_render_gfm.png" ></div>
 
 _Be advised that KaTeX only supports a [subset][katex-subset] of LaTeX._
 
@@ -455,15 +445,7 @@ Examples:
 
 Become:
 
-`#F00`  
-`#F00A`  
-`#FF0000`  
-`#FF0000AA`  
-`RGB(0,255,0)`  
-`RGB(0%,100%,0%)`  
-`RGBA(0,255,0,0.7)`  
-`HSL(540,70%,50%)`  
-`HSLA(540,70%,50%,0.7)`
+![alt color-inline-colorchip-render-gfm](img/color_inline_colorchip_render_gfm.png)
 
 #### Supported formats:
 
@@ -495,13 +477,7 @@ Example:
 
 Becomes:
 
-```mermaid
-graph TD;
-  A-->B;
-  A-->C;
-  B-->D;
-  C-->D;
-```
+<img src="./img/mermaid_diagram_render_gfm.png" width="200px" height="400px">
 
 For details see the [Mermaid official page][mermaid].
 

lib/gitlab/ci/templates/Python.gitlab-ci.yml

@@ -5,7 +5,7 @@ image: python:latest
 # Change pip's cache directory to be inside the project directory since we can
 # only cache local items.
 variables:
-  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache"
+  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
 
 # Pip's cache doesn't store the python packages
 # https://pip.pypa.io/en/stable/reference/pip_install/#caching

qa/qa/service/kubernetes_cluster.rb

@@ -25,6 +25,7 @@ module QA
           gcloud container clusters
           create #{cluster_name}
           #{auth_options}
+          --enable-basic-auth
           --zone #{Runtime::Env.gcloud_zone}
           && gcloud container clusters
           get-credentials
@@ -33,6 +34,15 @@ module QA
         CMD
 
         @api_url = `kubectl config view --minify -o jsonpath='{.clusters[].cluster.server}'`
+
+        @admin_user = "#{cluster_name}-admin"
+        master_auth = JSON.parse(`gcloud container clusters describe #{cluster_name} --zone #{Runtime::Env.gcloud_zone} --format 'json(masterAuth.username, masterAuth.password)'`)
+        shell <<~CMD.tr("\n", ' ')
+          kubectl config set-credentials #{@admin_user}
+          --username #{master_auth['masterAuth']['username']}
+          --password #{master_auth['masterAuth']['password']}
+        CMD
+
         if rbac
           create_service_account
 
@@ -64,7 +74,7 @@ module QA
 
       def create_service_account
         shell('kubectl create -f -', stdin_data: service_account)
-        shell('kubectl create -f -', stdin_data: service_account_role_binding)
+        shell("kubectl --user #{@admin_user} create -f -", stdin_data: service_account_role_binding)
       end
 
       def service_account

spec/controllers/projects/compare_controller_spec.rb

@@ -97,6 +97,30 @@ describe Projects::CompareController do
         expect(assigns(:commits)).to eq([])
       end
     end
+
+    context 'when the target ref is invalid' do
+      let(:target_ref) { "master%' AND 2554=4423 AND '%'='" }
+      let(:source_ref) { "improve%2Fawesome" }
+
+      it 'shows a flash message and redirects' do
+        show_request
+
+        expect(flash[:alert]).to eq('Invalid branch name')
+        expect(response).to have_http_status(302)
+      end
+    end
+
+    context 'when the source ref is invalid' do
+      let(:source_ref) { "master%' AND 2554=4423 AND '%'='" }
+      let(:target_ref) { "improve%2Fawesome" }
+
+      it 'shows a flash message and redirects' do
+        show_request
+
+        expect(flash[:alert]).to eq('Invalid branch name')
+        expect(response).to have_http_status(302)
+      end
+    end
   end
 
   describe 'GET diff_for_path' do

spec/features/profiles/user_edit_profile_spec.rb

@@ -129,6 +129,8 @@ describe 'User edit profile' do
         click_button 'js-clear-user-status-button'
         submit_settings
 
+        wait_for_requests
+
         visit user_path(user)
         expect(page).not_to have_selector '.cover-status'
       end