Merge branch 'master' into 25990-web-terminal-improvements

* master:
  Add introduced section
  Copyedit Bitbucket docs
  Docs: fix ee-conflict in bulk editing
  Add Bitbucket Server importer docs
  Backport of gitlab-org/gitlab-ee!6848
  Update doc/integration/oauth_provider.md
  Fix Bitbucket Cloud importer omitting replies

changelogs/unreleased/sh-fix-bitbucket-cloud-importer-replies.yml

+---
+title: Fix Bitbucket Cloud importer omitting replies
+merge_request: 21076
+author:
+type: fixed

doc/integration/bitbucket.md

-# Integrate your GitLab server with Bitbucket
+# Integrate your GitLab server with Bitbucket Cloud
 
 NOTE: **Note:**
 You need to [enable OmniAuth](omniauth.md) in order to use this.

doc/integration/oauth_provider.md

@@ -78,8 +78,8 @@ in the **Authorized applications** section under **Profile Settings > Applicatio
 ---
 
 GitLab's OAuth applications support scopes, which allow various actions that any given
-application can perform. Although there are only two scopes available at the
-moment – `read_user` and `api` – the groundwork has been laid to add more scopes easily.
+application can perform such as `read_user` and `api`. There are many more scopes
+available.
 
 At any time you can revoke any access by just clicking **Revoke**.
 

doc/user/project/bulk_editing.md

-# Bulk Editing
+# Bulk editing issues and merge requests
 
->**Note:**
+>
+**Notes:**
 - A permission level of `Reporter` or higher is required in order to manage
 issues.
 - A permission level of `Developer` or higher is required in order to manage
 merge requests.
 
-Fields across multiple issues or merge requests can be updated simutaneously by using the bulk edit feature.
+Attributes can be updated simultaneously across multiple issues or merge requests
+by using the bulk editing feature.
 
->**Note:**
-- Bulk editing of issues and merge requests is only available at the project level.
+![Bulk editing](img/bulk-editing.png)
 
-To access the feature, navigate to either the issue or merge request list for the project and click 'Edit Issues' or 'Edit Merge Requests'. This will cause a sidebar to be shown on the right-hand side of the screen, where the available, editable fields are displayed. Checkboxes will also appear to the left-hand side of each issue or merge request, ready to be selected.
+NOTE: **Note:**
+Bulk editing of issues and merge requests is only available at the project level.
 
-Once all items have been selected, choose the appropriate fields and their values from the sidebar and click 'Update All' to apply these changes.
+To update multiple project issues or merge requests at the same time, navigate to
+their respective lists and click **Edit issues** or **Edit merge requests** available
+in the tab bar. This will open a sidebar on the right-hand side of your screen
+where editable fields will be displayed. Checkboxes will also appear to the left-hand
+side of eachissue or merge request for you to select the items you want to update.
 
+Once you have selected all relevant items, choose the appropriate fields and their
+values from the sidebar and click **Update all** to apply your changes.

doc/user/project/import/bitbucket.md

-# Import your project from Bitbucket to GitLab
+# Import your project from Bitbucket Cloud to GitLab
 
-Import your projects from Bitbucket to GitLab with minimal effort.
+NOTE: **Note:**
+The Bitbucket Cloud importer works only with Bitbucket.org, not with Bitbucket
+Server (aka Stash). If you are trying to import projects from Bitbucket Server, use
+[the Bitbucket Server importer](bitbucket_server.md).
 
-## Overview
-
->**Note:**
-The [Bitbucket integration][bb-import] must be first enabled in order to be
-able to import your projects from Bitbucket. Ask your GitLab administrator
-to enable this if not already.
+Import your projects from Bitbucket Cloud to GitLab with minimal effort.
 
->**Note:**  
-The BitBucket importer currently only works with BitBucket's cloud offering 
-(bitbucket.org) and does not work with BitBucket Server (aka Stash).
+## Overview
 
 - At its current state, the Bitbucket importer can import:
   - the repository description (GitLab 7.7+)
@@ -26,6 +22,11 @@ The BitBucket importer currently only works with BitBucket's cloud offering
 - Repository public access is retained. If a repository is private in Bitbucket
   it will be created as private in GitLab as well.
 
+## Requirements
+
+The [Bitbucket Cloud integration][bb-import] must be first enabled in order to be
+able to import your projects from Bitbucket Cloud. Ask your GitLab administrator
+to enable this if not already.
 
 ## How it works
 
@@ -46,9 +47,7 @@ namespace that started the import process.
 1. Sign in to GitLab and go to your dashboard.
 1. Click on **New project**.
 
-    ![New project in GitLab](img/bitbucket_import_new_project.png)
-
-1. Click on the "Bitbucket" button
+1. Click on the "Bitbucket Cloud" button.
 
     ![Bitbucket](img/import_projects_from_new_project_page.png)
 

doc/user/project/import/bitbucket_server.md

+# Import your project from Bitbucket Server to GitLab
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20164)
+in GitLab 11.2.
+
+NOTE: **Note:**
+The Bitbucket Server importer does not work with Bitbucket Cloud (aka bitbucket.org).
+Use the [Bitbucket Cloud importer](bitbucket.md) for that.
+
+Import your projects from Bitbucket Server to GitLab with minimal effort.
+
+## Overview
+
+- In its current state, the Bitbucket importer can import:
+  - the repository description (GitLab 11.2+)
+  - the Git repository data (GitLab 11.2+)
+  - the pull requests (GitLab 11.2+)
+  - the pull request comments (GitLab 11.2+)
+- Repository public access is retained. If a repository is private in Bitbucket
+  it will be created as private in GitLab as well.
+
+## Limitations
+
+1. Currently GitLab doesn't allow comments on arbitrary lines of code, so any
+Bitbucket comments out of bounds will be inserted as comments in the merge
+request.
+1. Bitbucket Server allows multiple levels of threading. GitLab
+import will collapse this into one discussion and quote part of the original
+comment.
+1. Declined pull requests have unrecahable commits, which prevents the GitLab
+importer from generating a proper diff. These pull requests will show up as
+empty changes.
+1. Attachments in Markdown are currently not imported.
+1. Task lists are not imported.
+1. Emoji reactions are not imported
+
+## How it works
+
+The Bitbucket Server importer works as follows:
+
+1. The user will be prompted to enter the URl, username, and password or personal access token to login to Bitbucket.
+   These credentials are preserved only as long as the importer is running.
+1. The importer will attempt to list all the current repositories on the Bitbucket Server.
+1. Upon selection, the importer will clone the repository and import pull requests and comments.
+
+### User assignment
+
+When issues/pull requests are being imported, the Bitbucket importer tries to
+find the author's e-mail address with a confirmed e-mail address in the GitLab
+user database.  If no such user is available, the project creator is set as
+the author. The importer will append a note in the comment to mark the original
+creator.
+
+The importer will create any new namespaces (groups) if they don't exist or in
+the case the namespace is taken, the repository will be imported under the user's
+namespace that started the import process.
+
+## Importing your Bitbucket repositories
+
+1. Sign in to GitLab and go to your dashboard.
+1. Click on **New project**.
+1. Click on the "Bitbucket Server" button. If the button is not present, enable the importer in
+    **Admin > Application Settings > Visibility and access controls > Import sources**.
+
+    ![Bitbucket](img/import_projects_from_new_project_page.png)
+
+1. Enter your Bitbucket Server credentials.
+
+    ![Grant access](img/bitbucket_server_import_credentials.png)
+
+1. Click on the projects that you'd like to import or **Import all projects**.
+   You can also select the namespace under which each project will be
+   imported.
+
+    ![Import projects](img/bitbucket_server_import_select_project.png)

lib/gitlab/bitbucket_import/importer.rb

@@ -188,7 +188,8 @@ module Gitlab
       end
 
       def import_inline_comments(inline_comments, pull_request, merge_request)
-        line_code_map = {}
+        position_map = {}
+        discussion_map = {}
 
         children, parents = inline_comments.partition(&:has_parent?)
 
@@ -196,22 +197,28 @@ module Gitlab
         # relationships. We assume that the child can appear in any order in
         # the JSON.
         parents.each do |comment|
-          line_code_map[comment.iid] = generate_line_code(comment)
+          position_map[comment.iid] = build_position(merge_request, comment)
         end
 
         children.each do |comment|
-          line_code_map[comment.iid] = line_code_map.fetch(comment.parent_id, nil)
+          position_map[comment.iid] = position_map.fetch(comment.parent_id, nil)
         end
 
         inline_comments.each do |comment|
           begin
             attributes = pull_request_comment_attributes(comment)
+            attributes[:discussion_id] = discussion_map[comment.parent_id] if comment.has_parent?
+
             attributes.merge!(
-              position: build_position(merge_request, comment),
-              line_code: line_code_map.fetch(comment.iid),
+              position: position_map[comment.iid],
               type: 'DiffNote')
 
-            merge_request.notes.create!(attributes)
+            note = merge_request.notes.create!(attributes)
+
+            # We can't store a discussion ID until a note is created, so if
+            # replies are created before the parent the discussion ID won't be
+            # linked properly.
+            discussion_map[comment.iid] = note.discussion_id
           rescue StandardError => e
             errors << { type: :pull_request, iid: comment.iid, errors: e.message }
           end
@@ -240,10 +247,6 @@ module Gitlab
         end
       end
 
-      def generate_line_code(pr_comment)
-        Gitlab::Git.diff_line_code(pr_comment.file_path, pr_comment.new_pos, pr_comment.old_pos)
-      end
-
       def pull_request_comment_attributes(comment)
         {
           project: project,

lib/gitlab/git/repository.rb

@@ -382,7 +382,7 @@ module Gitlab
       end
 
       def new_blobs(newrev)
-        return [] if newrev == ::Gitlab::Git::BLANK_SHA
+        return [] if newrev.blank? || newrev == ::Gitlab::Git::BLANK_SHA
 
         strong_memoize("new_blobs_#{newrev}") do
           wrapped_gitaly_errors do

spec/lib/gitlab/bitbucket_import/importer_spec.rb

@@ -69,6 +69,7 @@ describe Gitlab::BitbucketImport::Importer do
   let(:project) do
     create(
       :project,
+      :repository,
       import_source: project_identifier,
       import_url: "https://bitbucket.org/#{project_identifier}.git",
       import_data_attributes: { credentials: data }
@@ -85,10 +86,84 @@ describe Gitlab::BitbucketImport::Importer do
     }
   end
 
+  let(:sample) { RepoHelpers.sample_compare }
+
   before do
     allow(importer).to receive(:gitlab_shell) { gitlab_shell }
   end
 
+  subject { described_class.new(project) }
+
+  describe '#import_pull_requests' do
+    before do
+      allow(subject).to receive(:import_wiki)
+      allow(subject).to receive(:import_issues)
+
+      pull_request = instance_double(
+        Bitbucket::Representation::PullRequest,
+        iid: 10,
+        source_branch_sha: sample.commits.last,
+        source_branch_name: Gitlab::Git::BRANCH_REF_PREFIX + sample.source_branch,
+        target_branch_sha: sample.commits.first,
+        target_branch_name: Gitlab::Git::BRANCH_REF_PREFIX + sample.target_branch,
+        title: 'This is a title',
+        description: 'This is a test pull request',
+        state: 'merged',
+        author: 'other',
+        created_at: Time.now,
+        updated_at: Time.now)
+
+      # https://gitlab.com/gitlab-org/gitlab-test/compare/c1acaa58bbcbc3eafe538cb8274ba387047b69f8...5937ac0a7beb003549fc5fd26fc247ad
+      @inline_note = instance_double(
+        Bitbucket::Representation::PullRequestComment,
+        iid: 2,
+        file_path: '.gitmodules',
+        old_pos: nil,
+        new_pos: 4,
+        note: 'Hello world',
+        author: 'root',
+        created_at: Time.now,
+        updated_at: Time.now,
+        inline?: true,
+        has_parent?: false)
+
+      @reply = instance_double(
+        Bitbucket::Representation::PullRequestComment,
+        iid: 3,
+        file_path: '.gitmodules',
+        note: 'Hello world',
+        author: 'root',
+        created_at: Time.now,
+        updated_at: Time.now,
+        inline?: true,
+        has_parent?: true,
+        parent_id: 2)
+
+      comments = [@inline_note, @reply]
+
+      allow(subject.client).to receive(:repo)
+      allow(subject.client).to receive(:pull_requests).and_return([pull_request])
+      allow(subject.client).to receive(:pull_request_comments).with(anything, pull_request.iid).and_return(comments)
+    end
+
+    it 'imports threaded discussions' do
+      expect { subject.execute }.to change { MergeRequest.count }.by(1)
+
+      merge_request = MergeRequest.first
+      expect(merge_request.notes.count).to eq(2)
+      expect(merge_request.notes.map(&:discussion_id).uniq.count).to eq(1)
+
+      notes = merge_request.notes.order(:id).to_a
+      start_note = notes.first
+      expect(start_note).to be_a(DiffNote)
+      expect(start_note.note).to eq(@inline_note.note)
+
+      reply_note = notes.last
+      expect(reply_note).to be_a(DiffNote)
+      expect(reply_note.note).to eq(@reply.note)
+    end
+  end
+
   context 'issues statuses' do
     before do
       # HACK: Bitbucket::Representation.const_get('Issue') seems to return ::Issue without this