Merge branch 'docs-junit-test-reports-ee' into 'master'

Documentation for JUnit XML Test Summary In MR widget

See merge request gitlab-org/gitlab-ee!6870

doc/README.md

@@ -147,6 +147,7 @@ scales to run your tests faster.
 - [GitLab CI/CD](ci/README.md): Explore the features and capabilities of Continuous Integration, Continuous Delivery, and Continuous Deployment with GitLab.
 - [Review Apps](ci/review_apps/index.md): Preview changes to your app right from a merge request.
 - [Pipeline Graphs](ci/pipelines.md#pipeline-graphs)
+- [JUnit test reports](ci/junit_test_reports.md)
 - [Multi-project pipelines](ci/multi_project_pipelines.md) **[PREMIUM]**
 - [Code Quality reports](user/project/merge_requests/code_quality.md) **[STARTER]**
 - [Static Application Security Testing (SAST) reports](user/project/merge_requests/sast.md) **[ULTIMATE]**

doc/ci/examples/README.md

@@ -43,6 +43,10 @@ There's also a collection of repositories with [example projects](https://gitlab
 - [Using `dpl` as deployment tool](deployment/README.md)
 - [The `.gitlab-ci.yml` file for GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml)
 
+## Test Reports
+
+[Collect test reports in Verify stage](../junit_test_reports.md).
+
 ## Code Quality analysis **[STARTER]**
 
 [Analyze your project's Code Quality](code_quality.md).

doc/ci/junit_test_reports.md

+# JUnit test reports
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/45318) in GitLab 11.2.
+Requires GitLab Runner 11.2 and above.
+
+## Overview
+
+It is very common that a [CI/CD pipeline](pipelines.md) contains a
+test job that will verify your code.
+If the tests fail, the pipeline fails and users get notified. The person that
+works on the merge request will have to check the job logs and see where the
+tests failed so that they can fix them.
+
+You can configure your job to use JUnit test reports, and GitLab will display a
+report on the merge request so that it's easier and faster to identify the
+failure without having to check the entire log.
+
+## Use cases
+
+Consider the following workflow:
+
+1. Your `master` branch is rock solid, your project is using GitLab CI/CD and
+   your pipelines indicate that there isn't anything broken.
+1. Someone from you team submits a merge request, a test fails and the pipeline
+   gets the known red icon. To investigate more, you have to go through the job
+   logs to figure out the cause of the failed test, which usually contain
+   thousands of lines.
+1. You configure the JUnit test reports and immediately GitLab collects and
+   exposes them in the merge request. No more searching in the job logs.
+1. Your development and debugging workflow becomes easier, faster and efficient.
+
+## How it works
+
+First, GitLab Runner uploads all JUnit XML files as artifacts to GitLab. Then,
+when you visit a merge request, GitLab starts comparing the head and base branch's
+JUnit test reports, where:
+
+- The base branch is the target branch (usually `master`).
+- The head branch is the source branch (the latest pipeline in each merge request).
+
+The reports panel has a summary showing how many tests failed and how many were fixed.
+If no comparison can be done because data for the base branch is not available,
+the panel will just show the list of failed tests for head.
+
+There are three types of results:
+
+1. **Newly failed tests:** Test cases which passed on base branch and failed on head branch
+1. **Existing failures:**  Test cases which failed on base branch and failed on head branch
+1. **Resolved failures:**  Test cases which failed on base branch and passed on head branch
+
+Each entry in the panel will show the test name and its type from the list
+above. Clicking on the test name will open a modal window with details of its
+execution time and the error output.
+
+![Test Reports Widget](img/junit_test_report.png)
+
+## How to set it up
+
+NOTE: **Note:**
+For a list of supported languages on JUnit tests, check the
+[Wikipedia article](https://en.wikipedia.org/wiki/JUnit#Ports).
+
+To enable the JUnit reports in merge requests, you need to add
+[`artifacts:reports:junit`](yaml/README.md#artifacts-reports-junit)
+in `.gitlab-ci.yml`, and specify the path(s) of the generated test reports.
+
+In the following examples, the job in the `test` stage runs and GitLab
+collects the JUnit test report from each job. After each job is executed, the
+XML reports are stored in GitLab as artifacts and their results are shown in the
+merge request widget.
+
+### Ruby example
+
+Use the following job in `.gitlab-ci.yml`:
+
+```yaml
+## Use https://github.com/sj26/rspec_junit_formatter to generate a JUnit report with rspec
+ruby:
+  stage: test
+  script:
+  - bundle install
+  - rspec spec/lib/ --format RspecJunitFormatter --out rspec.xml
+  artifacts:
+    reports:
+      junit: rspec.xml
+```
+
+### Go example
+
+Use the following job in `.gitlab-ci.yml`:
+
+```yaml
+## Use https://github.com/jstemmer/go-junit-report to generate a JUnit report with go
+golang:
+  stage: test
+  script:
+  - go get -u github.com/jstemmer/go-junit-report
+  - go test -v 2>&1 | go-junit-report > report.xml
+  artifacts:
+    reports:
+      junit: report.xml
+```

doc/ci/yaml/README.md

@@ -1092,6 +1092,52 @@ job:
     expire_in: 1 week
 ```
 
+### `artifacts:reports`
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20390) in
+GitLab 11.2. Requires GitLab Runner 11.2 and above.
+
+The `reports` keyword is used for collecting test reports from jobs and
+exposing them in GitLab's UI (merge requests, pipeline views). Read how to use
+this with [JUnit reports](#artifacts-reports-junit).
+
+NOTE: **Note:**
+The test reports are collected regardless of the job results (success or failure).
+You can use [`artifacts:expire_in`](#artifacts-expire_in) to set up an expiration
+date for their artifacts.
+
+#### `artifacts:reports:junit`
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20390) in
+GitLab 11.2. Requires GitLab Runner 11.2 and above.
+
+The `junit` report collects [JUnit XML files](https://www.ibm.com/support/knowledgecenter/en/SSQ2R2_14.1.0/com.ibm.rsar.analysis.codereview.cobol.doc/topics/cac_useresults_junit.html)
+as artifacts. Although JUnit was originally developed in Java, there are many
+[third party ports](https://en.wikipedia.org/wiki/JUnit#Ports) for other
+languages like Javascript, Python, Ruby, etc.
+
+Below is an example of collecting a JUnit XML file from Ruby's RSpec test tool:
+
+```yaml
+rspec:
+  stage: test
+  script:
+  - bundle install
+  - rspec --format RspecJunitFormatter --out rspec.xml
+  artifacts:
+    reports:
+      junit: rspec.xml
+```
+
+The collected JUnit reports will be uploaded to GitLab as an artifact and will
+be automatically [shown in merge requests](../junit_test_reports.md).
+
+NOTE: **Note:**
+In case the JUnit tool you use exports to multiple XML files, you can specify
+multiple test report paths within a single job
+(`junit: [rspec-1.xml, rspec-2.xml, rspec-3.xml]`) and they will be automatically
+concatenated into a single file.
+
 ## `dependencies`
 
 > Introduced in GitLab 8.6 and GitLab Runner v1.1.1.

doc/user/project/merge_requests/index.md

@@ -50,10 +50,10 @@ A. Consider you are a software developer working in a team:
 1. You checkout a new branch, and submit your changes through a merge request
 1. You gather feedback from your team
 1. You work on the implementation optimizing code with [Code Quality reports](code_quality.md) **[STARTER]**
+1. You verify your changes with [JUnit test reports](../../../ci/junit_test_reports.md) in GitLab CI/CD
 1. You avoid using dependencies whose license is not compatible with your project with [License Management reports](license_management.md) **[ULTIMATE]**
-1. You build and test your changes with GitLab CI/CD
 1. You request the [approval](#merge-request-approvals) from your manager
-1. Your manager pushes a commit with his final review, [approves the merge request](#merge-request-approvals), and set it to [merge when pipeline succeeds](#merge-when-pipeline-succeeds)
+1. Your manager pushes a commit with their final review, [approves the merge request](merge_request_approvals.md), and set it to [merge when pipeline succeeds](#merge-when-pipeline-succeeds) (Merge Request Approvals are available in GitLab Starter)
 1. Your changes get deployed to production with [manual actions](../../../ci/yaml/README.md#manual-actions) for GitLab CI/CD
 1. Your implementations were successfully shipped to your customer