Merge branch 'alberts-run-less-test-before-approval' into 'master'

Run minimal rspec jobs before merge request is approved

See merge request gitlab-org/gitlab!64903

.gitlab/ci/rails.gitlab-ci.yml

@@ -641,36 +641,33 @@ rspec-ee unit pg12 geo:
     - .rails:rules:ee-only-unit
     - .rspec-ee-unit-geo-parallel
 
-# FIXME: Temporarily disable geo minimal rspec jobs https://gitlab.com/gitlab-org/gitlab/-/issues/294212
-#rspec-ee unit pg12 geo minimal:
-#  extends:
-#    - rspec-ee unit pg12 geo
-#    - .minimal-rspec-tests
-#    - .rails:rules:ee-only-unit:minimal
+rspec-ee unit pg12 geo minimal:
+  extends:
+    - rspec-ee unit pg12 geo
+    - .minimal-rspec-tests
+    - .rails:rules:ee-only-unit:minimal
 
 rspec-ee integration pg12 geo:
   extends:
     - .rspec-ee-base-geo-pg12
     - .rails:rules:ee-only-integration
 
-# FIXME: Temporarily disable geo minimal rspec jobs https://gitlab.com/gitlab-org/gitlab/-/issues/294212
-#rspec-ee integration pg12 geo minimal:
-#  extends:
-#    - rspec-ee integration pg12 geo
-#    - .minimal-rspec-tests
-#    - .rails:rules:ee-only-integration:minimal
+rspec-ee integration pg12 geo minimal:
+  extends:
+    - rspec-ee integration pg12 geo
+    - .minimal-rspec-tests
+    - .rails:rules:ee-only-integration:minimal
 
 rspec-ee system pg12 geo:
   extends:
     - .rspec-ee-base-geo-pg12
     - .rails:rules:ee-only-system
 
-# FIXME: Temporarily disable geo minimal rspec jobs https://gitlab.com/gitlab-org/gitlab/-/issues/294212
-#rspec-ee system pg12 geo minimal:
-#  extends:
-#    - rspec-ee system pg12 geo
-#    - .minimal-rspec-tests
-#    - .rails:rules:ee-only-system:minimal
+rspec-ee system pg12 geo minimal:
+  extends:
+    - rspec-ee system pg12 geo
+    - .minimal-rspec-tests
+    - .rails:rules:ee-only-system:minimal
 
 db:rollback geo:
   extends:

doc/development/code_review.md

@@ -423,6 +423,7 @@ WARNING:
     do not merge the merge request** except for
     [very specific cases](https://about.gitlab.com/handbook/engineering/workflow/#criteria-for-merging-during-broken-master).
     For other cases, follow these [handbook instructions](https://about.gitlab.com/handbook/engineering/workflow/#merging-during-broken-master).
+  - If the latest pipeline was created before the merge request was approved, start a new pipeline to ensure that full RSpec suite has been run. You may skip this step only if the merge request does not contain any backend change.
   - If the **latest [Pipeline for Merged Results](../ci/pipelines/pipelines_for_merged_results.md)** finished less than 2 hours ago, you
     might merge without starting a new pipeline as the merge request is close
     enough to `main`.

doc/development/pipelines.md

@@ -421,58 +421,18 @@ This experiment is only enabled when the CI/CD variable `RSPEC_FAIL_FAST_ENABLED
 The test files related to the Merge Request are determined using the [`test_file_finder`](https://gitlab.com/gitlab-org/ci-cd/test_file_finder) gem.
 We are using a custom mapping between source file to test files, maintained in the `tests.yml` file.
 
-### RSpec minimal job experiment
+### RSpec minimal jobs
 
-As part of the objective to improve overall pipeline duration, we are experimenting with a minimal set of RSpec tests.
-The purpose of this experiment is to verify if we are able to run a minimal set of RSpec tests in a Merge Request pipeline,
-without resulting in increased number of broken main branch.
+Before a merge request is approved, the pipeline will run a minimal set of RSpec tests that are related to the merge request changes.
+This is to reduce the pipeline cost and shorten the job duration.
 
 To identify the minimal set of tests needed, we use [Crystalball gem](https://github.com/toptal/crystalball) to create a test mapping.
 The test mapping contains a map of each source files to a list of test files which is dependent of the source file.
 This mapping is currently generated using a combination of test coverage tracing and a static mapping.
 In the `detect-tests` job, we use this mapping to identify the minimal tests needed for the current Merge Request.
 
-In this experiment, each `rspec` job is accompanied with a `minimal` version.
-For example, `rspec unit` job has a corresponding `rspec unit minimal` job.
-During the experiment, each Merge Request pipeline will contain both versions of the job, running in parallel.
-
-To illustrate this:
-
-```mermaid
-graph LR
-    A --"artifact: list of test files"--> C1 & D1 & E1 & F1
-
-    subgraph "prepare stage";
-        A["detect-tests"];
-    end
-
-    subgraph "test stage";
-        C["rspec migration"];
-        C1["rspec migration minimal"];
-        D["rspec unit"];
-        D1["rspec unit minimal"];
-        E["rspec integration"];
-        E1["rspec integration minimal"];
-        F["rspec system"];
-        F1["rspec system minimal"];
-    end
-```
-
-The result of both set of jobs in the pipeline is then compared to identify any false positive.
-A list of such pipeline can be found in [Sisense](https://app.periscopedata.com/app/gitlab/496118/Engineering-Productivity-Sandbox?widget=10492739&udv=833427).
-
-A false positive is defined as a pipeline where the `minimal` jobs passed, but the non-`minimal` jobs failed.
-This indicates that the changeset resulted in a test failure, which was not detected by the `minimal` jobs.
-Consequently, this signifies a gap in the test mapping used, which would need to be rectified.
-
-#### Findings
-
-After a round of the experiment done in December 2020,
-we discovered that it was challenging to achieve a mapping that gives high confidence at the moment,
-because of 2 reasons:
-
-- Each identified gap in the test mapping is unique, each needing its own investigation and improvement to the creation of the test mapping.
-- There is a large number of flaky tests which added a lot of noise in the data, slowing down the investigation process.
+After a merge request has been approved, the pipeline would contain the full RSpec tests. This will ensure that all tests
+have been run before a merge request is merged.
 
 ### PostgreSQL versions testing