Merge branch 'bugs-and-regressions-process' into 'master'

Bugs and regression triage and prioritization process

See merge request gitlab-org/gitlab-ce!20623

PROCESS.md

@@ -15,8 +15,9 @@
   - [Between the 1st and the 7th](#between-the-1st-and-the-7th)
   - [On the 7th](#on-the-7th)
   - [After the 7th](#after-the-7th)
-- [Regressions](#regressions)
-  - [How to manage a regression](#how-to-manage-a-regression)
+- [Bugs](#bugs)
+  - [Regressions](#regressions)
+  - [Managing bugs](#managing-bugs)
 - [Release retrospective and kickoff](#release-retrospective-and-kickoff)
   - [Retrospective](#retrospective)
   - [Kickoff](#kickoff)
@@ -168,7 +169,7 @@ information, see
 Once the stable branch is frozen, the only MRs that can be cherry-picked into
 the stable branch are:
 
-* Fixes for [regressions](#regressions)
+* Fixes for [regressions](#regressions) where the affected version `xx.x` in `regression:xx.x` is the current release. See [Managing bugs](#managing-bugs) section.
 * Fixes for security issues
 * Fixes or improvements to automated QA scenarios
 * Documentation updates for changes in the same release
@@ -201,48 +202,59 @@ you can ask for an exception to be made.
 
 Check [this guide](https://gitlab.com/gitlab-org/release/docs/blob/master/general/exception-request/process.md) about how to open an exception request before opening one.
 
-## Regressions
+## Bugs
 
-A regression for a particular monthly release is a bug that exists in that
-release, but wasn't present in the release before. This includes bugs in
-features that were only added in that monthly release. Every regression **must**
-have the milestone of the release it was introduced in - if a regression doesn't
-have a milestone, it might be 'just' a bug!
+A ~bug is a defect, error, failure which causes the system to behave incorrectly or prevents it from fulfilling the product requirements.
 
-For instance, if 10.5.0 adds a feature, and that feature doesn't work correctly,
-then this is a regression in 10.5. If 10.5.1 then fixes that, but 10.5.3 somehow
-reintroduces the bug, then this bug is still a regression in 10.5.
+The level of impact of a ~bug can vary from blocking a whole functionality 
+or a feature usability bug. A bug should always be linked to a severity level. 
+Refer to our [severity levels](../CONTRIBUTING.md#severity-labels)
 
-Because GitLab.com runs release candidates of new releases, a regression can be
-reported in a release before its 'official' release date on the 22nd of the
-month. When we say 'the most recent monthly release', this can refer to either
-the version currently running on GitLab.com, or the most recent version
-available in the package repositories.
+Whether the bug is also a regression or not, the triage process should start as soon as possible. 
+Ensure that the Engineering Manager and/or the Product Manager for the relative area is involved to prioritize the work as needed.
 
-### How to manage a regression
+### Regressions
 
-Regressions are very important, and they should be considered high priority 
-issues that should be solved as soon as possible, especially if they affect
-users. Despite that, ~regression label itself does not imply when the issue
-will be scheduled.
+A ~regression implies that a previously **verified working functionality** no longer works.
+Regressions are a subset of bugs. We use the ~regression label to imply that the defect caused the functionality to regress.
+The label tells us that something worked before and it needs extra attention from Engineering and Product Managers to schedule/reschedule. 
 
-When a regression is found:
-1. Create an issue describing the problem in the most detailed way possible
-1. If possible, provide links to real examples and how to reproduce the problem
+The regression label does not apply to ~bugs for new features for which functionality was **never verified as working**. 
+These, by definition, are not regressions. 
+
+A regression should always have the `regression:xx.x` label on it to designate when it was introduced.
+
+Regressions should be considered high priority issues that should be solved as soon as possible, especially if they have severe impact on users. 
+
+### Managing bugs
+
+**Prioritization:** We give higher priority to regressions on features that worked in the last recent monthly release and the current release candidates. 
+The two scenarios below can [bypass the exception request in the release process](https://gitlab.com/gitlab-org/release/docs/blob/master/general/exception-request/process.md#after-the-7th), where the affected regression version matches the current monthly release version. 
+* A regression which worked in the **Last monthly release**
+   * **Example:** In 11.0 we released a new `feature X` that is verified as working. Then in release 11.1 the feature no longer works, this is regression for 11.1. The issue should have the `regression:11.1` label.
+   * *Note:* When we say `the last recent monthly release`, this can refer to either the version currently running on GitLab.com, or the most recent version available in the package repositories.
+* A regression which worked in the **Current release candidates**
+   * **Example:** In 11.1-RC3 we shipped a new feature which has been verified as working. Then in 11.1-RC5 the feature no longer works, this is regression for 11.1. The issue should have the `regression:11.1` label.
+   * *Note:* Because GitLab.com runs release candidates of new releases, a regression can be reported in a release before its 'official' release date on the 22nd of the month. 
+
+When a bug is found:
+1. Create an issue describing the problem in the most detailed way possible.
+1. If possible, provide links to real examples and how to reproduce the problem.
 1. Label the issue properly, using the [team label](../CONTRIBUTING.md#team-labels),
    the [subject label](../CONTRIBUTING.md#subject-labels)
    and any other label that may apply in the specific case
-1. Add the ~bug and ~regression labels
-1. Notify the respective Engineering Manager to evaluate the Severity of the regression and add a [Severity label](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#bug-severity-labels). The counterpart Product Manager is included to weigh-in on prioritization as needed to set the [Priority label](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#bug-priority-labels).
-1. If the regression is either an ~S1, ~S2 or ~S3 severity, label the regression with the current milestone as it should be fixed in the current milestone.
-  1. If the regression was introduced in an RC of the current release, label with ~Deliverable
-  1. If the regression was introduced in the previous release, label with ~"Next Patch Release"
-1. If the regression is an ~S4 severity, the regression may be scheduled for later milestones at the discretion of Engineering Manager and Product Manager.
-
-When a new issue is found, the fix should start as soon as possible. You can
-ping the Engineering Manager or the Product Manager for the relative area to
-make them aware of the issue earlier. They will analyze the priority and change
-it if needed.
+1. Notify the respective Engineering Manager to evaluate and apply the [Severity label](../CONTRIBUTING.md#bug-severity-labels) and [Priority label](../CONTRIBUTING.md#bug-priority-labels).
+The counterpart Product Manager is included to weigh-in on prioritization as needed.
+1. If the ~bug is **NOT** a regression:
+   1. The Engineering Manager decides which milestone the bug will be fixed. The appropriate milestone is applied.
+1. If the bug is a ~regression: 
+   1. Determine the release that the regression affects and add the corresponding `regression:xx.x` label.
+      1. If the affected release version can't be determined, add the generic ~regression label for the time being.
+   1. If the affected version `xx.x` in `regression:xx.x` is the **current release**, it's recommended to schedule the fix for the current milestone. 
+      1. This falls under regressions which worked in the last release and the current RCs. More detailed explanations in the **Prioritization** section above. 
+   1. If the affected version `xx.x` in `regression:xx.x` is older than the **current release**
+      1. If the regression is an ~S1 severity, it's recommended to schedule the fix for the current milestone. We would like to fix the highest severity regression as soon as we can.
+      1. If the regression is an ~S2, ~S3 or ~S4 severity, the regression may be scheduled for later milestones at the discretion of the Engineering Manager and Product Manager.
 
 ## Release retrospective and kickoff