This implements atomic processing service
that is able in bulk to update multiple builds,
stages and pipelines.

This uses in-memory structure to calculate statuses
and uses an efficient fetch of DAG dependencies
for efficient processing

This still performs some redundant operations,
but overall it should be significantly faster
processing pipelines of 1000 builds, with
multiple DAG dependencies, as it removes
a number of N+1 problems present in previous
implementation.

This also is order-independent, so it is able
to easily reschedule itself once needed.

This code is also concurrent safe, as if needed
the further processing will be re-scheduled
that will resolve any conflicts.

Adds a counter on completed Auto DevOps pipelines, labeled by pipeline
status. This is a first step in increasing the visibility on the impact
that the changes we make have on Auto DevOps functionality.

This reverts commit 0445278a.

This reverts merge request !20930

Fix include config handling

Define child and parent pipeline in model

* Define scopes for child and parent pipelines
* Define child? and parent? methods in Ci::Pipeline
* Expose the project in pipeline.json so that it can
  be compared with downstream and upstream pipelines

Refactor same_family_pipeline_ids

Besides .gitlab-ci.yml file,
there are other factors that affect pipeline creation.
So we need to store the content of its configuration.

This commit supports Ci Resource Group status transition

This makes the logic when to remove only/except
much simpler

The first of many MRs to make an attempt to have a test suite that can
run successfully in any locale.

- Moved the :merge_train logic branch of
  Ci::Pipeline#merge_request_event_type into EE
- Added specs for EE::Ci::Pipeline to include base and EE cases

It uses scopes from Ci::Pipeline, which are public and used once,
what about just moving them in this class and make them private?

As they are not really a 'build-specific' ones

Move all config related methods from Ci::Pipeline
out into a dedicated class.

This solves small technical debt,
where pipeline processing should be
done by dedicated service instead
of method running on a model.

Auto-correct some references in RSpec as:
- expect_any_instance_of -> expect_next_instance_of
- allow_any_instance_of -> allow_next_instance_of

Related to https://gitlab.com/gitlab-org/gitlab/issues/34997

* Remove config transient
* Remove factory for 1 job pipeline
* Remove factory for 0 job pipeline

This adds a missing support for the `parallel:`
that got broken after merging the
https://gitlab.com/gitlab-org/gitlab/merge_requests/17539.

This also improves `Undefined error` to log the error
and add correlation ID.

Signed-off-by: Rémy Coutable <remy@rymai.me>

Allow jobs to use CI_JOB_TOKEN to trigger downstream pipelines

This commits introduces dedicated pipeline ref

Instead of preloading the status, we now preload the
latest pipelines instead for each commit. This gives us more
flexibility on how to display the status.

This also helps us avoid passing the current_user deep through
the model methods. Best to call the detailed_status in the
view/presentation layer.

We also extracted commit's pipeline behavior into its own class.
This is to better isolate and organize the pipeline related behavior
of the Commit object.

Moving the pipeline behavior into its own class resulted
to a lot of breaking changes though. So instead of always having to
create your own instance of a CommitWithPipeline, we're just gonna
use the same commit object but just delegate the pipeline related
calls to the CommitWithPipeline instance.

In the Merge Request view, under pipelines tab
the user can see a run pipeline button

Adds axios post request to button click

Adds the logic to handle the user click,
refresh the table and disable the button while thee
request is being made

Updates UI for desktop and mobile

Adds specs
Regenerates potfile

Follow-up after review

Uses .finally to avoid code repetition

In the Merge Request view, under pipelines tab
the user can see a run pipeline button

Adds axios post request to button click

Adds the logic to handle the user click,
refresh the table and disable the button while thee
request is being made

Updates UI for desktop and mobile

Adds specs
Regenerates potfile

Follow-up after review

Uses .finally to avoid code repetition

When using a mirror for CI/CD only we register a pull_request
webhook. When a pull_request webhook is received, if the
source branch SHA matches the actual head of the branch in the
repository we create immediately a new pipeline for the
external pull request. Otherwise we store the
pull request info for when the push webhook is received.

When using "only/except: external_pull_requests" we can detect
if the pipeline has a open pull request on GitHub and create or
not the job based on that.

Feedback from review

Split big non-transactional migration into smaller ones
Refactorings by moving methods to relative models
Return 422 when webhook has unsupported actions

Detect if pipeline runs for a GitHub pull request

When using a mirror for CI/CD only we register a pull_request
webhook. When a pull_request webhook is received, if the
source branch SHA matches the actual head of the branch in the
repository we create immediately a new pipeline for the
external pull request. Otherwise we store the
pull request info for when the push webhook is received.

When using "only/except: external_pull_requests" we can detect
if the pipeline has a open pull request on GitHub and create or
not the job based on that.

This commit adds pipeline.type key to PipelineEntity.
This key will be used in MR widget in the next iteration.

Also change test URL sequest to .test TLD

Also change test URL sequest to .test TLD

Also change test URL sequest to .test TLD

Also change test URL sequest to .test TLD

Ports changes from
https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/12343

Refactors the bridge status update logic to be centralized in one single
place.

* Reword Project#latest_successful_build_for to
Project#latest_successful_build_for_ref
* Reword Ci::Pipeline#latest_successful_for to
Ci::Pipeline#latest_successful_build_for_ref