GitLab

Move the metrics gems out of the group since these are installed all the
time anyway.

Prometheus client in particular is required to boot the application.

This adds Ruby 3 support.

In order to make use of the new `list_blobs()` RPC, we need a newer
definition of the Gitaly protobuf Gem. Update it as a preparatory step.

This enables Azure Government Cloud to work as an OAuth2 provider since
this version refactors out the use of
`https://graph.microsoft.com/v1.0/me` via
https://github.com/RIPGlobal/omniauth-azure-activedirectory-v2/pull/1.

Azure Government Cloud uses `.us` as a suffix, so instead of
`https://graph.microsoft.com` the base URL should be
`https://graph.microsoft.us`. `base_azure_url` needs to be set to
`https://login.microsoftonline.us` as well.

This upgrade requires upgrading the related OmniAuth gems as well. There
are minor changes in those gems.

Relates to https://gitlab.com/gitlab-org/gitlab/-/issues/330880

Changelog: fixed

With release 14.0, Gitaly has introduced the ability to specify
in-memory remotes for the `UpdateRemoteMirror` RPC. We're about to start
using this new feature and thus need the new protobuf definitions which
include the new remote parameter.

Update the Gitaly Gem to 14.0.0.pre.rc2 to get the new definitions.

This takes advantage of the shared rule from the gem.
Signed-off-by: Rémy Coutable <remy@rymai.me>

https://gitlab.com/gitlab-org/gitlab/-/merge_requests/62646

This reverts merge request !62601

- Add Cop to enforce have_enqueued_sidekiq_job
- Exclude existing ocurrences of receive(:perform_async)
- Update docs

Changelog: other

This reverts merge request !62387

Removes unicorn gem, gemfile group and license.

Changelog: removed

Changelog: changed
Issue: https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/849
MR: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/62387

After the model `HangoutsChatService` was renamed
`Integrations::HangoutsChat`, and therefore the file renamed as
`hangsouts_chat.rb`,
`config/initializers/hangouts_chat_http_override.rb` began to define new
classes, rather than to monkeypatch the gem classes.

Requiring the gem this way allows us to continue to monkeypatch the gem.

https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61957#note_586260689
https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61957
https://gitlab.com/gitlab-org/gitlab/-/issues/201855

Due to https://github.com/protocolbuffers/protobuf/issues/8559,
google-protobuf v3.15.8 can seg fault in the FindCommits RPC call if the
options hash is garbage collected before gRPC encodes the message. This
was fixed in google-protobuf v3.17.1 via
https://github.com/protocolbuffers/protobuf/pull/8639.

Unfortunately, pg_query has a hard dependency on google-protobuf
v3.15.x.  This was bumped in
https://github.com/pganalyze/pg_query/pull/212, but an official version
has not yet been tagged.

In addition, https://github.com/pganalyze/pg_query/pull/213 would relax
the dependency so that google-protobuf can be upgraded without having to
update pg_query.

Until pg_query releases a new version, we use our fork to ensure this
seg fault cannot happen.

Relates to https://gitlab.com/gitlab-org/gitlab/-/issues/330998

Changelog: fixed

This means we can remove our own patch to immediately unlink temporary
files, as it's now in the gem: https://github.com/puma/puma/pull/2613

Changelog: other

This allows us to remove a custom patch that has now been accepted
upstream.

Drops unicorn-worker-killer gem and related configs, code, docs.
Split from https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61888.

Changelog: removed

It contains multiple security fixes.

One of them prevents string polymorphic route arguments and
causes some additional changes to be made along with just bumping
gem version

When Sentry gets Rack context, it can grab information that is very
large (like Grape routing information) from the Rack environment. It can
waste a lot of time and memory processing that information, only to then
... not send it to Sentry as it's not a standard key.

https://github.com/getsentry/sentry-ruby/pull/1197 describes this
perfectly, but as we're not on the latest version,
https://github.com/getsentry/sentry-ruby/pull/1198 is the PR that we're
pulling in to fix this issue.

To test this locally:

1. Create a project.
2. Move or delete its on-disk repo.
3. Configure Sentry.
4. Run `curl http://$host/api/v4/projects/$id/repository/branches/main`.
5. Check the `mem_bytes` field in `api_json.log`.

This change takes several hundred megabytes from that value in my local
environment.

Changelog: performance

With removing Unicorn support in 14.0, UnicornSampler is no longer
needed.
Raindrops was only used with Unicorn, so it could be removed too.
Update the docs related to the change.

Changelog: removed

This brings in a memory usage improvement and adds support for Ruby 3

This uses the PostgreSQL v13 parser instead of v10.

Full list of changes:
https://github.com/pganalyze/pg_query/blob/main/CHANGELOG.md

Relates to https://gitlab.com/gitlab-org/gitlab/-/issues/330003

This brings in a performance improvement and a subsequent fix:

* https://gitlab.com/gitlab-org/grape-path-helpers/-/merge_requests/33
* https://gitlab.com/gitlab-org/grape-path-helpers/-/merge_requests/35

Changelog: performance

Adds project id, project path to message
Creates verdict type NOOP

Gitaly has extended the FindRemoteRootRefs RPC to accept a URL instead
of a remote name, so we need to upgrade the Gitaly Gem hosting protobuf
declarations to start using it.

This opens up the possibility to actually re-use Danger rules accross
projects.
Signed-off-by: Rémy Coutable <remy@rymai.me>

This reverts merge request !60082

This reverts merge request !58130

commit a2ef9f1e1c0ae4fd119481a493cf118fb0e2113d Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Thu Apr 29 12:24:51 2021 -0400

    Responding to code review comments

commit 107845cfcfefe3056bb772a3eeb09d7579faa07f Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Wed Apr 28 21:42:27 2021 -0400

    Add support for Recaptcha and CONDITIONAL_ALLOW

commit 82b0f2d5a4cfb5698f01d9c98b43cc6911e1496a Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Wed Apr 28 21:06:35 2021 -0400

    Add GRPC credentials for TLS in production

commit 5d569e39feea23a2606d9a2a23d5794251f1e704 Merge: cd12ff5e748
d6fb606a38d Author: Ethan Urie <ethan.urie@gmail.com> Date:   Wed Apr 28
17:17:01 2021 -0400

    Merge branch '299558-integrate-with-anti-spam-service' of
    gitlab.com:gitlab-org/gitlab into
    299558-integrate-with-anti-spam-service

commit cd12ff5e748855c10b90f68394e3be60f82204fc Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Wed Apr 28 17:12:41 2021 -0400

    Addressing code review comments

    Mostly removed unused or unnecessary code

commit f71f72cac863375273eee1360ac2c4497c960e76 Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Wed Apr 28 16:10:39 2021 -0400

    Adding changelog

commit d6fb606a38de171b3c3c0347e581c2f7c4bb74aa Author: agroleau
<agroleau@gitlab.com> Date:   Wed Apr 28 16:10:36 2021 -0400

    Integrate Spamcheck gRPC gem into GitLab spam verdict service

    Introduction of the Security Automation anti-spam engine protobuf
    gem.

    Sets up a GRPC channel from the SpamVerdictService to the anti-spam
    engine.

commit 292046c74fdaf5f67c55ef361973835427fd253f Merge: 9058ba98ad6
c1e4eca80ca Author: Ethan Urie <ethan.urie@gmail.com> Date:   Wed Apr 28
16:03:03 2021 -0400

    Merge branch '299558-integrate-with-anti-spam-service' of
    gitlab.com:gitlab-org/gitlab into
    299558-integrate-with-anti-spam-service

commit c1e4eca80ca26b5d0f0f0c28171170213b1ad85d Author: agroleau
<agroleau@gitlab.com> Date:   Wed Apr 28 14:39:09 2021 -0400

    Integrate Spamcheck gRPC gem into GitLab spam verdict service

    The anti-spam engine, spamcheck, is going to utilize gRPC in
    production but for now, it provides an HTTP endpoint that accepts
    and returns JSON. However, the JSON has to be formatted to match the
    structure of the protocol buffers that will eventually be used. This
    MR reformats the JSON to match spamcheck's protobufs.

commit 9058ba98ad614ad5c610382cecf727fab832bf9c Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Mon Apr 26 14:17:04 2021 -0400

    Re-implement rescuing a specific error type

commit 4015eebfcc2eebab096ad5f4815f23477ad3388c Merge: d6b5a6ec4c8
ba710a0d Author: Ethan Urie <ethan.urie@gmail.com> Date:   Mon Apr 26
12:57:38 2021 -0400

    Merged with master

commit d6b5a6ec4c87467cff8e8235cd9a90d21dc50a61 Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Fri Apr 23 12:53:32 2021 -0400

    Fix build issues

commit 1e43f06fa678263779ec432818e6f4750aca8779 Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Fri Apr 23 10:50:08 2021 -0400

    Set spamcheck gem to 0.0.5 from rubygems

    Refactor `SpamVerdictService` to use the new Spamcheck client class

commit 2c5d10fdf0716f7b843ca16abe7d29c54fa7adde Merge: 4cb5cbd1bc4
69594f08 Author: Ethan Urie <ethan.urie@gmail.com> Date:   Thu Apr 22
16:53:16 2021 -0400

    Merged with master

commit 4cb5cbd1bc4ac15429c69d726bb2f649f87cadc8 Merge: 6d7eeb78d98
0dd8c84b Author: Ethan Urie <ethan.urie@gmail.com> Date:   Tue Apr 20
16:03:46 2021 -0400

    Merged with master

commit 6d7eeb78d9818320dabf550a8796487f7d3414f8 Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Wed Apr 14 16:14:33 2021 -0400

    Fix Ruby 2.7 deprecations

commit 863ee96aafe1071c985f74decc59b624a04e507f Merge: 7cfd704de6f
2b6e5cab Author: Ethan Urie <ethan.urie@gmail.com> Date:   Wed Apr 14
08:47:41 2021 -0400

    Merge remote-tracking branch 'origin/master' into
    299558-integrate-with-anti-spam-service

commit 7cfd704de6f915230aacb01d5b45647ebadb66a7 Merge: 50f0fcba818
b49dd8e1 Author: Ethan Urie <ethan.urie@gmail.com> Date:   Tue Apr 13
15:09:35 2021 -0400

    Merged with master

commit 50f0fcba818e877e4382677b69b5ed04993da442 Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Fri Apr 9 10:03:37 2021 -0400

    Move `spamcheck_client` to `spamcheck` and updated tests

commit ee7011fc712f53e88d4d84ef3576bd06723b7314 Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Thu Apr 8 15:02:50 2021 -0400

    Capture the user's primary email address

commit 324464315216f4759411cb9383457afd9f8255c2 Merge: 1dd4f510955
5dbbb980ac1 Author: Ethan Urie <ethan.urie@gmail.com> Date:   Thu Apr 8
10:47:24 2021 -0400

    Merge branch '299558-integrate-with-anti-spam-service' of
    https://gitlab.com/gitlab-org/gitlab into
    299558-integrate-with-anti-spam-service

commit 5dbbb980ac1200e94e757354e484df17b1a7d34d Author: Jayson Salazar
<jsalazar@gitlab.com> Date:   Thu Apr 8 14:45:46 2021 +0000

    Add a BLOCK_USER case to spam_action_service.rb as otherwise the
    result would be ALLOW for external_verdict='block'

commit 1dd4f510955d137b3680d3417fb993781e945418 Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Thu Apr 8 10:44:43 2021 -0400

    Set spamcheck branch to master

commit ff64700c27f80dc0ee85dd56d5bb77f19a8abacb Author: Jayson Salazar
<jsalazar@gitlab.com> Date:   Thu Apr 8 14:43:34 2021 +0000

    Add a BLOCK_USER case to spam_action_service.rb as otherwise the
    result would be ALLOW for external_verdict='block'

commit f2af0e2833c5176dbd6907340f805a13eb566ed1 Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Wed Mar 10 17:14:15 2021 -0500

    Update application settings to support grpc spam check URL

commit e016cb86d767f6750d78fa1d4e5b42fc614c0ae1 Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Wed Mar 10 15:26:54 2021 -0500

    Created initial SpamcheckClient specs

commit ccccc3a682df9ad6730365e3dbca4f22f115cc0b Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Tue Mar 9 08:41:12 2021 -0500

    SpamVerdictService specs are passing, starting client specs

commit 1193aeeea5295aea8b81de3a6c05d5975949f5e0 Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Mon Mar 8 14:14:16 2021 -0500

    Move verdict mappings into a hash from a method

commit f014a98307b765202f784ec2c62dc9ebedbc72bb Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Fri Mar 5 15:31:49 2021 -0500

    Updated specs for GRPC

commit b40b348702b843bc5d98c45ce01516d7ce60c43d Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Wed Mar 3 14:12:15 2021 -0500

    Initial client tests with updated gem files

    Updated gems and created the first, skeletal tests for the new
    client code for Spamcheck.

commit b55a29c0bf15c0aaacf6fd5b5b6bd1eb8781b323 Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Fri Jan 22 08:36:19 2021 -0500

    Reformat JSON message to match the expeted protobuf message

commit 8e89d41ccddfdbcc84c8b678c70da7c2a667ed7a Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Thu Feb 25 13:54:16 2021 -0500

    Initial client code

commit 0e9621825cbf709bf2a5d6c40488ad38e8536be3 Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Mon Feb 22 14:35:03 2021 -0500

    Add spamcheck gem to the Gemfile

commit 1d70b4e8dd849163321a6319421438579efe85ba Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Mon Feb 1 14:32:08 2021 -0500

    Add changes to format to include multiple emails and updated_at

commit 9d4b43a234a1f5c4983a6c5d839b8b5a471cb95e Author: Ethan Urie
<ethan.urie@gmail.com> Date:   Tue Jan 26 14:59:42 2021 -0500

    Reformat JSON message to match the expeted protobuf message

It fixes Ruby 2.7 warning related to Selenium

https://github.com/teamcapybara/capybara/blob/master/History.md

Changelog: other

This opens up the possibility to actually re-use Danger rules accross
projects.
Signed-off-by: Rémy Coutable <remy@rymai.me>