When the snippet content we need to replicate it
to the snippet repository.

Not all Snippets contain Markdown content and so
the `content` field should not always be cached.

This change will attempt to determine if the content
is Markdown based on the Snippet filename and only cache
if it is

This makes a transitive dependency direct, and is a first step towards
removing `::Repository#{project,container}`

If the feature flag is enabled, the snippet has
a repository attached and it's not empty, then we'll use
the blob stored in the repository. If not, we show the snippet
stored in the database.

This behavior will change once we perform the migration.

Instead of calling store_mentions! for each model save, move it
to after_save callback to get it called and not worry to miss any save
or update calls.

We need to create the repository model
for snippets. It will be the same for both
project and personal ones.

This commit also refactors some code in
order to reuse it in projects and snippets

This renames to_reference_with_postfix to to_reference, and introduces
to_reference_base to handle the need for a base for child references

This combines all reference methods into a single example context

Mostly a search and replace for *_project_snippet and *_personal_snippet
abilities across app and spec files. Replaced with just *_snippet and
falling back on the type of subject to determine which policies to
apply.

There are some less trivial changes included which relate to inferring
an abilities name from the subject class. Because ProjectSnippet is a
child of Snippet there is some special handling around the place. There
is perhaps potential to clean this up a bit as there is the same logic
spread out in various locations.

Various changes required after review

- Renamed before_action names
- Fixed snippet note mailer with spec
- Removed incorrect/unecessary policy parameter
- Fix personal snippet note policy spec

When a note is created or updated it gets parsed and
users, groups, projects mentioned in the note are stored in respective
db table, or deleted when note is deleted.

This commit add a setting to snippets in order
to set a max limit to the content.

It can only be set through the Rails console
or the API.

The limit will affect to new snippets and existing
ones when the content is updated.

The default limit is 50MB.

For the new secret snippets feature we need to add a couple
of migrations and also a new index to include the new fields.

The "Explore snippets" page loads very slowly with a large number of
snippets, due to the complexity of the authorization checks for project
snippets.

Since project snippets are of limited interest on the explore page, we
can restrict the query to personal snippets which are visible to the
current user.

We're also replacing the index on `snippets.project_id` with a combined
index on `(project_id, visibility_level)` to further speed up these
queries.

Consider the scenario:

1. The snippet visibility level is set to internal
2. A user attempts to create a private snippet

Previously this would always fail because `default_value_for` would
overwrite the private visibility setting, no matter what
visibility_level was. This was happening because `default_value_for` was
confused by the default value of 0 specified by the database schema.

`default_value_for` attempts to assign the default value in the block by
checking whether the attribute has changed. The problem is that since
the default value by the database was 0, and the user requested 0, this
appeared as though no changes were made. As a result,
`default_value_for` would always overwrite the user's preference.

To fix this, we remove the use of `default_value_for` and only set the
visibility level to the default application setting when no preference
has been given at creation time.

This is similar to the fix in
https://gitlab.com/gitlab-org/gitlab-foss/merge_requests/29578.

Closes https://gitlab.com/gitlab-org/gitlab/issues/34016

- only for issues and merge requests
- snippets still require solving reCaptcha

This changes any mention of gitlab-ce to gitlab-foss, and any mention of
gitlab-ee to just gitlab.

- Avoid N+1 queries for authors and comment counts
- Avoid an additional snippet existence query

All instances of injecting an EE specific module have been changed to
use the new methods for this: prepend_if_ee, extend_if_ee, and
include_if_ee. This allows these lines to be included in CE, even when
the modules to inject do not exist. This in turn allows us to backport
these lines to CE and keep them there, instead of having to strip them
out.

Adding new `AddAwardEmoji`, `RemoveAwardEmoji` and `ToggleAwardEmoji`
GraphQL mutations.

Adding new `#authorized_find_with_pre_checks!` and (unused, but for
completeness `#authorized_find_with_post_checks!`) authorization
methods. These allow us to perform an authorized find, and run our own
additional checks before or after the authorization runs.

https://gitlab.com/gitlab-org/gitlab-ce/issues/62826

Adding new `AddAwardEmoji`, `RemoveAwardEmoji` and `ToggleAwardEmoji`
GraphQL mutations.

Adding new `#authorized_find_with_pre_checks!` and (unused, but for
completeness `#authorized_find_with_post_checks!`) authorization
methods. These allow us to perform an authorized find, and run our own
additional checks before or after the authorization runs.

https://gitlab.com/gitlab-org/gitlab-ce/issues/62826

Signed-off-by: Rémy Coutable <remy@rymai.me>

Signed-off-by: Rémy Coutable <remy@rymai.me>

These lines were not handled by previous work to move EE code out of CE
code, in most cases because our scripts did not cover all cases. This
commit moves a variety of EE specific Ruby changes, with the exception
of `prepend` calls, out of CE code.

This completely rewrites the SnippetsFinder class from the ground up in
order to improve its performance. The old code was beyond salvaging. It
was complex, included various Rails 5 workarounds, comments that
shouldn't be necessary, and most important of all: it produced a really
poorly performing database query.

As a result, I opted for rewriting the finder from scratch, instead of
trying to patch the existing code. Instead of trying to reuse as many
existing methods as possible, I opted for defining new methods
specifically meant for the SnippetsFinder. This requires some extra code
here and there, but allows us to have much more control over the
resulting SQL queries. It is these changes that then allow us to produce
a _much_ more efficient query.

To illustrate how bad the old query was, we will use my own snippets as
an example. Currently I have 52 snippets, most of which are global ones.
To retrieve these, you would run the following Ruby code:

    user = User.find_by(username: 'yorickpeterse')

    SnippetsFinder.new(user, author: user).execute

On GitLab.com the resulting query will take between 10 and 15 seconds to
run, producing the query plan found at
https://explain.depesz.com/s/Y5IX. Apart from the long execution time,
the total number of buffers (the sum of all shared hits) is around 185
GB, though the real number is probably (hopefully) much lower as I doubt
simply summing these numbers produces the true total number of buffers
used.

The new query's plan can be found at https://explain.depesz.com/s/wHdN,
and this query takes between 10 and 100-ish milliseconds to run. The
total number of buffers used is only about 30 MB.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/52639

This completely rewrites the SnippetsFinder class from the ground up in
order to improve its performance. The old code was beyond salvaging. It
was complex, included various Rails 5 workarounds, comments that
shouldn't be necessary, and most important of all: it produced a really
poorly performing database query.

As a result, I opted for rewriting the finder from scratch, instead of
trying to patch the existing code. Instead of trying to reuse as many
existing methods as possible, I opted for defining new methods
specifically meant for the SnippetsFinder. This requires some extra code
here and there, but allows us to have much more control over the
resulting SQL queries. It is these changes that then allow us to produce
a _much_ more efficient query.

To illustrate how bad the old query was, we will use my own snippets as
an example. Currently I have 52 snippets, most of which are global ones.
To retrieve these, you would run the following Ruby code:

    user = User.find_by(username: 'yorickpeterse')

    SnippetsFinder.new(user, author: user).execute

On GitLab.com the resulting query will take between 10 and 15 seconds to
run, producing the query plan found at
https://explain.depesz.com/s/Y5IX. Apart from the long execution time,
the total number of buffers (the sum of all shared hits) is around 185
GB, though the real number is probably (hopefully) much lower as I doubt
simply summing these numbers produces the true total number of buffers
used.

The new query's plan can be found at https://explain.depesz.com/s/wHdN,
and this query takes between 10 and 100-ish milliseconds to run. The
total number of buffers used is only about 30 MB.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/52639

It's possible that user pastes accidentally also unsubscribe link
which is included in footer of notification emails. This unsubscribe
link contains personal token which attacker then use to act as the
original user (e.g. for sending comments under his/her identity).

It's possible that user pastes accidentally also unsubscribe link
which is included in footer of notification emails. This unsubscribe
link contains personal token which attacker then use to act as the
original user (e.g. for sending comments under his/her identity).

This commit adds the module `FromUnion`, which provides the class method
`from_union`. This simplifies the process of selecting data from the
result of a UNION, and reduces the likelihood of making mistakes. As a
result, instead of this:

    union = Gitlab::SQL::Union.new([foo, bar])

    Foo.from("(#{union.to_sql}) #{Foo.table_name}")

We can now write this instead:

    Foo.from_union([foo, bar])

This commit also includes some changes to make this new setup work
properly. For example, a bug in Rails 4
(https://github.com/rails/rails/issues/24193) would break the use of
`from("sub-query-here").includes(:relation)` in certain cases. There was
also a CI query which appeared to repeat a lot of conditions from an
outer query on an inner query, which isn't necessary.

Finally, we include a RuboCop cop to ensure developers use this new
module, instead of using Gitlab::SQL::Union directly.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/51307

This commit adds the module `FromUnion`, which provides the class method
`from_union`. This simplifies the process of selecting data from the
result of a UNION, and reduces the likelihood of making mistakes. As a
result, instead of this:

    union = Gitlab::SQL::Union.new([foo, bar])

    Foo.from("(#{union.to_sql}) #{Foo.table_name}")

We can now write this instead:

    Foo.from_union([foo, bar])

This commit also includes some changes to make this new setup work
properly. For example, a bug in Rails 4
(https://github.com/rails/rails/issues/24193) would break the use of
`from("sub-query-here").includes(:relation)` in certain cases. There was
also a CI query which appeared to repeat a lot of conditions from an
outer query on an inner query, which isn't necessary.

Finally, we include a RuboCop cop to ensure developers use this new
module, instead of using Gitlab::SQL::Union directly.

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/51307

Partially addresses #47424.

The status is shown for
- The author of a commit when viewing a commit
- Notes on a commit (regular/diff)
- The user that triggered a pipeline when viewing a pipeline
- The author of a merge request when viewing a merge request
- The author of notes on a merge request (regular/diff)
- The author of an issue when viewing an issue
- The author of notes on an issue
- The author of a snippet when viewing a snippet
- The author of notes on a snippet
- A user's profile page
- The list of members of a group/user