Clean environment variables when running git hooks

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/20977

See merge request !6344

Bump rubocop to 0.42.0

Also updates `.rubocop_todo.yml` for the latest cops.

See merge request !6354

Allow flay and flog to fail for now

flay and flog started working somehow (despite #17858). Disabling them for now since the broken builds are impeding progress.

See merge request !6360

Ignore SVG whitelist code from flay

The SVG whitelist code was generated from table definitions, and simplifying it would actually make it harder to read.

See merge request !6361

Fix wiki headeline linking

Fixes #21817

See merge request !6300

[ci skip]
Signed-off-by: Rémy Coutable <remy@rymai.me>

[ci skip]
Signed-off-by: Rémy Coutable <remy@rymai.me>

This MR fixes the migration that removes the unique index/constraint from `merge_request_diffs.merge_request_id`

See merge request !6346

For whatever reason on some PostgreSQL installations there would be a
separate UNIQUE constraint on the merge_request_id column. Rails'
index_exists?() returns false for this constraint (even when using the
full name), yet the indexes() method returns it.

This commit changes the migration so that the constraint is dropped as
well. MySQL installations don't appear to be affected.

Fixes gitlab-org/gitlab-ce#22136

Update gitlab_git to 10.6.6

## What does this MR do?

This updates gitlab_git to version 10.6.6

## Are there points in the code the reviewer needs to double check?

No.

## Why was this MR needed?

10.6.4/10.6.5/10.6.6 contains some changes regarding marking blobs as binary, this can improve page loading times.

## Does this MR meet the acceptance criteria?

- [x] ~~[CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added~~
- Tests
  - [ ] All builds are passing
- [x] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

## What are the relevant issue numbers?

Fixes https://gitlab.com/gitlab-org/gitlab-ce/issues/22144

See merge request !6279

Back-port from EE. This MR just moves the `parse_search_results` method from models/repository.rb to search_helper.rb

https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/699

See merge request !6341

Add specs for removing project

## What does this MR do?
Added more tests to removing project.

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

## What are the relevant issue numbers?

Closes #5874 

cc: @kradydal @yorickpeterse @grzesiek 

See merge request !6240

Update omniauth-facebook to '~>4.0.0'

## What does this MR do?
Update omniauth-facebook to '~>4.0.0'

## What are the relevant issue numbers?

Closes #21748 

cc @ubudzisz @yorickpeterse @grzesiek @tmaczukin

See merge request !6321

Document the fact that merge requests from forks can be checked out locally

As pointed out by @JobV, our documentation doesn't clearly state that you can checkout a merge request from a fork locally.

See merge request !6324

gitlab_git 10.6.4 relies on Rugged marking blobs as binary or not,
instead of relying on Linguist. Linguist in turn would mark text blobs
as binary whenever they would contain byte sequences that could not be
encoded using UTF-8.

However, marking such blobs as binary is not correct. If one pushes a
Markdown document with invalid character sequences it's still a text
based Markdown document and not some random binary blob.

This commit overwrites Blob#data so it automatically converts text-based
content to UTF-8 (the encoding we use everywhere else) while taking care
of replacing any invalid sequences with the UTF-8 replacement character.
The data of binary blobs is left as-is.

Exclude some pending or inactivated rows in Member scopes

An unapproved request or not-yet-accepted invite should not give access rights. Neither should a blocked user be considered a member of anything.

One visible outcome of this behaviour is that owners and masters of a group or project may be blocked, yet still receive notification emails for access requests.

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/21650

See merge request !1994

Update GITLAB_SHELL_VERSION to 3.5.0 to include the 2FA recovery commands.

See merge request !6227

Signed-off-by: Rémy Coutable <remy@rymai.me>

add tests with dependent destroy

add tests with dependent destroy

add tests

add tests to projects spec

update it title  and remove let

update it title  and remove let

remove changes after rebase

remove changes after rebase

update changelog

Fix Gitlab::Popen.popen thread-safety issue

## What does this MR do?
It changes the instance variables of `Gitlab::Popen.popen` to a local ones.

## Are there points in the code the reviewer needs to double check?
N/A

## Why was this MR needed?
It fixes a bug!

## Screenshots (if relevant)
N/A

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [ ] ~~[Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)~~
- [ ] ~~API support added~~
- Tests
  - [ ] ~~Added for this feature/bug~~
  - [x] All builds are passing
- [x] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

## What are the relevant issue numbers?
 #21842

See merge request !6194

Move pushes_since_gc to Redis

## What does this MR do?

This moves tracking of the pushes since the last Git GC to Redis to reduce DB load.

## Are there points in the code the reviewer needs to double check?

Styling mostly.

## Why was this MR needed?

Updating this column can lead to a lot of writes which in turn puts a lot of load on table vacuuming. For example, in the last hour alone we had 5067 UPDATEs for this column (per InfluxDB):

```
> select count(increment_pushes_since_gc_call_count) from sidekiq_transactions where time > now() - 1h;
name: sidekiq_transactions
--------------------------
time                    count
1473780996567714622     5067
```

## What are the relevant issue numbers?

https://gitlab.com/gitlab-org/gitlab-ce/issues/22125

See merge request !6326

Allow trailing newline in secret base64 data

Fixes https://gitlab.com/gitlab-org/omnibus-gitlab/issues/1575

See merge request !6327

This moves tracking of the pushes since the last Git GC from PostgreSQL
to Redis. This reduces the number of writes on the "projects" table.
This in turn reduces the vacuuming overhead.

The lease used for incrementing the counter has been removed. This lease
was mostly put in place to prevent high database load but this isn't
needed anymore due to the counter now being stored in Redis.

Fixes gitlab-org/gitlab-ce#22125

Use `haml_lint` for views linting

## What does this MR do?

Add HAML-linting to the GitLab codebase.

## What are the relevant issue numbers?

Closes #19984.

See merge request !6307

Fixes #21842

Change response for /ci/api/v1/builds/register.json from 404 to 204

## What does this MR do?

To check if there are new builds scheduled for a runner, runner is sending `POST /ci/api/v1/builds/register.json` requests. If there is a build then a `200` response with build data is returned. However if there is no builds scheduled for this runner it receives a `404 Not Found` response. This may end with a lot of `404 Not Found` lines in logs that are an expected behavior (please read #14445 for a reference).

Since `v1.0.1` version GitLab Runner is ready to receive a `204 No Content` response in case of no builds.

This MR adds a support for this status code for each Runner's version that can be determined while handling `POST /ci/api/v1/builds/register.json` request.

## Are there points in the code the reviewer needs to double check?

Specs in `spec/requests/ci/api/builds_spec.rb`

## Does this MR meet the acceptance criteria?

- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [x] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [ ] ~~API support added~~
- Tests
  - [x] Added for this feature/bug
  - [x] All builds are passing
- [ ] ~~Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)~~
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if you do - rebase it please)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)

## What are the relevant issue numbers?

Fixes #14445

See merge request !6225

Signed-off-by: Rémy Coutable <remy@rymai.me>

Restore SSH Key title auto-population behavior

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/21823

See merge request !6186

Update gitlab shell secret file also when it is empty

Use `File.size?` instead of `File.exist?` so that secret token is
generated when the file is empty.

See merge request !3774