Cache hovers offsets in a file instead of a hash

See merge request gitlab-org/gitlab-workhorse!501

Storing them in a hash consumes a lot of RAM
Introducing a file helps to reduce RAM consumption significantly

Cache ranges value in a file instead of a hash

See merge request gitlab-org/gitlab-workhorse!500

Storing them in a hash consumes a lot of RAM
Introducing a file helps to reduce RAM consumption significantly

Use simplified setup for code intelligence

See merge request gitlab-org/gitlab-workhorse!520

Truncate hover values to 250 utf-8 characters

See merge request gitlab-org/gitlab-workhorse!511

Super long documentation is not very usable, but consumes a lot
of memory

[ci skip]

Send hover tokens instead of raw html

See merge request gitlab-org/gitlab-workhorse!517

Fix SendURL functionality when the upstream lacks Content-Type

Closes #267

See merge request gitlab-org/gitlab-workhorse!510

Two cases in particular, lacking a `Content-Type`,  gave us trouble:

* Transfer-Encoding: chunked
* No content-type and no transfer-encoding

Both of these are permitted by the HTTP RFC (cases 3 and 7), and we
can talk to arbitrary HTTP servers via sendurl, so it's imperative that
we handle them correctly. This commit adds tests for both cases.

Responses of the latter type are transparently converted to responses
of the former type. This is an automatic behaviour of the Go stdlib,
which doesn't really support making the second type of response
directly. Since Transfer-Encoding is a hop-by-hop header, this type of
encoding is extremely common, and we're still streaming, instead of
accumulating, the data, I think this is acceptable.

It's a more secure way to display documentation hovers
Rendering html is also a natural task for frontend

[ci skip]

Support Workhorse directly uploading files to S3 [3/3]

See merge request gitlab-org/gitlab-workhorse!466

This adds the AWS client directly to Workhorse and a new configuration
section for specifying credentials. This makes it possible to use S3
buckets with KMS encryption and proper MD5 checksums.

This is disabled by default. For this to be used:

1. GitLab Rails needs to send the `UseWorkhorseClient` and
`RemoteTempObjectID` in the `/authorize`
endpoint. (https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29389)

2. S3 configuration must be specified in `config.toml`, or Rails must be
configured to use IAM instance profiles (`use_iam_profile` in Fog
connection parameters).

S3 sessions are created lazily and cached for 10 minutes to avoid
unnecessary local I/O access. When IAM instance profiles are used, this
also cuts down the number of HTTP requests needed to request AWS
credentials.

Related issues:

1. https://gitlab.com/gitlab-org/gitlab-workhorse/issues/222
2. https://gitlab.com/gitlab-org/gitlab-workhorse/issues/185
3. https://gitlab.com/gitlab-org/gitlab-workhorse/-/issues/210

Add configuration to support an S3 client inside Workhorse [2/3]

See merge request gitlab-org/gitlab-workhorse!516

Refactor Preparer and SaveFileOpts handling [1/3]

See merge request gitlab-org/gitlab-workhorse!515

This commit adds the configuration needed to support the Preparer
objects to extract the S3 bucket, endpoint, and other parameters needed
from Rails for Workhorse to upload a file. It does not yet use this
configuration.

This is in preparation for adding an S3 client in
https://gitlab.com/gitlab-org/gitlab-workhorse/-/merge_requests/466.

This commit ensures that a Preparer is always passed into an upload
handler and eliminates the possibility of a nil handler.

SaveFileOpts is now passed directly to downstream handlers to ensure a
cleaner interface separation between the Preparer and the handlers.

This is split out from the work for
https://gitlab.com/gitlab-org/gitlab-workhorse/-/merge_requests/466.

Disable compression for openHttpArchive

See merge request gitlab-org/gitlab-workhorse!508

[ci skip]

Gather gitlab-zip-cat/metadata stderr and log output in the current context

Closes #249

See merge request gitlab-org/gitlab-workhorse!498

Accept string or int type for LSIF ids

See merge request gitlab-org/gitlab-workhorse!499

Sync v13.0.1 security release

See merge request gitlab-org/gitlab-workhorse!512

Use labkit 96e583c57891

See merge request gitlab-org/gitlab-workhorse!506

Add test for reading artifact zips over HTTP

See merge request gitlab-org/gitlab-workhorse!509

Define Id type with custom marshalling which accepts either
int or string, because some languages (for example, Javascript)
define ids in LSIF as ints, but some languages (like Golang) use
string type for ids

It also changes int to uint32 for easier binary conversion

Add route for group imports via the UI

See merge request gitlab-org/gitlab-workhorse!493

Group imports via the UI has been implemented in the GitLab application
as part of https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29270.
Ensure that the workhorse configuration is setup for this at the same
time.

Relates to: https://gitlab.com/gitlab-org/gitlab/-/issues/211807