- 13 Aug, 2020 1 commit
-
-
Stan Hu authored
Previously it was particularly tricky to add a new object storage method because you had to be aware of how to deal with different Goroutines and contexts to handle the Workhorse upload flow (https://docs.gitlab.com/ee/development/uploads.html#direct-upload). In addition, the execution engine to handle this was duplicated across multiple files. The execution engine essentially did the following: 1. Set up an upload context with a deadline 2. Record upload metrics 3. Initialize cleanup functions 4. Initiate upload 5. Validate upload ETag 6. Do cleanup (e.g. delete the temporary file) To reduce code duplication and to make it easier to add new object stores, the common execution sequence is now encapsulated in the `uploader` `Execute()` method. We also introduce an `UploadStrategy` interface that handles the details of the uploads, and `Execute()` calls methods on this interface. Now adding a new object storage type is a matter of implementing the `UploadStrategy` interface without needing to understand the details of the execution engine.
-
- 12 Aug, 2020 5 commits
-
-
Nick Thomas authored
-
Nick Thomas authored
[ci skip]
-
Jacob Vosmaer authored
Cache references in file See merge request gitlab-org/gitlab-workhorse!544
-
Jacob Vosmaer authored
Fix HTTP Range Requests not working on some S3 providers Closes gitlab#223806 See merge request gitlab-org/gitlab-workhorse!549
-
Patrick Bajao authored
This is to improve RAM usage when parsing references utilizing the existing `cache` struct. Reduces memory usage when parsing LSIF file of gitlab-workhorse from 7MB to 4MB.
-
- 11 Aug, 2020 4 commits
-
-
Stan Hu authored
According to https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35: Origin servers that accept byte-range requests MAY send Accept-Ranges: bytes, but are not required to do so. Clients MAY generate byte-range requests without having received this header for the resource involved. Since some servers (e.g. Dell ECS) don't send an `Accept-Ranges: bytes` header and we are already checking that range requests are supported when we call `HttpReadSeeker.rangeRequest`, `canSeek` only becomes a problem and should be removed. Since it's not clear whether https://github.com/jfbus/httprs is actively maintained, this commit applies https://github.com/jfbus/httprs/pull/6 to our vendored module. Closes https://gitlab.com/gitlab-org/gitlab/-/issues/223806
-
Jacob Vosmaer authored
Vendor httprs module See merge request gitlab-org/gitlab-workhorse!550
-
Nick Thomas authored
Bump Go version in go.mod to v1.13 See merge request gitlab-org/gitlab-workhorse!552
-
Nick Thomas authored
Release v8.38.0 See merge request gitlab-org/gitlab-workhorse!551
-
- 10 Aug, 2020 7 commits
-
-
Stan Hu authored
-
Stan Hu authored
-
Stan Hu authored
Since we dropped support for v1.12 in https://gitlab.com/gitlab-org/gitlab-workhorse/-/merge_requests/532, we should set the go.mod version to the minimum supported version, which is Go v1.13.
-
Stan Hu authored
-
Stan Hu authored
It's not clear whether this source code is maintained anymore. This is done in preparation to apply https://github.com/jfbus/httprs/pull/6, which is 2 years old.
-
Jacob Vosmaer authored
Add References struct See merge request gitlab-org/gitlab-workhorse!540
-
Jacob Vosmaer authored
Add support for AWS S3 Server Side Encryption (SSE-KMS) See merge request gitlab-org/gitlab-workhorse!537
-
- 07 Aug, 2020 1 commit
-
-
Patrick Bajao authored
Extract references parsing logic to a separate file and struct. Also modified `Item` struct to have `Line` as `int32` so it'll be fixed size. This is in preparation for another MR wherein references will be stored in file instead of memory.
-
- 06 Aug, 2020 5 commits
-
-
Stan Hu authored
Prior to this change, uploads to AWS S3 were only encrypted on the server if a default encryption were specified on the bucket. With this change, admins can now configure the encryption and the AWS Key Management Service (KMS) key ID in GitLab Rails, and the configuration will be used in uploads. Bucket policies to enforce encryption can now be used since Workhorse sends the required headers (`x-amz-server-side-encryption` and `x-amz-server-side-encryption-aws-kms-key-id`). The bucket policy cannot be enforced with default encryption, since that is applied after the check. This requires the changes in https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38240 to work. Part of https://gitlab.com/gitlab-org/gitlab/-/issues/22200
-
Nick Thomas authored
Add changelog for MR 529 See merge request gitlab-org/gitlab-workhorse!545
-
Jacob Vosmaer authored
-
Jacob Vosmaer authored
Added configuration option PropagateCorrelationID See merge request gitlab-org/gitlab-workhorse!529
-
Mahmoud Rahbar Azad authored
added configuration option PropagateCorrelationID and corresponding flag argument propagateCorrelationID to enable correlation.InboundHandlerOption: WithPropagation() for correlation.InjectCorrelationID
-
- 31 Jul, 2020 1 commit
-
-
Alessio Caiazza authored
Improve test report for performance_test.go See merge request gitlab-org/gitlab-workhorse!535
-
- 30 Jul, 2020 1 commit
-
-
Igor Drozdov authored
-
- 24 Jul, 2020 1 commit
-
-
Jacob Vosmaer authored
Drop Go v1.12 support See merge request gitlab-org/gitlab-workhorse!532
-
- 23 Jul, 2020 1 commit
-
-
Igor Drozdov authored
We officially use Go v1.13+ for everything (https://docs.gitlab.com/ee/install/requirements.html#go-versions), and dropping Go v1.12 allows us to use new methods such as `http.NewRequestWithContext`.
-
- 16 Jul, 2020 4 commits
-
-
Jacob Vosmaer authored
Document changelog as part of process See merge request gitlab-org/gitlab-workhorse!530
-
Patrick Bajao authored
Generating a changelog entry is now required when a merge request warrants a CHANGELOG entry.
-
Jacob Vosmaer authored
-
Jacob Vosmaer authored
[ci skip]
-
- 15 Jul, 2020 2 commits
-
-
Jacob Vosmaer authored
Read and parse LSIF references See merge request gitlab-org/gitlab-workhorse!524
-
Patrick Bajao authored
In order to be able to "Find references" of a given range or definition, we need to be able to read `references` items from the LSIF file. This will then be written to the JSON file for each range via `references` property. The property will look something like: ``` "references": [ { "path": "main.go#L7" } ] ``` Each reference will be an object with `path` property. The `path` property will point to the exact line where it is being used. This is currently behind a feature flag (`code_navigation_references`) which will set `ProcessLsifReferences` header to `true` when enabled. If `false`, the `references` won't be read and generated.
-
- 14 Jul, 2020 3 commits
-
-
Jacob Vosmaer authored
Refactor to prepare for parsing references See merge request gitlab-org/gitlab-workhorse!523
-
Nick Thomas authored
Eagerly delete private tempfiles See merge request gitlab-org/gitlab-workhorse!526
-
Patrick Bajao authored
Preparation for supporting other type of item (e.g. references). This MR includes the following changes: - Rename `RawDefRef` to `RawItem`. - Rename `DefRef` to `Item`.
-
- 13 Jul, 2020 3 commits
-
-
Jacob Vosmaer authored
Remove deprecated option See merge request gitlab-org/gitlab-workhorse!513
-
Nick Thomas authored
Add missing bounds checks in LSIF parser See merge request gitlab-org/gitlab-workhorse!527
-
Jacob Vosmaer authored
-
- 03 Jul, 2020 1 commit
-
-
Nick Thomas authored
Make upload filter code (EXIF, LSIF) more uniform See merge request gitlab-org/gitlab-workhorse!525
-