Merge branch '27376-go-package-mvc' into 'master'

Go Module Proxy MVC

Closes #27376

See merge request gitlab-org/gitlab!27746

config/settings.rb

@@ -66,6 +66,12 @@ class Settings < Settingslogic
       (base_url(gitlab) + [gitlab.relative_url_root]).join('')
     end
 
+    def build_gitlab_go_url
+      # "Go package paths are not URLs, and do not include port numbers"
+      # https://github.com/golang/go/issues/38213#issuecomment-607851460
+      "#{gitlab.host}#{gitlab.relative_url_root}"
+    end
+
     def kerberos_protocol
       kerberos.https ? "https" : "http"
     end

doc/administration/packages/index.md

@@ -13,6 +13,7 @@ The Packages feature allows GitLab to act as a repository for the following:
 | [Conan Repository](../../user/packages/conan_repository/index.md) | The GitLab Conan Repository enables every project in GitLab to have its own space to store [Conan](https://conan.io/) packages. | 12.4+ |
 | [Maven Repository](../../user/packages/maven_repository/index.md) | The GitLab Maven Repository enables every project in GitLab to have its own space to store [Maven](https://maven.apache.org/) packages. | 11.3+ |
 | [NPM Registry](../../user/packages/npm_registry/index.md)   | The GitLab NPM Registry enables every project in GitLab to have its own space to store [NPM](https://www.npmjs.com/) packages. | 11.7+ |
+| [Go Proxy](../../user/packages/go_proxy/index.md) | The Go proxy for GitLab enables every project in GitLab to be fetched with the [Go proxy protocol](https://proxy.golang.org/). | 13.1+ |
 
 Don't you see your package management system supported yet?
 Please consider contributing

doc/user/packages/go_proxy/index.md

+# GitLab Go Proxy **(PREMIUM)**
+
+> - [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/27376) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.1.
+> - It's deployed behind a feature flag, disabled by default.
+> - It's disabled on GitLab.com.
+> - It's not recommended for production use.
+> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-the-go-proxy). **(PREMIUM)**
+
+With the Go proxy for GitLab, every project in GitLab can be fetched with the
+[Go proxy protocol](https://proxy.golang.org/).
+
+## Prerequisites
+
+### Enable the Go proxy
+
+The Go proxy for GitLab is under development and not ready for production use, due to
+[potential performance issues with large repositories](https://gitlab.com/gitlab-org/gitlab/-/issues/218083).
+
+It is deployed behind a feature flag that is **disabled by default**.
+
+[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md)
+can enable it for your instance.
+
+To enable it:
+
+```ruby
+Feature.enable(:go_proxy) # or
+```
+
+To disable it:
+
+```ruby
+Feature.disable(:go_proxy)
+```
+
+To enable or disable it for specific projects:
+
+```ruby
+Feature.enable(:go_proxy, Project.find(1))
+Feature.disable(:go_proxy, Project.find(2))
+```
+
+### Enable the Package Registry
+
+The Package Registry is enabled for new projects by default. If you cannot find
+the **{package}** **Packages > List** entry under your project's sidebar, verify
+the following:
+
+1. Your GitLab administrator has
+   [enabled support for the Package Registry](../../../administration/packages/index.md). **(PREMIUM ONLY)**
+1. The Package Registry is [enabled for your project](../index.md).
+
+NOTE: **Note:**
+GitLab does not currently display Go modules in the **Packages Registry** of a project.
+Follow [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/213770) for details.
+
+### Fetch modules from private projects
+
+`go` does not support transmitting credentials over insecure connections. The
+steps below work only if GitLab is configured for HTTPS.
+
+1. Configure Go to include HTTP basic authentication credentials when fetching
+   from the Go proxy for GitLab.
+1. Configure Go to skip downloading of checksums for private GitLab projects
+   from the public checksum database.
+
+#### Enable Request Authentication
+
+Create a [personal access token](../../profile/personal_access_tokens.md) with
+the `api` or `read_api` scope and add it to
+[`~/.netrc`](https://ec.haxx.se/usingcurl/usingcurl-netrc):
+
+```netrc
+machine <url> login <username> password <token>
+```
+
+`<url>` should be the URL of the GitLab instance, for example `gitlab.com`.
+`<username>` and `<token>` should be your username and the personal access
+token, respectively.
+
+#### Disable checksum database queries
+
+Go can be configured to query a checksum database for module checksums. Go 1.13
+and later query `sum.golang.org` by default. This fails for modules that are not
+public and thus not accessible to `sum.golang.org`. To resolve this issue, set
+`GONOSUMDB` to a comma-separated list of projects or namespaces for which Go
+should not query the checksum database. For example, `go env -w
+GONOSUMDB=gitlab.com/my/project` persistently configures Go to skip checksum
+queries for the project `gitlab.com/my/project`.
+
+Checksum database queries can be disabled for arbitrary prefixes or disabled
+entirely. However, checksum database queries are a security mechanism and as
+such they should be disabled selectively and only when necessary. `GOSUMDB=off`
+or `GONOSUMDB=*` disables checksum queries entirely. `GONOSUMDB=gitlab.com`
+disables checksum queries for all projects hosted on GitLab.com.
+
+## Add GitLab as a Go proxy
+
+NOTE: **Note:**
+To use a Go proxy, you must be using Go 1.13 or later.
+
+The available proxy endpoints are:
+
+- Project - can fetch modules defined by a project - `/api/v4/projects/:id/packages/go`
+
+Go's use of proxies is configured with the `GOPROXY` environment variable, as a
+comma separated list of URLs. Go 1.14 adds support for comma separated list of
+URLs. Go 1.14 adds support for using `go env -w` to manage Go's environment
+variables. For example, `go env -w GOPROXY=...` writes to `$GOPATH/env`
+(which defaults to `~/.go/env`). `GOPROXY` can also be configured as a normal
+environment variable, with RC files or `export GOPROXY=...`.
+
+The default value of `$GOPROXY` is `https://proxy.golang.org,direct`, which
+tells `go` to first query `proxy.golang.org` and fallback to direct VCS
+operations (`git clone`, `svc checkout`, etc). Replacing
+`https://proxy.golang.org` with a GitLab endpoint will direct all fetches
+through GitLab. Currently GitLab's Go proxy does not support dependency
+proxying, so all external dependencies will be handled directly. If GitLab's
+endpoint is inserted before `https://proxy.golang.org`, then all fetches will
+first go through GitLab. This can help avoid making requests for private
+packages to the public proxy, but `GOPRIVATE` is a much safer way of achieving
+that.
+
+For example, with the following configuration, Go will attempt to fetch modules
+from 1) GitLab project 1234's Go module proxy, 2) `proxy.golang.org`, and
+finally 3) directly with Git (or another VCS, depending on where the module
+source is hosted).
+
+```shell
+go env -w GOPROXY=https://gitlab.com/api/v4/projects/1234/packages/go,https://proxy.golang.org,direct
+```
+
+## Release a module
+
+Go modules and module versions are handled entirely with Git (or SVN, Mercurial,
+and so on). A module is a repository containing Go source and a `go.mod` file. A
+version of a module is a Git tag (or equivalent) that is a valid [semantic
+version](https://semver.org), prefixed with 'v'. For example, `v1.0.0` and
+`v1.3.2-alpha` are valid module versions, but `v1` or `v1.2` are not.
+
+Go requires that major versions after v1 involve a change in the import path of
+the module. For example, version 2 of the module `gitlab.com/my/project` must be
+imported and released as `gitlab.com/my/project/v2`.
+
+For a complete understanding of Go modules and versioning, see [this series of
+blog posts](https://blog.golang.org/using-go-modules) on the official Go
+website.
+
+## Valid modules and versions
+
+The GitLab Go proxy will ignore modules and module versions that have an invalid
+`module` directive in their `go.mod`. Go requires that a package imported as
+`gitlab.com/my/project` can be accessed with that same URL, and that the first
+line of `go.mod` is `module gitlab.com/my/project`. If `go.mod` names a
+different module, compilation will fail. Additionally, Go requires, for major
+versions after 1, that the name of the module have an appropriate suffix, for
+example `gitlab.com/my/project/v2`. If the `module` directive does not also have
+this suffix, compilation will fail.
+
+Go supports 'pseudo-versions' that encode the timestamp and SHA of a commit.
+Tags that match the pseudo-version pattern are ignored, as otherwise they could
+interfere with fetching specific commits using a pseudo-version. Pseudo-versions
+follow one of three formats:
+
+- `vX.0.0-yyyymmddhhmmss-abcdefabcdef`, when no earlier tagged commit exists for X.
+- `vX.Y.Z-pre.0.yyyymmddhhmmss-abcdefabcdef`, when most recent prior tag is vX.Y.Z-pre.
+- `vX.Y.(Z+1)-0.yyyymmddhhmmss-abcdefabcdef`, when most recent prior tag is vX.Y.Z.

doc/user/packages/index.md

@@ -21,6 +21,7 @@ The Packages feature allows GitLab to act as a repository for the following:
 | [NPM Registry](npm_registry/index.md) **(PREMIUM)**  | The GitLab NPM Registry enables every project in GitLab to have its own space to store [NPM](https://www.npmjs.com/) packages. | 11.7+ |
 | [NuGet Repository](nuget_repository/index.md) **(PREMIUM)**  | The GitLab NuGet Repository will enable every project in GitLab to have its own space to store [NuGet](https://www.nuget.org/) packages. | 12.8+ |
 | [PyPi Repository](pypi_repository/index.md) **(PREMIUM)**  | The GitLab PyPi Repository will enable every project in GitLab to have its own space to store [PyPi](https://pypi.org/) packages. | 12.10+ |
+| [Go Proxy](go_proxy/index.md) **(PREMIUM)** | The Go proxy for GitLab enables every project in GitLab to be fetched with the [Go proxy protocol](https://proxy.golang.org/). | 13.1+ |
 
 ## Enable the Package Registry for your project
 
@@ -116,7 +117,6 @@ are adding support for [PHP](https://gitlab.com/gitlab-org/gitlab/-/merge_reques
 | [Conda](https://gitlab.com/gitlab-org/gitlab/-/issues/36891) | Secure and private local Conda repositories. |
 | [CRAN](https://gitlab.com/gitlab-org/gitlab/-/issues/36892) | Deploy and resolve CRAN packages for the R language. |
 | [Debian](https://gitlab.com/gitlab-org/gitlab/-/issues/5835) | Host and provision Debian packages. |
-| [Go](https://gitlab.com/gitlab-org/gitlab/-/issues/9773) | Resolve Go dependencies from and publish your Go packages to GitLab.  |
 | [Opkg](https://gitlab.com/gitlab-org/gitlab/-/issues/36894) | Optimize your work with OpenWrt using Opkg repositories. |
 | [P2](https://gitlab.com/gitlab-org/gitlab/-/issues/36895) | Host all your Eclipse plugins in your own GitLab P2 repository. |
 | [Puppet](https://gitlab.com/gitlab-org/gitlab/-/issues/36897) | Configuration management meets repository management with Puppet repositories. |

ee/app/finders/packages/go/module_finder.rb

+# frozen_string_literal: true
+
+module Packages
+  module Go
+    class ModuleFinder
+      include ::API::Helpers::Packages::Go::ModuleHelpers
+
+      attr_reader :project, :module_name
+
+      def initialize(project, module_name)
+        module_name = Pathname.new(module_name).cleanpath.to_s
+
+        @project = project
+        @module_name = module_name
+      end
+
+      def execute
+        return if @module_name.blank? || !@module_name.start_with?(gitlab_go_url)
+
+        module_path = @module_name[gitlab_go_url.length..].split('/')
+        project_path = project.full_path.split('/')
+        module_project_path = module_path.shift(project_path.length)
+        return unless module_project_path == project_path
+
+        Packages::GoModule.new(@project, @module_name, module_path.join('/'))
+      end
+
+      private
+
+      def gitlab_go_url
+        @gitlab_go_url ||= Settings.build_gitlab_go_url + '/'
+      end
+    end
+  end
+end

ee/app/finders/packages/go/version_finder.rb

+# frozen_string_literal: true
+
+module Packages
+  module Go
+    class VersionFinder
+      include ::API::Helpers::Packages::Go::ModuleHelpers
+
+      attr_reader :mod
+
+      def initialize(mod)
+        @mod = mod
+      end
+
+      def execute
+        @mod.project.repository.tags
+          .filter { |tag| semver? tag }
+          .map    { |tag| @mod.version_by(ref: tag) }
+          .filter { |ver| ver.valid? }
+      end
+
+      def find(target)
+        case target
+        when String
+          if pseudo_version? target
+            semver = parse_semver(target)
+            commit = pseudo_version_commit(@mod.project, semver)
+            Packages::GoModuleVersion.new(@mod, :pseudo, commit, name: target, semver: semver)
+          else
+            @mod.version_by(ref: target)
+          end
+
+        when Gitlab::Git::Ref
+          @mod.version_by(ref: target)
+
+        when ::Commit, Gitlab::Git::Commit
+          @mod.version_by(commit: target)
+
+        else
+          raise ArgumentError.new 'not a valid target'
+        end
+      end
+    end
+  end
+end

ee/app/models/packages/go_module.rb

+# frozen_string_literal: true
+
+class Packages::GoModule
+  include Gitlab::Utils::StrongMemoize
+
+  attr_reader :project, :name, :path
+
+  def initialize(project, name, path)
+    @project = project
+    @name = name
+    @path = path
+  end
+
+  def versions
+    strong_memoize(:versions) { Packages::Go::VersionFinder.new(self).execute }
+  end
+
+  def version_by(ref: nil, commit: nil)
+    raise ArgumentError.new 'no filter specified' unless ref || commit
+    raise ArgumentError.new 'ref and commit are mutually exclusive' if ref && commit
+
+    if commit
+      return version_by_sha(commit) if commit.is_a? String
+
+      return version_by_commit(commit)
+    end
+
+    return version_by_name(ref) if ref.is_a? String
+
+    version_by_ref(ref)
+  end
+
+  def path_valid?(major)
+    m = /\/v(\d+)$/i.match(@name)
+
+    case major
+    when 0, 1
+      m.nil?
+    else
+      !m.nil? && m[1].to_i == major
+    end
+  end
+
+  def gomod_valid?(gomod)
+    if Feature.enabled?(:go_proxy_disable_gomod_validation, @project)
+      return gomod&.start_with?("module ")
+    end
+
+    gomod&.split("\n", 2)&.first == "module #{@name}"
+  end
+
+  private
+
+  def version_by_name(name)
+    # avoid a Gitaly call if possible
+    if strong_memoized?(:versions)
+      v = versions.find { |v| v.name == ref }
+      return v if v
+    end
+
+    ref = @project.repository.find_tag(name) || @project.repository.find_branch(name)
+    return unless ref
+
+    version_by_ref(ref)
+  end
+
+  def version_by_ref(ref)
+    # reuse existing versions
+    if strong_memoized?(:versions)
+      v = versions.find { |v| v.ref == ref }
+      return v if v
+    end
+
+    commit = ref.dereferenced_target
+    semver = Packages::SemVer.parse(ref.name, prefixed: true)
+    Packages::GoModuleVersion.new(self, :ref, commit, ref: ref, semver: semver)
+  end
+
+  def version_by_sha(sha)
+    commit = @project.commit_by(oid: sha)
+    return unless ref
+
+    version_by_commit(commit)
+  end
+
+  def version_by_commit(commit)
+    Packages::GoModuleVersion.new(self, :commit, commit)
+  end
+end

ee/app/models/packages/go_module_version.rb

+# frozen_string_literal: true
+
+class Packages::GoModuleVersion
+  include Gitlab::Utils::StrongMemoize
+  include ::API::Helpers::Packages::Go::ModuleHelpers
+
+  VALID_TYPES = %i[ref commit pseudo].freeze
+
+  attr_reader :mod, :type, :ref, :commit
+
+  delegate :major, to: :@semver, allow_nil: true
+  delegate :minor, to: :@semver, allow_nil: true
+  delegate :patch, to: :@semver, allow_nil: true
+  delegate :prerelease, to: :@semver, allow_nil: true
+  delegate :build, to: :@semver, allow_nil: true
+
+  def initialize(mod, type, commit, name: nil, semver: nil, ref: nil)
+    raise ArgumentError.new("invalid type '#{type}'") unless VALID_TYPES.include? type
+    raise ArgumentError.new("mod is required") unless mod
+    raise ArgumentError.new("commit is required") unless commit
+
+    if type == :ref
+      raise ArgumentError.new("ref is required") unless ref
+    elsif type == :pseudo
+      raise ArgumentError.new("name is required") unless name
+      raise ArgumentError.new("semver is required") unless semver
+    end
+
+    @mod = mod
+    @type = type
+    @commit = commit
+    @name = name if name
+    @semver = semver if semver
+    @ref = ref if ref
+  end
+
+  def name
+    @name || @ref&.name
+  end
+
+  def full_name
+    "#{mod.name}@#{name || commit.sha}"
+  end
+
+  def gomod
+    strong_memoize(:gomod) do
+      if strong_memoized?(:blobs)
+        blob_at(@mod.path + '/go.mod')
+      elsif @mod.path.empty?
+        @mod.project.repository.blob_at(@commit.sha, 'go.mod')&.data
+      else
+        @mod.project.repository.blob_at(@commit.sha, @mod.path + '/go.mod')&.data
+      end
+    end
+  end
+
+  def archive
+    suffix_len = @mod.path == '' ? 0 : @mod.path.length + 1
+
+    Zip::OutputStream.write_buffer do |zip|
+      files.each do |file|
+        zip.put_next_entry "#{full_name}/#{file[suffix_len...]}"
+        zip.write blob_at(file)
+      end
+    end
+  end
+
+  def files
+    strong_memoize(:files) do
+      ls_tree.filter { |e| !excluded.any? { |n| e.start_with? n } }
+    end
+  end
+
+  def excluded
+    strong_memoize(:excluded) do
+      ls_tree
+        .filter { |f| f.end_with?('/go.mod') && f != @mod.path + '/go.mod' }
+        .map    { |f| f[0..-7] }
+    end
+  end
+
+  def valid?
+    @mod.path_valid?(major) && @mod.gomod_valid?(gomod)
+  end
+
+  private
+
+  def blob_at(path)
+    return if path.nil? || path.empty?
+
+    path = path[1..] if path.start_with? '/'
+
+    blobs.find { |x| x.path == path }&.data
+  end
+
+  def blobs
+    strong_memoize(:blobs) { @mod.project.repository.batch_blobs(files.map { |x| [@commit.sha, x] }) }
+  end
+
+  def ls_tree
+    strong_memoize(:ls_tree) do
+      path =
+        if @mod.path.empty?
+          '.'
+        else
+          @mod.path
+        end
+
+      @mod.project.repository.gitaly_repository_client.search_files_by_name(@commit.sha, path)
+    end
+  end
+end

ee/app/models/packages/sem_ver.rb

+# frozen_string_literal: true
+
+class Packages::SemVer
+  attr_accessor :major, :minor, :patch, :prerelease, :build
+
+  def initialize(major = 0, minor = 0, patch = 0, prerelease = nil, build = nil, prefixed: false)
+    @major = major
+    @minor = minor
+    @patch = patch
+    @prerelease = prerelease
+    @build = build
+    @prefixed = prefixed
+  end
+
+  def prefixed?
+    @prefixed
+  end
+
+  def ==(other)
+    self.class == other.class &&
+    self.major == other.major &&
+    self.minor == other.minor &&
+    self.patch == other.patch &&
+    self.prerelease == other.prerelease &&
+    self.build == other.build
+  end
+
+  def to_s
+    s = "#{prefixed? ? 'v' : ''}#{major || 0}.#{minor || 0}.#{patch || 0}"
+    s += "-#{prerelease}" if prerelease
+    s += "+#{build}" if build
+
+    s
+  end
+
+  def self.match(str, prefixed: false)
+    return unless str&.start_with?('v') == prefixed
+
+    str = str[1..] if prefixed
+
+    Gitlab::Regex.semver_regex.match(str)
+  end
+
+  def self.match?(str, prefixed: false)
+    !match(str, prefixed: prefixed).nil?
+  end
+
+  def self.parse(str, prefixed: false)
+    m = match str, prefixed: prefixed
+    return unless m
+
+    new(m[1].to_i, m[2].to_i, m[3].to_i, m[4], m[5], prefixed: prefixed)
+  end
+end

ee/app/presenters/packages/go/module_version_presenter.rb

+# frozen_string_literal: true
+
+module Packages
+  module Go
+    class ModuleVersionPresenter
+      def initialize(version)
+        @version = version
+      end
+
+      def name
+        @version.name
+      end
+
+      def time
+        @version.commit.committed_date
+      end
+    end
+  end
+end

ee/changelogs/unreleased/27376-go-package-mvc.yml

+---
+title: Implement Go module proxy MVC (package manager for Go)
+merge_request: 27746
+author: Ethan Reesor
+type: added

ee/lib/api/go_proxy.rb

+# frozen_string_literal: true
+module API
+  class GoProxy < Grape::API
+    helpers ::API::Helpers::PackagesHelpers
+    helpers ::API::Helpers::Packages::Go::ModuleHelpers
+
+    # basic semver, except case encoded (A => !a)
+    MODULE_VERSION_REGEX = /v(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-([-.!a-z0-9]+))?(?:\+([-.!a-z0-9]+))?/.freeze
+
+    MODULE_VERSION_REQUIREMENTS = { module_version: MODULE_VERSION_REGEX }.freeze
+
+    before { require_packages_enabled! }
+
+    helpers do
+      def find_project!(id)
+        # based on API::Helpers::Packages::BasicAuthHelpers#authorized_project_find!
+
+        project = find_project(id)
+
+        return project if project && can?(current_user, :read_project, project)
+
+        if current_user
+          not_found!('Project')
+        else
+          unauthorized!
+        end
+      end
+
+      def find_module
+        not_found! unless Feature.enabled?(:go_proxy, user_project)
+
+        module_name = case_decode params[:module_name]
+        bad_request!('Module Name') if module_name.blank?
+
+        mod = ::Packages::Go::ModuleFinder.new(user_project, module_name).execute
+
+        not_found! unless mod
+
+        mod
+      end
+
+      def find_version
+        module_version = case_decode params[:module_version]
+        ver = ::Packages::Go::VersionFinder.new(find_module).find(module_version)
+
+        not_found! unless ver&.valid?
+
+        ver
+
+      rescue ArgumentError
+        not_found!
+      end
+    end
+
+    params do
+      requires :id, type: String, desc: 'The ID of a project'
+      requires :module_name, type: String, desc: 'Module name', coerce_with: ->(val) { CGI.unescape(val) }
+    end
+    route_setting :authentication, job_token_allowed: true, basic_auth_personal_access_token: true
+    resource :projects, requirements: API::NAMESPACE_OR_PROJECT_REQUIREMENTS do
+      before do
+        authorize_read_package!
+        authorize_packages_feature!
+      end
+
+      namespace ':id/packages/go/*module_name/@v' do
+        desc 'Get all tagged versions for a given Go module' do
+          detail 'See `go help goproxy`, GET $GOPROXY/<module>/@v/list. This feature was introduced in GitLab 13.1.'
+        end
+        get 'list' do
+          mod = find_module
+
+          content_type 'text/plain'
+          mod.versions.map { |t| t.name }.join("\n")
+        end
+
+        desc 'Get information about the given module version' do
+          detail 'See `go help goproxy`, GET $GOPROXY/<module>/@v/<version>.info. This feature was introduced in GitLab 13.1.'
+          success EE::API::Entities::GoModuleVersion
+        end
+        params do
+          requires :module_version, type: String, desc: 'Module version'
+        end
+        get ':module_version.info', requirements: MODULE_VERSION_REQUIREMENTS do
+          ver = find_version
+
+          present ::Packages::Go::ModuleVersionPresenter.new(ver), with: EE::API::Entities::GoModuleVersion
+        end
+
+        desc 'Get the module file of the given module version' do
+          detail 'See `go help goproxy`, GET $GOPROXY/<module>/@v/<version>.mod. This feature was introduced in GitLab 13.1.'
+        end
+        params do
+          requires :module_version, type: String, desc: 'Module version'
+        end
+        get ':module_version.mod', requirements: MODULE_VERSION_REQUIREMENTS do
+          ver = find_version
+
+          content_type 'text/plain'
+          ver.gomod
+        end
+
+        desc 'Get a zip of the source of the given module version' do
+          detail 'See `go help goproxy`, GET $GOPROXY/<module>/@v/<version>.zip. This feature was introduced in GitLab 13.1.'
+        end
+        params do
+          requires :module_version, type: String, desc: 'Module version'
+        end
+        get ':module_version.zip', requirements: MODULE_VERSION_REQUIREMENTS do
+          ver = find_version
+
+          content_type 'application/zip'
+          env['api.format'] = :binary
+          header['Content-Disposition'] = ActionDispatch::Http::ContentDisposition.format(disposition: 'attachment', filename: ver.name + '.zip')
+          header['Content-Transfer-Encoding'] = 'binary'
+          status :ok
+          body ver.archive.string
+        end
+      end
+    end
+  end
+end

ee/lib/api/helpers/packages/go/module_helpers.rb

+# frozen_string_literal: true
+
+module API
+  module Helpers
+    module Packages
+      module Go
+        module ModuleHelpers
+          def case_encode(str)
+            # Converts "github.com/Azure" to "github.com/!azure"
+            #
+            # From `go help goproxy`:
+            #
+            # > To avoid problems when serving from case-sensitive file systems,
+            # > the <module> and <version> elements are case-encoded, replacing
+            # > every uppercase letter with an exclamation mark followed by the
+            # > corresponding lower-case letter: github.com/Azure encodes as
+            # > github.com/!azure.
+
+            str.gsub(/A-Z/) { |s| "!#{s.downcase}"}
+          end
+
+          def case_decode(str)
+            # Converts "github.com/!azure" to "github.com/Azure"
+            #
+            # See #case_encode
+
+            str.gsub(/![[:alpha:]]/) { |s| s[1..].upcase }
+          end
+
+          def semver?(tag)
+            return false if tag.dereferenced_target.nil?
+
+            ::Packages::SemVer.match?(tag.name, prefixed: true)
+          end
+
+          def pseudo_version?(version)
+            return false unless version
+
+            if version.is_a? String
+              version = parse_semver version
+              return false unless version
+            end
+
+            pre = version.prerelease
+
+            # Valid pseudo-versions are:
+            #   vX.0.0-yyyymmddhhmmss-sha1337beef0, when no earlier tagged commit exists for X
+            #   vX.Y.Z-pre.0.yyyymmddhhmmss-sha1337beef0, when most recent prior tag is vX.Y.Z-pre
+            #   vX.Y.(Z+1)-0.yyyymmddhhmmss-sha1337beef0, when most recent prior tag is vX.Y.Z
+
+            if version.minor != 0 || version.patch != 0
+              m = /\A(.*\.)?0\./.freeze.match pre
+              return false unless m
+
+              pre = pre[m[0].length..]
+            end
+
+            # This pattern is intentionally more forgiving than the patterns
+            # above. Correctness is verified by #pseudo_version_commit.
+            /\A\d{14}-\h+\z/.freeze.match? pre
+          end
+
+          def pseudo_version_commit(project, semver)
+            # Per Go's implementation of pseudo-versions, a tag should be
+            # considered a pseudo-version if it matches one of the patterns
+            # listed in #pseudo_version?, regardless of the content of the
+            # timestamp or the length of the SHA fragment. However, an error
+            # should be returned if the timestamp is not correct or if the SHA
+            # fragment is not exactly 12 characters long. See also Go's
+            # implementation of:
+            #
+            # - [*codeRepo.validatePseudoVersion](https://github.com/golang/go/blob/daf70d6c1688a1ba1699c933b3c3f04d6f2f73d9/src/cmd/go/internal/modfetch/coderepo.go#L530)
+            # - [Pseudo-version parsing](https://github.com/golang/go/blob/master/src/cmd/go/internal/modfetch/pseudo.go)
+            # - [Pseudo-version request processing](https://github.com/golang/go/blob/master/src/cmd/go/internal/modfetch/coderepo.go)
+
+            # Go ignores anything before '.' or after the second '-', so we will do the same
+            timestamp, sha = semver.prerelease.split('-').last 2
+            timestamp = timestamp.split('.').last
+            commit = project.repository.commit_by(oid: sha)
+
+            # Error messages are based on the responses of proxy.golang.org
+
+            # Verify that the SHA fragment references a commit
+            raise ArgumentError.new 'invalid pseudo-version: unknown commit' unless commit
+
+            # Require the SHA fragment to be 12 characters long
+            raise ArgumentError.new 'invalid pseudo-version: revision is shorter than canonical' unless sha.length == 12
+
+            # Require the timestamp to match that of the commit
+            raise ArgumentError.new 'invalid pseudo-version: does not match version-control timestamp' unless commit.committed_date.strftime('%Y%m%d%H%M%S') == timestamp
+
+            commit
+          end
+
+          def parse_semver(str)
+            ::Packages::SemVer.parse(str, prefixed: true)
+          end
+        end
+      end
+    end
+  end
+end

ee/lib/ee/api/api.rb

@@ -38,6 +38,7 @@ module EE
         mount ::API::ConanPackages
         mount ::API::MavenPackages
         mount ::API::NpmPackages
+        mount ::API::GoProxy
         mount ::API::MergeTrains
         mount ::API::ProjectPackages
         mount ::API::GroupPackages

ee/lib/ee/api/entities/go_module_version.rb

+# frozen_string_literal: true
+
+module EE
+  module API
+    module Entities
+      class GoModuleVersion < Grape::Entity
+        expose :name, as: 'Version'
+        expose :time, as: 'Time'
+      end
+    end
+  end
+end

ee/spec/factories/go_module_commits.rb

+# frozen_string_literal: true
+
+FactoryBot.define do
+  factory :go_module_commit, class: 'Commit' do
+    skip_create
+
+    transient do
+      project { raise ArgumentError.new("project is required") }
+      service { raise ArgumentError.new("this factory cannot be used without specifying a trait") }
+      tag { nil }
+      tag_message { nil }
+
+      commit do
+        r = service.execute
+
+        raise "operation failed: #{r}" unless r[:status] == :success
+
+        commit = project.repository.commit_by(oid: r[:result])
+
+        if tag
+          r = Tags::CreateService.new(project, project.owner).execute(tag, commit.sha, tag_message)
+
+          raise "operation failed: #{r}" unless r[:status] == :success
+        end
+
+        commit
+      end
+    end
+
+    initialize_with do
+      commit
+    end
+
+    trait :files do
+      transient do
+        files { raise ArgumentError.new("files is required") }
+        message { 'Add files' }
+      end
+
+      service do
+        Files::MultiService.new(
+          project,
+          project.owner,
+          commit_message: message,
+          start_branch: project.repository.root_ref || 'master',
+          branch_name: project.repository.root_ref || 'master',
+          actions: files.map do |path, content|
+            { action: :create, file_path: path, content: content }
+          end
+        )
+      end
+    end
+
+    trait :package do
+      transient do
+        path { raise ArgumentError.new("path is required") }
+        message { 'Add package' }
+      end
+
+      service do
+        Files::MultiService.new(
+          project,
+          project.owner,
+          commit_message: message,
+          start_branch: project.repository.root_ref || 'master',
+          branch_name: project.repository.root_ref || 'master',
+          actions: [
+            { action: :create, file_path: path + '/b.go', content: "package b\nfunc Bye() { println(\"Goodbye world!\") }\n" }
+          ]
+        )
+      end
+    end
+
+    trait :module do
+      transient do
+        name { nil }
+        message { 'Add module' }
+
+        url do
+          v = "#{::Gitlab.config.gitlab.host}/#{project.path_with_namespace}"
+
+          if name
+            v + '/' + name
+          else
+            v
+          end
+        end
+
+        path do
+          if name
+            name + '/'
+          else
+            ''
+          end
+        end
+      end
+
+      service do
+        Files::MultiService.new(
+          project,
+          project.owner,
+          commit_message: message,
+          start_branch: project.repository.root_ref || 'master',
+          branch_name: project.repository.root_ref || 'master',
+          actions: [
+            { action: :create, file_path: path + 'go.mod', content: "module #{url}\n" },
+            { action: :create, file_path: path + 'a.go', content: "package a\nfunc Hi() { println(\"Hello world!\") }\n" }
+          ]
+        )
+      end
+    end
+  end
+end

ee/spec/factories/go_module_versions.rb

+# frozen_string_literal: true
+
+FactoryBot.define do
+  factory :go_module_version, class: 'Packages::GoModuleVersion' do
+    skip_create
+
+    initialize_with do
+      p = attributes[:params]
+      s = Packages::SemVer.parse(p.semver, prefixed: true)
+
+      raise ArgumentError.new("invalid sematic version: '#{p.semver}''") if !s && p.semver
+
+      new(p.mod, p.type, p.commit, name: p.name, semver: s, ref: p.ref)
+    end
+
+    mod { create :go_module }
+    type { :commit }
+    commit { raise ArgumentError.new("commit is required") }
+    name { nil }
+    semver { nil }
+    ref { nil }
+
+    params { OpenStruct.new(mod: mod, type: type, commit: commit, name: name, semver: semver, ref: ref) }
+
+    trait :tagged do
+      name { raise ArgumentError.new("name is required") }
+      ref { mod.project.repository.find_tag(name) }
+      commit { ref.dereferenced_target }
+
+      params { OpenStruct.new(mod: mod, type: :ref, commit: commit, semver: name, ref: ref) }
+    end
+
+    trait :pseudo do
+      transient do
+        prefix { raise ArgumentError.new("prefix is required") }
+      end
+
+      type { :pseudo }
+      name { "#{prefix}#{commit.committed_date.strftime('%Y%m%d%H%M%S')}-#{commit.sha[0..11]}" }
+
+      params { OpenStruct.new(mod: mod, type: :pseudo, commit: commit, name: name, semver: name) }
+    end
+  end
+end

ee/spec/factories/go_modules.rb

+# frozen_string_literal: true
+
+FactoryBot.define do
+  factory :go_module, class: 'Packages::GoModule' do
+    initialize_with { new(attributes[:project], attributes[:name], attributes[:path]) }
+    skip_create
+
+    project
+    path { '' }
+    name { "#{Settings.build_gitlab_go_url}/#{project.full_path}#{path.empty? ? '' : '/'}#{path}" }
+  end
+end

ee/spec/finders/packages/go/module_finder_spec.rb

+# frozen_string_literal: true
+
+require 'spec_helper'
+
+describe Packages::Go::ModuleFinder do
+  let_it_be(:project) { create :project }
+  let_it_be(:other_project) { create :project }
+  let(:finder) { described_class.new project, module_name }
+
+  shared_examples 'an invalid path' do
+    describe '#module_name' do
+      it 'returns the expected name' do
+        expect(finder.module_name).to eq(expected_name)
+      end
+    end
+
+    describe '#execute' do
+      it 'returns nil' do
+        expect(finder.execute).to be_nil
+      end
+    end
+  end
+
+  describe '#execute' do
+    context 'with module name equal to project name' do
+      let(:module_name) { base_url(project) }
+
+      it 'returns a module with empty path' do
+        mod = finder.execute
+        expect(mod).not_to be_nil
+        expect(mod.path).to eq('')
+      end
+    end
+
+    context 'with module name starting with project name and slash' do
+      let(:module_name) { base_url(project) + '/mod' }
+
+      it 'returns a module with non-empty path' do
+        mod = finder.execute
+        expect(mod).not_to be_nil
+        expect(mod.path).to eq('mod')
+      end
+    end
+
+    context 'with a module name not equal to and not starting with project name' do
+      let(:module_name) { base_url(other_project) }
+
+      it 'returns nil' do
+        expect(finder.execute).to be_nil
+      end
+    end
+  end
+
+  context 'with relative path component' do
+    it_behaves_like 'an invalid path' do
+      let(:module_name) { base_url(project) + '/../xyz' }
+      let(:expected_name) { base_url(project.namespace) + '/xyz' }
+    end
+  end
+
+  context 'with many relative path components' do
+    it_behaves_like 'an invalid path' do
+      let(:module_name) { base_url(project) + ('/..' * 10) + '/xyz' }
+      let(:expected_name) { ('../' * 7) + 'xyz' }
+    end
+  end
+
+  def base_url(project)
+    "#{Settings.build_gitlab_go_url}/#{project.full_path}"
+  end
+end

ee/spec/finders/packages/go/version_finder_spec.rb

+# frozen_string_literal: true
+
+require 'spec_helper'
+
+describe Packages::Go::VersionFinder do
+  let_it_be(:user) { create :user }
+  let_it_be(:project) { create :project_empty_repo, creator: user, path: 'my-go-lib' }
+
+  let(:finder) { described_class.new mod }
+
+  before :all do
+    create :go_module_commit, :files,   project: project, tag: 'v1.0.0', files: { 'README.md' => 'Hi' }
+    create :go_module_commit, :module,  project: project, tag: 'v1.0.1'
+    create :go_module_commit, :package, project: project, tag: 'v1.0.2', path: 'pkg'
+    create :go_module_commit, :module,  project: project, tag: 'v1.0.3', name: 'mod'
+    create :go_module_commit, :module,  project: project, tag: 'v1.0.4', name: 'bad-mod', url: 'example.com/go-lib'
+    create :go_module_commit, :files,   project: project, tag: 'c1',     files: { 'y.go' => "package a\n" }
+    create :go_module_commit, :module,  project: project, tag: 'c2',     name: 'v2'
+    create :go_module_commit, :files,   project: project, tag: 'v2.0.0', files: { 'v2/x.go' => "package a\n" }
+  end
+
+  before do
+    stub_feature_flags(go_proxy_disable_gomod_validation: false)
+  end
+
+  shared_examples '#execute' do |*expected|
+    it "returns #{expected.empty? ? 'nothing' : expected.join(', ')}" do
+      actual = finder.execute.map { |x| x.name }
+      expect(actual.to_set).to eq(expected.to_set)
+    end
+  end
+
+  shared_examples '#find with an invalid argument' do |message|
+    it "raises an argument exception: #{message}" do
+      expect { finder.find(target) }.to raise_error(ArgumentError, message)
+    end
+  end
+
+  describe '#execute' do
+    context 'for the root module' do
+      let(:mod) { create :go_module, project: project }
+
+      it_behaves_like '#execute', 'v1.0.1', 'v1.0.2', 'v1.0.3', 'v1.0.4'
+    end
+
+    context 'for the package' do
+      let(:mod) { create :go_module, project: project, path: 'pkg' }
+
+      it_behaves_like '#execute'
+    end
+
+    context 'for the submodule' do
+      let(:mod) { create :go_module, project: project, path: 'mod' }
+
+      it_behaves_like '#execute', 'v1.0.3', 'v1.0.4'
+    end
+
+    context 'for the root module v2' do
+      let(:mod) { create :go_module, project: project, path: 'v2' }
+
+      it_behaves_like '#execute', 'v2.0.0'
+    end
+
+    context 'for the bad module' do
+      let(:mod) { create :go_module, project: project, path: 'bad-mod' }
+
+      context 'with gomod checking enabled' do
+        it_behaves_like '#execute'
+      end
+
+      context 'with gomod checking disabled' do
+        before do
+          stub_feature_flags(go_proxy_disable_gomod_validation: true)
+        end
+
+        it_behaves_like '#execute', 'v1.0.4'
+      end
+    end
+  end
+
+  describe '#find' do
+    let(:mod) { create :go_module, project: project }
+
+    context 'with a ref' do
+      it 'returns a ref version' do
+        ref = project.repository.find_branch 'master'
+        v = finder.find(ref)
+        expect(v.type).to eq(:ref)
+        expect(v.ref).to eq(ref)
+      end
+    end
+
+    context 'with a semver tag' do
+      it 'returns a version with a semver' do
+        v = finder.find(project.repository.find_tag('v1.0.0'))
+        expect(v.major).to eq(1)
+        expect(v.minor).to eq(0)
+        expect(v.patch).to eq(0)
+        expect(v.prerelease).to be_nil
+        expect(v.build).to be_nil
+      end
+    end
+
+    context 'with a semver tag string' do
+      it 'returns a version with a semver' do
+        v = finder.find('v1.0.1')
+        expect(v.major).to eq(1)
+        expect(v.minor).to eq(0)
+        expect(v.patch).to eq(1)
+        expect(v.prerelease).to be_nil
+        expect(v.build).to be_nil
+      end
+    end
+
+    context 'with a commit' do
+      it 'retruns a commit version' do
+        v = finder.find(project.repository.head_commit)
+        expect(v.type).to eq(:commit)
+      end
+    end
+
+    context 'with a pseudo-version' do
+      it 'returns a pseudo version' do
+        commit = project.repository.head_commit
+        pseudo = "v0.0.0-#{commit.committed_date.strftime('%Y%m%d%H%M%S')}-#{commit.sha[0..11]}"
+        v = finder.find(pseudo)
+        expect(v.type).to eq(:pseudo)
+        expect(v.commit).to eq(commit)
+        expect(v.name).to eq(pseudo)
+      end
+    end
+
+    context 'with a string that is not a semantic version' do
+      it 'returns nil' do
+        expect(finder.find('not-a-semver')).to be_nil
+      end
+    end
+
+    context 'with a pseudo-version that does not reference a commit' do
+      it_behaves_like '#find with an invalid argument', 'invalid pseudo-version: unknown commit' do
+        let(:commit) { project.repository.head_commit }
+        let(:target) { "v0.0.0-#{commit.committed_date.strftime('%Y%m%d%H%M%S')}-#{'0' * 12}" }
+      end
+    end
+
+    context 'with a pseudo-version with a short sha' do
+      it_behaves_like '#find with an invalid argument', 'invalid pseudo-version: revision is shorter than canonical' do
+        let(:commit) { project.repository.head_commit }
+        let(:target) { "v0.0.0-#{commit.committed_date.strftime('%Y%m%d%H%M%S')}-#{commit.sha[0..10]}" }
+      end
+    end
+
+    context 'with a pseudo-version with an invalid timestamp' do
+      it_behaves_like '#find with an invalid argument', 'invalid pseudo-version: does not match version-control timestamp' do
+        let(:commit) { project.repository.head_commit }
+        let(:target) { "v0.0.0-#{'0' * 14}-#{commit.sha[0..11]}" }
+      end
+    end
+  end
+end

ee/spec/models/packages/go_module_spec.rb

+# frozen_string_literal: true
+
+require 'spec_helper'
+
+describe Packages::GoModule, type: :model do
+  before do
+    stub_feature_flags(go_proxy_disable_gomod_validation: false)
+  end
+
+  describe '#path_valid?' do
+    context 'with root path' do
+      let_it_be(:package) { create(:go_module) }
+
+      context 'with major version 0' do
+        it('returns true') { expect(package.path_valid?(0)).to eq(true) }
+      end
+
+      context 'with major version 1' do
+        it('returns true') { expect(package.path_valid?(1)).to eq(true) }
+      end
+
+      context 'with major version 2' do
+        it('returns false') { expect(package.path_valid?(2)).to eq(false) }
+      end
+    end
+
+    context 'with path ./v2' do
+      let_it_be(:package) { create(:go_module, path: '/v2') }
+
+      context 'with major version 0' do
+        it('returns false') { expect(package.path_valid?(0)).to eq(false) }
+      end
+
+      context 'with major version 1' do
+        it('returns false') { expect(package.path_valid?(1)).to eq(false) }
+      end
+
+      context 'with major version 2' do
+        it('returns true') { expect(package.path_valid?(2)).to eq(true) }
+      end
+    end
+  end
+
+  describe '#gomod_valid?' do
+    let_it_be(:package) { create(:go_module) }
+
+    context 'with good gomod' do
+      it('returns true') { expect(package.gomod_valid?("module #{package.name}")).to eq(true) }
+    end
+
+    context 'with bad gomod' do
+      it('returns false') { expect(package.gomod_valid?("module #{package.name}/v2")).to eq(false) }
+    end
+
+    context 'with empty gomod' do
+      it('returns false') { expect(package.gomod_valid?("")).to eq(false) }
+    end
+  end
+end

ee/spec/models/packages/go_module_version_spec.rb

+# frozen_string_literal: true
+
+require 'spec_helper'
+
+describe Packages::GoModuleVersion, type: :model do
+  let_it_be(:user) { create :user }
+  let_it_be(:project) { create :project_empty_repo, creator: user, path: 'my-go-lib' }
+  let_it_be(:mod) { create :go_module, project: project }
+
+  before :all do
+    create :go_module_commit, :files,   project: project, tag: 'v1.0.0', files: { 'README.md' => 'Hi' }
+    create :go_module_commit, :module,  project: project, tag: 'v1.0.1'
+    create :go_module_commit, :package, project: project, tag: 'v1.0.2', path: 'pkg'
+    create :go_module_commit, :module,  project: project, tag: 'v1.0.3', name: 'mod'
+    create :go_module_commit, :files,   project: project,                files: { 'y.go' => "package a\n" }
+    create :go_module_commit, :module,  project: project,                name: 'v2'
+    create :go_module_commit, :files,   project: project, tag: 'v2.0.0', files: { 'v2/x.go' => "package a\n" }
+  end
+
+  shared_examples '#files' do |desc, *entries|
+    it "returns #{desc}" do
+      actual = version.files.map { |x| x }.to_set
+      expect(actual).to eq(entries.to_set)
+    end
+  end
+
+  shared_examples '#archive' do |desc, *entries|
+    it "returns an archive of #{desc}" do
+      expected = entries.map { |e| "#{version.full_name}/#{e}" }.to_set
+
+      actual = Set[]
+      Zip::InputStream.open(StringIO.new(version.archive.string)) do |zip|
+        while (entry = zip.get_next_entry)
+          actual.add(entry.name)
+        end
+      end
+
+      expect(actual).to eq(expected)
+    end
+  end
+
+  describe '#name' do
+    context 'with ref and name specified' do
+      let_it_be(:version) { create :go_module_version, mod: mod, name: 'foobar', commit: project.repository.head_commit, ref: project.repository.find_tag('v1.0.0') }
+      it('returns that name') { expect(version.name).to eq('foobar') }
+    end
+
+    context 'with ref specified and name unspecified' do
+      let_it_be(:version) { create :go_module_version, mod: mod, commit: project.repository.head_commit, ref: project.repository.find_tag('v1.0.0') }
+      it('returns the name of the ref') { expect(version.name).to eq('v1.0.0') }
+    end
+
+    context 'with ref and name unspecified' do
+      let_it_be(:version) { create :go_module_version, mod: mod, commit: project.repository.head_commit }
+      it('returns nil') { expect(version.name).to eq(nil) }
+    end
+  end
+
+  describe '#gomod' do
+    context 'with go.mod missing' do
+      let_it_be(:version) { create :go_module_version, :tagged, mod: mod, name: 'v1.0.0' }
+      it('returns nil') { expect(version.gomod).to eq(nil) }
+    end
+
+    context 'with go.mod present' do
+      let_it_be(:version) { create :go_module_version, :tagged, mod: mod, name: 'v1.0.1' }
+      it('returns the contents of go.mod') { expect(version.gomod).to eq("module #{mod.name}\n") }
+    end
+  end
+
+  describe '#files' do
+    context 'with a root module' do
+      context 'with an empty module path' do
+        let_it_be(:version) { create :go_module_version, :tagged, mod: mod, name: 'v1.0.2' }
+        it_behaves_like '#files', 'all the files', 'README.md', 'go.mod', 'a.go', 'pkg/b.go'
+      end
+    end
+
+    context 'with a root module and a submodule' do
+      context 'with an empty module path' do
+        let_it_be(:version) { create :go_module_version, :tagged, mod: mod, name: 'v1.0.3' }
+        it_behaves_like '#files', 'files excluding the submodule', 'README.md', 'go.mod', 'a.go', 'pkg/b.go'
+      end
+
+      context 'with the submodule\'s path' do
+        let_it_be(:mod) { create :go_module, project: project, path: 'mod' }
+        let_it_be(:version) { create :go_module_version, :tagged, mod: mod, name: 'v1.0.3' }
+        it_behaves_like '#files', 'the submodule\'s files', 'mod/go.mod', 'mod/a.go'
+      end
+    end
+  end
+
+  describe '#archive' do
+    context 'with a root module' do
+      context 'with an empty module path' do
+        let_it_be(:version) { create :go_module_version, :tagged, mod: mod, name: 'v1.0.2' }
+        it_behaves_like '#archive', 'all the files', 'README.md', 'go.mod', 'a.go', 'pkg/b.go'
+      end
+    end
+
+    context 'with a root module and a submodule' do
+      context 'with an empty module path' do
+        let_it_be(:version) { create :go_module_version, :tagged, mod: mod, name: 'v1.0.3' }
+        it_behaves_like '#archive', 'files excluding the submodule', 'README.md', 'go.mod', 'a.go', 'pkg/b.go'
+      end
+
+      context 'with the submodule\'s path' do
+        let_it_be(:mod) { create :go_module, project: project, path: 'mod' }
+        let_it_be(:version) { create :go_module_version, :tagged, mod: mod, name: 'v1.0.3' }
+        it_behaves_like '#archive', 'the submodule\'s files', 'go.mod', 'a.go'
+      end
+    end
+  end
+end

ee/spec/models/packages/sem_ver_spec.rb

+# frozen_string_literal: true
+
+require 'spec_helper'
+
+describe Packages::SemVer, type: :model do
+  shared_examples '#parse with a valid semver' do |str, major, minor, patch, prerelease, build|
+    context "with #{str}" do
+      it "returns #{described_class.new(major, minor, patch, prerelease, build, prefixed: true)} with prefix" do
+        expected = described_class.new(major, minor, patch, prerelease, build, prefixed: true)
+        expect(described_class.parse('v' + str, prefixed: true)).to eq(expected)
+      end
+
+      it "returns #{described_class.new(major, minor, patch, prerelease, build)} without prefix" do
+        expected = described_class.new(major, minor, patch, prerelease, build)
+        expect(described_class.parse(str)).to eq(expected)
+      end
+    end
+  end
+
+  shared_examples '#parse with an invalid semver' do |str|
+    context "with #{str}" do
+      it 'returns nil with prefix' do
+        expect(described_class.parse('v' + str, prefixed: true)).to be_nil
+      end
+
+      it 'returns nil without prefix' do
+        expect(described_class.parse(str)).to be_nil
+      end
+    end
+  end
+
+  describe '#parse' do
+    it_behaves_like '#parse with a valid semver', '1.0.0', 1, 0, 0, nil, nil
+    it_behaves_like '#parse with a valid semver', '1.0.0-pre', 1, 0, 0, 'pre', nil
+    it_behaves_like '#parse with a valid semver', '1.0.0+build', 1, 0, 0, nil, 'build'
+    it_behaves_like '#parse with a valid semver', '1.0.0-pre+build', 1, 0, 0, 'pre', 'build'
+    it_behaves_like '#parse with an invalid semver', '01.0.0'
+    it_behaves_like '#parse with an invalid semver', '0.01.0'
+    it_behaves_like '#parse with an invalid semver', '0.0.01'
+    it_behaves_like '#parse with an invalid semver', '1.0.0asdf'
+  end
+end

ee/spec/requests/api/go_proxy_spec.rb

+# frozen_string_literal: true
+
+require 'spec_helper'
+
+describe API::GoProxy do
+  include EE::PackagesManagerApiSpecHelpers
+
+  let_it_be(:user) { create :user }
+  let_it_be(:project) { create :project_empty_repo, creator: user, path: 'my-go-lib' }
+  let_it_be(:base) { "#{Settings.build_gitlab_go_url}/#{project.full_path}" }
+
+  let_it_be(:oauth) { create :oauth_access_token, scopes: 'api', resource_owner: user }
+  let_it_be(:job) { create :ci_build, user: user }
+  let_it_be(:pa_token) { create :personal_access_token, user: user }
+
+  let_it_be(:modules) do
+    commits = [
+      create(:go_module_commit, :files,   project: project, tag: 'v1.0.0', files: { 'README.md' => 'Hi' }       ),
+      create(:go_module_commit, :module,  project: project, tag: 'v1.0.1'                                       ),
+      create(:go_module_commit, :package, project: project, tag: 'v1.0.2', path: 'pkg'                          ),
+      create(:go_module_commit, :module,  project: project, tag: 'v1.0.3', name: 'mod'                          ),
+      create(:go_module_commit, :files,   project: project,                files: { 'y.go' => "package a\n" }   ),
+      create(:go_module_commit, :module,  project: project,                name: 'v2'                           ),
+      create(:go_module_commit, :files,   project: project, tag: 'v2.0.0', files: { 'v2/x.go' => "package a\n" })
+    ]
+
+    { sha: [commits[4].sha, commits[5].sha] }
+  end
+
+  before do
+    project.add_developer(user)
+
+    stub_licensed_features(packages: true)
+    stub_feature_flags(go_proxy_disable_gomod_validation: false)
+
+    modules
+  end
+
+  shared_examples 'an unavailable resource' do
+    it 'returns not found' do
+      get_resource(user)
+
+      expect(response).to have_gitlab_http_status(:not_found)
+    end
+  end
+
+  shared_examples 'a module version list resource' do |*versions, path: ''|
+    let(:module_name) { "#{base}#{path}" }
+    let(:resource) { "list" }
+
+    it "returns #{versions.empty? ? 'nothing' : versions.join(', ')}" do
+      get_resource(user)
+
+      expect(response).to have_gitlab_http_status(:ok)
+      expect(response.body.split("\n").to_set).to eq(versions.to_set)
+    end
+  end
+
+  shared_examples 'a missing module version list resource' do |path: ''|
+    let(:module_name) { "#{base}#{path}" }
+    let(:resource) { "list" }
+
+    it_behaves_like 'an unavailable resource'
+  end
+
+  shared_examples 'a module version information resource' do |version, path: ''|
+    let(:module_name) { "#{base}#{path}" }
+    let(:resource) { "#{version}.info" }
+
+    it "returns information for #{version}" do
+      get_resource(user)
+
+      time = project.repository.find_tag(version).dereferenced_target.committed_date
+
+      expect(response).to have_gitlab_http_status(:ok)
+      expect(json_response).to be_kind_of(Hash)
+      expect(json_response['Version']).to eq(version)
+      expect(json_response['Time']).to eq(time.strftime('%Y-%m-%dT%H:%M:%S.%L%:z'))
+    end
+  end
+
+  shared_examples 'a missing module version information resource' do |version, path: ''|
+    let(:module_name) { "#{base}#{path}" }
+    let(:resource) { "#{version}.info" }
+
+    it_behaves_like 'an unavailable resource'
+  end
+
+  shared_examples 'a module pseudo-version information resource' do |prefix, path: ''|
+    let(:module_name) { "#{base}#{path}" }
+    let(:commit) { project.repository.commit_by(oid: sha) }
+    let(:version) { fmt_pseudo_version prefix, commit }
+    let(:resource) { "#{version}.info" }
+
+    it "returns information for #{prefix}yyyymmddhhmmss-abcdefabcdef" do
+      get_resource(user)
+
+      expect(response).to have_gitlab_http_status(:ok)
+      expect(json_response).to be_kind_of(Hash)
+      expect(json_response['Version']).to eq(version)
+      expect(json_response['Time']).to eq(commit.committed_date.strftime('%Y-%m-%dT%H:%M:%S.%L%:z'))
+    end
+  end
+
+  shared_examples 'a missing module pseudo-version information resource' do |path: ''|
+    let(:module_name) { "#{base}#{path}" }
+    let(:commit) do
+      raise "tried to reference :commit without defining :sha" unless defined?(sha)
+
+      project.repository.commit_by(oid: sha)
+    end
+    let(:resource) { "#{version}.info" }
+
+    it_behaves_like 'an unavailable resource'
+  end
+
+  shared_examples 'a module file resource' do |version, path: ''|
+    let(:module_name) { "#{base}#{path}" }
+    let(:resource) { "#{version}.mod" }
+
+    it "returns #{path}/go.mod from the repo" do
+      get_resource(user)
+
+      expect(response).to have_gitlab_http_status(:ok)
+      expect(response.body.split("\n", 2).first).to eq("module #{module_name}")
+    end
+  end
+
+  shared_examples 'a missing module file resource' do |version, path: ''|
+    let(:module_name) { "#{base}#{path}" }
+    let(:resource) { "#{version}.mod" }
+
+    it_behaves_like 'an unavailable resource'
+  end
+
+  shared_examples 'a module archive resource' do |version, entries, path: ''|
+    let(:module_name) { "#{base}#{path}" }
+    let(:resource) { "#{version}.zip" }
+
+    it "returns an archive of #{path.empty? ? '/' : path} @ #{version} from the repo" do
+      get_resource(user)
+
+      expect(response).to have_gitlab_http_status(:ok)
+
+      entries = entries.map { |e| "#{module_name}@#{version}/#{e}" }.to_set
+      actual = Set[]
+      Zip::InputStream.open(StringIO.new(response.body)) do |zip|
+        while (entry = zip.get_next_entry)
+          actual.add(entry.name)
+        end
+      end
+
+      expect(actual).to eq(entries)
+    end
+  end
+
+  describe 'GET /projects/:id/packages/go/*module_name/@v/list' do
+    context 'for the root module' do
+      it_behaves_like 'a module version list resource', 'v1.0.1', 'v1.0.2', 'v1.0.3'
+    end
+
+    context 'for the package' do
+      it_behaves_like 'a module version list resource', path: '/pkg'
+    end
+
+    context 'for the submodule' do
+      it_behaves_like 'a module version list resource', 'v1.0.3', path: '/mod'
+    end
+
+    context 'for the root module v2' do
+      it_behaves_like 'a module version list resource', 'v2.0.0', path: '/v2'
+    end
+
+    context 'with a URL encoded relative path component' do
+      it_behaves_like 'a missing module version list resource', path: '/%2E%2E%2Fxyz'
+    end
+
+    context 'with the feature disabled' do
+      before do
+        stub_feature_flags(go_proxy: false)
+      end
+
+      it_behaves_like 'a missing module version list resource'
+    end
+  end
+
+  describe 'GET /projects/:id/packages/go/*module_name/@v/:module_version.info' do
+    context 'with the root module v1.0.1' do
+      it_behaves_like 'a module version information resource', 'v1.0.1'
+    end
+
+    context 'with the submodule v1.0.3' do
+      it_behaves_like 'a module version information resource', 'v1.0.3', path: '/mod'
+    end
+
+    context 'with the root module v2.0.0' do
+      it_behaves_like 'a module version information resource', 'v2.0.0', path: '/v2'
+    end
+
+    context 'with an invalid path' do
+      it_behaves_like 'a missing module version information resource', 'v1.0.3', path: '/pkg'
+    end
+
+    context 'with an invalid version' do
+      it_behaves_like 'a missing module version information resource', 'v1.0.1', path: '/mod'
+    end
+
+    context 'with a pseudo-version for v1' do
+      it_behaves_like 'a module pseudo-version information resource', 'v1.0.4-0.' do
+        let(:sha) { modules[:sha][0] }
+      end
+    end
+
+    context 'with a pseudo-version for v2' do
+      it_behaves_like 'a module pseudo-version information resource', 'v2.0.0-', path: '/v2' do
+        let(:sha) { modules[:sha][1] }
+      end
+    end
+
+    context 'with a pseudo-version with an invalid timestamp' do
+      it_behaves_like 'a missing module pseudo-version information resource' do
+        let(:version) { "v1.0.4-0.00000000000000-#{modules[:sha][0][0..11]}" }
+      end
+    end
+
+    context 'with a pseudo-version with an invalid commit sha' do
+      it_behaves_like 'a missing module pseudo-version information resource' do
+        let(:sha) { modules[:sha][0] }
+        let(:version) { "v1.0.4-0.#{commit.committed_date.strftime('%Y%m%d%H%M%S')}-000000000000" }
+      end
+    end
+
+    context 'with a pseudo-version with a short commit sha' do
+      it_behaves_like 'a missing module pseudo-version information resource' do
+        let(:sha) { modules[:sha][0] }
+        let(:version) { "v1.0.4-0.#{commit.committed_date.strftime('%Y%m%d%H%M%S')}-#{modules[:sha][0][0..10]}" }
+      end
+    end
+  end
+
+  describe 'GET /projects/:id/packages/go/*module_name/@v/:module_version.mod' do
+    context 'with the root module v1.0.1' do
+      it_behaves_like 'a module file resource', 'v1.0.1'
+    end
+
+    context 'with the submodule v1.0.3' do
+      it_behaves_like 'a module file resource', 'v1.0.3', path: '/mod'
+    end
+
+    context 'with the root module v2.0.0' do
+      it_behaves_like 'a module file resource', 'v2.0.0', path: '/v2'
+    end
+
+    context 'with an invalid path' do
+      it_behaves_like 'a missing module file resource', 'v1.0.3', path: '/pkg'
+    end
+
+    context 'with an invalid version' do
+      it_behaves_like 'a missing module file resource', 'v1.0.1', path: '/mod'
+    end
+  end
+
+  describe 'GET /projects/:id/packages/go/*module_name/@v/:module_version.zip' do
+    context 'with the root module v1.0.1' do
+      it_behaves_like 'a module archive resource', 'v1.0.1', ['README.md', 'go.mod', 'a.go']
+    end
+
+    context 'with the root module v1.0.2' do
+      it_behaves_like 'a module archive resource', 'v1.0.2', ['README.md', 'go.mod', 'a.go', 'pkg/b.go']
+    end
+
+    context 'with the root module v1.0.3' do
+      it_behaves_like 'a module archive resource', 'v1.0.3', ['README.md', 'go.mod', 'a.go', 'pkg/b.go']
+    end
+
+    context 'with the submodule v1.0.3' do
+      it_behaves_like 'a module archive resource', 'v1.0.3', ['go.mod', 'a.go'], path: '/mod'
+    end
+
+    context 'with the root module v2.0.0' do
+      it_behaves_like 'a module archive resource', 'v2.0.0', ['go.mod', 'a.go', 'x.go'], path: '/v2'
+    end
+  end
+
+  context 'with an invalid module directive' do
+    let_it_be(:project) { create :project_empty_repo, :public, creator: user }
+    let_it_be(:base) { "#{Settings.build_gitlab_go_url}/#{project.full_path}" }
+
+    let_it_be(:modules) do
+      create(:go_module_commit, :files, project: project,                files: { 'a.go' => "package\a" }                   )
+      create(:go_module_commit, :files, project: project, tag: 'v1.0.0', files: { 'go.mod' => "module not/a/real/module\n" })
+      create(:go_module_commit, :files, project: project,                files: { 'v2/a.go' => "package a\n" }              )
+      create(:go_module_commit, :files, project: project, tag: 'v2.0.0', files: { 'v2/go.mod' => "module #{base}\n" }       )
+    end
+
+    describe 'GET /projects/:id/packages/go/*module_name/@v/list' do
+      context 'with a completely wrong directive for v1' do
+        it_behaves_like 'a module version list resource'
+      end
+
+      context 'with a directive omitting the suffix for v2' do
+        it_behaves_like 'a module version list resource', path: '/v2'
+      end
+    end
+
+    describe 'GET /projects/:id/packages/go/*module_name/@v/:module_version.info' do
+      context 'with a completely wrong directive for v1' do
+        it_behaves_like 'a missing module version information resource', 'v1.0.0'
+      end
+
+      context 'with a directive omitting the suffix for v2' do
+        it_behaves_like 'a missing module version information resource', 'v2.0.0', path: '/v2'
+      end
+    end
+  end
+
+  context 'with a case sensitive project and versions' do
+    let_it_be(:project) { create :project_empty_repo, :public, creator: user, path: 'MyGoLib' }
+    let_it_be(:base) { "#{Settings.build_gitlab_go_url}/#{project.full_path}" }
+    let_it_be(:base_encoded) { base.gsub(/[A-Z]/) { |s| "!#{s.downcase}"} }
+
+    let_it_be(:modules) do
+      create(:go_module_commit, :files,   project: project, files: { 'README.md' => "Hi" })
+      create(:go_module_commit, :module,  project: project, tag: 'v1.0.1-prerelease')
+      create(:go_module_commit, :package, project: project, tag: 'v1.0.1-Prerelease', path: 'pkg')
+    end
+
+    describe 'GET /projects/:id/packages/go/*module_name/@v/list' do
+      let(:resource) { "list" }
+
+      context 'with a case encoded path' do
+        it_behaves_like 'a module version list resource', 'v1.0.1-prerelease', 'v1.0.1-Prerelease' do
+          let(:module_name) { base_encoded }
+        end
+      end
+
+      context 'without a case encoded path' do
+        it_behaves_like 'a missing module version list resource' do
+          let(:module_name) { base.downcase }
+        end
+      end
+    end
+
+    describe 'GET /projects/:id/packages/go/*module_name/@v/:module_version.info' do
+      context 'with a case encoded path' do
+        it_behaves_like 'a module version information resource', 'v1.0.1-Prerelease' do
+          let(:module_name) { base_encoded }
+          let(:resource) { "v1.0.1-!prerelease.info" }
+        end
+      end
+
+      context 'without a case encoded path' do
+        it_behaves_like 'a module version information resource', 'v1.0.1-prerelease' do
+          let(:module_name) { base_encoded }
+          let(:resource) { "v1.0.1-prerelease.info" }
+        end
+      end
+    end
+  end
+
+  context 'with a private project' do
+    let(:module_name) { base }
+
+    before do
+      project.update(visibility_level: Gitlab::VisibilityLevel::PRIVATE)
+    end
+
+    describe 'GET /projects/:id/packages/go/*module_name/@v/list' do
+      let(:resource) { "list" }
+
+      it 'returns ok with an oauth token' do
+        get_resource(oauth_access_token: oauth)
+
+        expect(response).to have_gitlab_http_status(:ok)
+      end
+
+      it 'returns ok with a job token' do
+        get_resource(oauth_access_token: job)
+
+        expect(response).to have_gitlab_http_status(:ok)
+      end
+
+      it 'returns ok with a personal access token' do
+        get_resource(personal_access_token: pa_token)
+
+        expect(response).to have_gitlab_http_status(:ok)
+      end
+
+      it 'returns ok with a personal access token and basic authentication' do
+        get_resource(headers: build_basic_auth_header(user.username, pa_token.token))
+
+        expect(response).to have_gitlab_http_status(:ok)
+      end
+
+      it 'returns unauthorized with no authentication' do
+        get_resource
+
+        expect(response).to have_gitlab_http_status(:unauthorized)
+      end
+    end
+  end
+
+  context 'with a public project' do
+    let(:module_name) { base }
+
+    before do
+      project.update(visibility_level: Gitlab::VisibilityLevel::PUBLIC)
+    end
+
+    describe 'GET /projects/:id/packages/go/*module_name/@v/list' do
+      let(:resource) { "list" }
+
+      it 'returns ok with no authentication' do
+        get_resource
+
+        expect(response).to have_gitlab_http_status(:ok)
+      end
+    end
+  end
+
+  context 'with a non-existent project' do
+    def get_resource(user = nil, **params)
+      get api("/projects/not%2fa%2fproject/packages/go/#{base}/@v/list", user, params)
+    end
+
+    describe 'GET /projects/:id/packages/go/*module_name/@v/list' do
+      it 'returns not found with a user' do
+        get_resource(user)
+
+        expect(response).to have_gitlab_http_status(:not_found)
+      end
+
+      it 'returns not found with an oauth token' do
+        get_resource(oauth_access_token: oauth)
+
+        expect(response).to have_gitlab_http_status(:not_found)
+      end
+
+      it 'returns not found with a job token' do
+        get_resource(oauth_access_token: job)
+
+        expect(response).to have_gitlab_http_status(:not_found)
+      end
+
+      it 'returns not found with a personal access token' do
+        get_resource(personal_access_token: pa_token)
+
+        expect(response).to have_gitlab_http_status(:not_found)
+      end
+
+      it 'returns unauthorized with no authentication' do
+        get_resource
+
+        expect(response).to have_gitlab_http_status(:unauthorized)
+      end
+    end
+  end
+
+  def get_resource(user = nil, headers: {}, **params)
+    get api("/projects/#{project.id}/packages/go/#{module_name}/@v/#{resource}", user, params), headers: headers
+  end
+
+  def fmt_pseudo_version(prefix, commit)
+    "#{prefix}#{commit.committed_date.strftime('%Y%m%d%H%M%S')}-#{commit.sha[0..11]}"
+  end
+end