Merge branch '14108-instance-level-ci-variables-api' into 'master'

Add API CRUD actions for managing instance-level CI variables

Closes #14108

See merge request gitlab-org/gitlab!31342

changelogs/unreleased/14108-instance-level-ci-variables-api.yml

+---
+title: Add API CRUD actions for instance-level CI/CD variables
+merge_request: 31342
+author:
+type: added

doc/api/admin_ci_instance_level_variables.md

+# Instance-level CI/CD variables API
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14108) in GitLab 13.0
+
+## List all instance variables
+
+Get the list of all instance-level variables.
+
+```plaintext
+GET /admin/ci/variables
+```
+
+```shell
+curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/admin/ci/variables"
+```
+
+```json
+[
+    {
+        "key": "TEST_VARIABLE_1",
+        "variable_type": "env_var",
+        "value": "TEST_1",
+        "protected": false,
+        "masked": false
+    },
+    {
+        "key": "TEST_VARIABLE_2",
+        "variable_type": "env_var",
+        "value": "TEST_2",
+        "protected": false,
+        "masked": false
+    }
+]
+```
+
+## Show instance variable details
+
+Get the details of a specific instance-level variable.
+
+```plaintext
+GET /admin/ci/variables/:key
+```
+
+| Attribute | Type    | required | Description           |
+|-----------|---------|----------|-----------------------|
+| `key`     | string  | yes      | The `key` of a variable |
+
+```shell
+curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/admin/ci/variables/TEST_VARIABLE_1"
+```
+
+```json
+{
+    "key": "TEST_VARIABLE_1",
+    "variable_type": "env_var",
+    "value": "TEST_1",
+    "protected": false,
+    "masked": false
+}
+```
+
+## Create instance variable
+
+Create a new instance-level variable.
+
+```plaintext
+POST /admin/ci/variables
+```
+
+| Attribute       | Type    | required | Description           |
+|-----------------|---------|----------|-----------------------|
+| `key`           | string  | yes      | The `key` of a variable. Max 255 characters, only `A-Z`, `a-z`, `0-9`, and `_` are allowed. |
+| `value`         | string  | yes      | The `value` of a variable. |
+| `variable_type` | string  | no       | The type of a variable. Available types are: `env_var` (default) and `file`. |
+| `protected`     | boolean | no       | Whether the variable is protected. |
+| `masked`        | boolean | no       | Whether the variable is masked. |
+
+```shell
+curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/admin/ci/variables" --form "key=NEW_VARIABLE" --form "value=new value"
+```
+
+```json
+{
+    "key": "NEW_VARIABLE",
+    "value": "new value",
+    "variable_type": "env_var",
+    "protected": false,
+    "masked": false
+}
+```
+
+## Update instance variable
+
+Update an instance-level variable.
+
+```plaintext
+PUT /admin/ci/variables/:key
+```
+
+| Attribute       | Type    | required | Description             |
+|-----------------|---------|----------|-------------------------|
+| `key`           | string  | yes      | The `key` of a variable.   |
+| `value`         | string  | yes      | The `value` of a variable. |
+| `variable_type` | string  | no       | The type of a variable. Available types are: `env_var` (default) and `file`. |
+| `protected`     | boolean | no       | Whether the variable is protected. |
+| `masked`        | boolean | no       | Whether the variable is masked. |
+
+```shell
+curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/admin/ci/variables/NEW_VARIABLE" --form "value=updated value"
+```
+
+```json
+{
+    "key": "NEW_VARIABLE",
+    "value": "updated value",
+    "variable_type": "env_var",
+    "protected": true,
+    "masked": true
+}
+```
+
+## Remove instance variable
+
+Remove an instance-level variable.
+
+```plaintext
+DELETE /admin/ci/variables/:key
+```
+
+| Attribute | Type    | required | Description             |
+|-----------|---------|----------|-------------------------|
+| `key`     | string  | yes      | The `key` of a variable |
+
+```shell
+curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/admin/ci/variables/VARIABLE_1"
+```

doc/api/api_resources.md

@@ -113,7 +113,8 @@ The following API resources are available outside of project and group contexts
 
 | Resource                                           | Available endpoints                                                     |
 |:---------------------------------------------------|:------------------------------------------------------------------------|
-| [Admin Sidekiq queues](admin_sidekiq_queues.md)    | `/admin/sidekiq/queues/:queue_name`                                               |
+| [Instance-level CI/CD variables](admin_ci_instance_level_variables.md) | `/admin/ci/variables`                               |
+| [Admin Sidekiq queues](admin_sidekiq_queues.md)    | `/admin/sidekiq/queues/:queue_name`                                     |
 | [Appearance](appearance.md) **(CORE ONLY)**        | `/application/appearance`                                               |
 | [Applications](applications.md)                    | `/applications`                                                         |
 | [Audit Events](audit_events.md) **(PREMIUM ONLY)** | `/audit_events`                                                         |

lib/api/admin/ci/variables.rb

+# frozen_string_literal: true
+
+module API
+  module Admin
+    module Ci
+      class Variables < Grape::API
+        include PaginationParams
+
+        before { authenticated_as_admin! }
+
+        namespace 'admin' do
+          namespace 'ci' do
+            namespace 'variables' do
+              desc 'Get instance-level variables' do
+                success Entities::Variable
+              end
+              params do
+                use :pagination
+              end
+              get '/' do
+                variables = ::Ci::InstanceVariable.all
+
+                present paginate(variables), with: Entities::Variable
+              end
+
+              desc 'Get a specific variable from a group' do
+                success Entities::Variable
+              end
+              params do
+                requires :key, type: String, desc: 'The key of the variable'
+              end
+              get ':key' do
+                key = params[:key]
+                variable = ::Ci::InstanceVariable.find_by_key(key)
+
+                break not_found!('InstanceVariable') unless variable
+
+                present variable, with: Entities::Variable
+              end
+
+              desc 'Create a new instance-level variable' do
+                success Entities::Variable
+              end
+              params do
+                requires :key,
+                  type: String,
+                  desc: 'The key of the variable'
+
+                requires :value,
+                  type: String,
+                  desc: 'The value of the variable'
+
+                optional :protected,
+                  type: String,
+                  desc: 'Whether the variable is protected'
+
+                optional :masked,
+                  type: String,
+                  desc: 'Whether the variable is masked'
+
+                optional :variable_type,
+                  type: String,
+                  values: ::Ci::InstanceVariable.variable_types.keys,
+                  desc: 'The type of variable, must be one of env_var or file. Defaults to env_var'
+              end
+              post '/' do
+                variable_params = declared_params(include_missing: false)
+
+                variable = ::Ci::InstanceVariable.new(variable_params)
+
+                if variable.save
+                  present variable, with: Entities::Variable
+                else
+                  render_validation_error!(variable)
+                end
+              end
+
+              desc 'Update an existing instance-variable' do
+                success Entities::Variable
+              end
+              params do
+                optional :key,
+                  type: String,
+                  desc: 'The key of the variable'
+
+                optional :value,
+                  type: String,
+                  desc: 'The value of the variable'
+
+                optional :protected,
+                  type: String,
+                  desc: 'Whether the variable is protected'
+
+                optional :masked,
+                  type: String,
+                  desc: 'Whether the variable is masked'
+
+                optional :variable_type,
+                  type: String,
+                  values: ::Ci::InstanceVariable.variable_types.keys,
+                  desc: 'The type of variable, must be one of env_var or file'
+              end
+              put ':key' do
+                variable = ::Ci::InstanceVariable.find_by_key(params[:key])
+
+                break not_found!('InstanceVariable') unless variable
+
+                variable_params = declared_params(include_missing: false).except(:key)
+
+                if variable.update(variable_params)
+                  present variable, with: Entities::Variable
+                else
+                  render_validation_error!(variable)
+                end
+              end
+
+              desc 'Delete an existing instance-level variable' do
+                success Entities::Variable
+              end
+              params do
+                requires :key, type: String, desc: 'The key of the variable'
+              end
+              delete ':key' do
+                variable = ::Ci::InstanceVariable.find_by_key(params[:key])
+                not_found!('InstanceVariable') unless variable
+
+                variable.destroy
+
+                no_content!
+              end
+            end
+          end
+        end
+      end
+    end
+  end
+end

lib/api/api.rb

@@ -120,6 +120,7 @@ module API
 
       # Keep in alphabetical order
       mount ::API::AccessRequests
+      mount ::API::Admin::Ci::Variables
       mount ::API::Admin::Sidekiq
       mount ::API::Appearance
       mount ::API::Applications

spec/requests/api/admin/ci/variables_spec.rb

+# frozen_string_literal: true
+
+require 'spec_helper'
+
+describe ::API::Admin::Ci::Variables do
+  let_it_be(:admin) { create(:admin) }
+  let_it_be(:user) { create(:user) }
+
+  describe 'GET /admin/ci/variables' do
+    let!(:variable) { create(:ci_instance_variable) }
+
+    it 'returns instance-level variables for admins', :aggregate_failures do
+      get api('/admin/ci/variables', admin)
+
+      expect(response).to have_gitlab_http_status(:ok)
+      expect(json_response).to be_a(Array)
+    end
+
+    it 'does not return instance-level variables for regular users' do
+      get api('/admin/ci/variables', user)
+
+      expect(response).to have_gitlab_http_status(:forbidden)
+    end
+
+    it 'does not return instance-level variables for unauthorized users' do
+      get api('/admin/ci/variables')
+
+      expect(response).to have_gitlab_http_status(:unauthorized)
+    end
+  end
+
+  describe 'GET /admin/ci/variables/:key' do
+    let!(:variable) { create(:ci_instance_variable) }
+
+    it 'returns instance-level variable details for admins', :aggregate_failures do
+      get api("/admin/ci/variables/#{variable.key}", admin)
+
+      expect(response).to have_gitlab_http_status(:ok)
+      expect(json_response['value']).to eq(variable.value)
+      expect(json_response['protected']).to eq(variable.protected?)
+      expect(json_response['variable_type']).to eq(variable.variable_type)
+    end
+
+    it 'responds with 404 Not Found if requesting non-existing variable' do
+      get api('/admin/ci/variables/non_existing_variable', admin)
+
+      expect(response).to have_gitlab_http_status(:not_found)
+    end
+
+    it 'does not return instance-level variable details for regular users' do
+      get api("/admin/ci/variables/#{variable.key}", user)
+
+      expect(response).to have_gitlab_http_status(:forbidden)
+    end
+
+    it 'does not return instance-level variable details for unauthorized users' do
+      get api("/admin/ci/variables/#{variable.key}")
+
+      expect(response).to have_gitlab_http_status(:unauthorized)
+    end
+  end
+
+  describe 'POST /admin/ci/variables' do
+    context 'authorized user with proper permissions' do
+      let!(:variable) { create(:ci_instance_variable) }
+
+      it 'creates variable for admins', :aggregate_failures do
+        expect do
+          post api('/admin/ci/variables', admin),
+            params: {
+              key: 'TEST_VARIABLE_2',
+              value: 'PROTECTED_VALUE_2',
+              protected: true,
+              masked: true
+            }
+        end.to change { ::Ci::InstanceVariable.count }.by(1)
+
+        expect(response).to have_gitlab_http_status(:created)
+        expect(json_response['key']).to eq('TEST_VARIABLE_2')
+        expect(json_response['value']).to eq('PROTECTED_VALUE_2')
+        expect(json_response['protected']).to be_truthy
+        expect(json_response['masked']).to be_truthy
+        expect(json_response['variable_type']).to eq('env_var')
+      end
+
+      it 'creates variable with optional attributes', :aggregate_failures do
+        expect do
+          post api('/admin/ci/variables', admin),
+            params: {
+              variable_type: 'file',
+              key: 'TEST_VARIABLE_2',
+              value: 'VALUE_2'
+            }
+        end.to change { ::Ci::InstanceVariable.count }.by(1)
+
+        expect(response).to have_gitlab_http_status(:created)
+        expect(json_response['key']).to eq('TEST_VARIABLE_2')
+        expect(json_response['value']).to eq('VALUE_2')
+        expect(json_response['protected']).to be_falsey
+        expect(json_response['masked']).to be_falsey
+        expect(json_response['variable_type']).to eq('file')
+      end
+
+      it 'does not allow to duplicate variable key' do
+        expect do
+          post api('/admin/ci/variables', admin),
+            params: { key: variable.key, value: 'VALUE_2' }
+        end.not_to change { ::Ci::InstanceVariable.count }
+
+        expect(response).to have_gitlab_http_status(:bad_request)
+      end
+    end
+
+    context 'authorized user with invalid permissions' do
+      it 'does not create variable' do
+        post api('/admin/ci/variables', user)
+
+        expect(response).to have_gitlab_http_status(:forbidden)
+      end
+    end
+
+    context 'unauthorized user' do
+      it 'does not create variable' do
+        post api('/admin/ci/variables')
+
+        expect(response).to have_gitlab_http_status(:unauthorized)
+      end
+    end
+  end
+
+  describe 'PUT /admin/ci/variables/:key' do
+    let!(:variable) { create(:ci_instance_variable) }
+
+    context 'authorized user with proper permissions' do
+      it 'updates variable data', :aggregate_failures do
+        put api("/admin/ci/variables/#{variable.key}", admin),
+          params: {
+            variable_type: 'file',
+            value: 'VALUE_1_UP',
+            protected: true,
+            masked: true
+          }
+
+        expect(response).to have_gitlab_http_status(:ok)
+        expect(variable.reload.value).to eq('VALUE_1_UP')
+        expect(variable.reload).to be_protected
+        expect(json_response['variable_type']).to eq('file')
+        expect(json_response['masked']).to be_truthy
+      end
+
+      it 'responds with 404 Not Found if requesting non-existing variable' do
+        put api('/admin/ci/variables/non_existing_variable', admin)
+
+        expect(response).to have_gitlab_http_status(:not_found)
+      end
+    end
+
+    context 'authorized user with invalid permissions' do
+      it 'does not update variable' do
+        put api("/admin/ci/variables/#{variable.key}", user)
+
+        expect(response).to have_gitlab_http_status(:forbidden)
+      end
+    end
+
+    context 'unauthorized user' do
+      it 'does not update variable' do
+        put api("/admin/ci/variables/#{variable.key}")
+
+        expect(response).to have_gitlab_http_status(:unauthorized)
+      end
+    end
+  end
+
+  describe 'DELETE /admin/ci/variables/:key' do
+    let!(:variable) { create(:ci_instance_variable) }
+
+    context 'authorized user with proper permissions' do
+      it 'deletes variable' do
+        expect do
+          delete api("/admin/ci/variables/#{variable.key}", admin)
+
+          expect(response).to have_gitlab_http_status(:no_content)
+        end.to change { ::Ci::InstanceVariable.count }.by(-1)
+      end
+
+      it 'responds with 404 Not Found if requesting non-existing variable' do
+        delete api('/admin/ci/variables/non_existing_variable', admin)
+
+        expect(response).to have_gitlab_http_status(:not_found)
+      end
+    end
+
+    context 'authorized user with invalid permissions' do
+      it 'does not delete variable' do
+        delete api("/admin/ci/variables/#{variable.key}", user)
+
+        expect(response).to have_gitlab_http_status(:forbidden)
+      end
+    end
+
+    context 'unauthorized user' do
+      it 'does not delete variable' do
+        delete api("/admin/ci/variables/#{variable.key}")
+
+        expect(response).to have_gitlab_http_status(:unauthorized)
+      end
+    end
+  end
+end