Commit 8a297df8 authored by Thong Kuah's avatar Thong Kuah

Merge branch 'tw/kas-grpc' into 'master'

GraphQL endpoint for exposing agent configuration details for a project

See merge request gitlab-org/gitlab!62646
parents 9e9b3904 d91cbf73
...@@ -482,6 +482,9 @@ gem 'spamcheck', '~> 0.1.0' ...@@ -482,6 +482,9 @@ gem 'spamcheck', '~> 0.1.0'
# Gitaly GRPC protocol definitions # Gitaly GRPC protocol definitions
gem 'gitaly', '~> 13.12.0.pre.rc1' gem 'gitaly', '~> 13.12.0.pre.rc1'
# KAS GRPC protocol definitions
gem 'kas-grpc', '~> 0.0.2'
gem 'grpc', '~> 1.30.2' gem 'grpc', '~> 1.30.2'
gem 'google-protobuf', '~> 3.17.1' gem 'google-protobuf', '~> 3.17.1'
......
...@@ -682,6 +682,8 @@ GEM ...@@ -682,6 +682,8 @@ GEM
activerecord activerecord
kaminari-core (= 1.2.1) kaminari-core (= 1.2.1)
kaminari-core (1.2.1) kaminari-core (1.2.1)
kas-grpc (0.0.2)
grpc (~> 1.0)
knapsack (1.21.1) knapsack (1.21.1)
rake rake
kramdown (2.3.1) kramdown (2.3.1)
...@@ -1520,6 +1522,7 @@ DEPENDENCIES ...@@ -1520,6 +1522,7 @@ DEPENDENCIES
json_schemer (~> 0.2.12) json_schemer (~> 0.2.12)
jwt (~> 2.1.0) jwt (~> 2.1.0)
kaminari (~> 1.0) kaminari (~> 1.0)
kas-grpc (~> 0.0.2)
knapsack (~> 1.21.1) knapsack (~> 1.21.1)
kramdown (~> 2.3.1) kramdown (~> 2.3.1)
kubeclient (~> 4.9.1) kubeclient (~> 4.9.1)
......
...@@ -4247,6 +4247,29 @@ Some of the types in the schema exist solely to model connections. Each connecti ...@@ -4247,6 +4247,29 @@ Some of the types in the schema exist solely to model connections. Each connecti
has a distinct, named type, with a distinct named edge type. These are listed separately has a distinct, named type, with a distinct named edge type. These are listed separately
below. below.
#### `AgentConfigurationConnection`
The connection type for [`AgentConfiguration`](#agentconfiguration).
##### Fields
| Name | Type | Description |
| ---- | ---- | ----------- |
| <a id="agentconfigurationconnectionedges"></a>`edges` | [`[AgentConfigurationEdge]`](#agentconfigurationedge) | A list of edges. |
| <a id="agentconfigurationconnectionnodes"></a>`nodes` | [`[AgentConfiguration]`](#agentconfiguration) | A list of nodes. |
| <a id="agentconfigurationconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. |
#### `AgentConfigurationEdge`
The edge type for [`AgentConfiguration`](#agentconfiguration).
##### Fields
| Name | Type | Description |
| ---- | ---- | ----------- |
| <a id="agentconfigurationedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. |
| <a id="agentconfigurationedgenode"></a>`node` | [`AgentConfiguration`](#agentconfiguration) | The item at the end of the edge. |
#### `AlertManagementAlertConnection` #### `AlertManagementAlertConnection`
The connection type for [`AlertManagementAlert`](#alertmanagementalert). The connection type for [`AlertManagementAlert`](#alertmanagementalert).
...@@ -6886,6 +6909,16 @@ Represents the access level of a relationship between a User and object that it ...@@ -6886,6 +6909,16 @@ Represents the access level of a relationship between a User and object that it
| <a id="accesslevelintegervalue"></a>`integerValue` | [`Int`](#int) | Integer representation of access level. | | <a id="accesslevelintegervalue"></a>`integerValue` | [`Int`](#int) | Integer representation of access level. |
| <a id="accesslevelstringvalue"></a>`stringValue` | [`AccessLevelEnum`](#accesslevelenum) | String representation of access level. | | <a id="accesslevelstringvalue"></a>`stringValue` | [`AccessLevelEnum`](#accesslevelenum) | String representation of access level. |
### `AgentConfiguration`
Configuration details for an Agent.
#### Fields
| Name | Type | Description |
| ---- | ---- | ----------- |
| <a id="agentconfigurationagentname"></a>`agentName` | [`String`](#string) | Name of the agent. |
### `AlertManagementAlert` ### `AlertManagementAlert`
Describes an alert from the project's Alert Management. Describes an alert from the project's Alert Management.
...@@ -11036,6 +11069,7 @@ Represents vulnerability finding of a security report on the pipeline. ...@@ -11036,6 +11069,7 @@ Represents vulnerability finding of a security report on the pipeline.
| Name | Type | Description | | Name | Type | Description |
| ---- | ---- | ----------- | | ---- | ---- | ----------- |
| <a id="projectactualrepositorysizelimit"></a>`actualRepositorySizeLimit` | [`Float`](#float) | Size limit for the repository in bytes. | | <a id="projectactualrepositorysizelimit"></a>`actualRepositorySizeLimit` | [`Float`](#float) | Size limit for the repository in bytes. |
| <a id="projectagentconfigurations"></a>`agentConfigurations` | [`AgentConfigurationConnection`](#agentconfigurationconnection) | Agent configurations defined by the project. (see [Connections](#connections)) |
| <a id="projectallowmergeonskippedpipeline"></a>`allowMergeOnSkippedPipeline` | [`Boolean`](#boolean) | If `only_allow_merge_if_pipeline_succeeds` is true, indicates if merge requests of the project can also be merged with skipped jobs. | | <a id="projectallowmergeonskippedpipeline"></a>`allowMergeOnSkippedPipeline` | [`Boolean`](#boolean) | If `only_allow_merge_if_pipeline_succeeds` is true, indicates if merge requests of the project can also be merged with skipped jobs. |
| <a id="projectapifuzzingciconfiguration"></a>`apiFuzzingCiConfiguration` | [`ApiFuzzingCiConfiguration`](#apifuzzingciconfiguration) | API fuzzing configuration for the project. | | <a id="projectapifuzzingciconfiguration"></a>`apiFuzzingCiConfiguration` | [`ApiFuzzingCiConfiguration`](#apifuzzingciconfiguration) | API fuzzing configuration for the project. |
| <a id="projectarchived"></a>`archived` | [`Boolean`](#boolean) | Indicates the archived status of the project. | | <a id="projectarchived"></a>`archived` | [`Boolean`](#boolean) | Indicates the archived status of the project. |
......
...@@ -89,6 +89,12 @@ module EE ...@@ -89,6 +89,12 @@ module EE
resolver: ::Resolvers::DastSiteValidationResolver, resolver: ::Resolvers::DastSiteValidationResolver,
description: 'DAST Site Validations associated with the project.' description: 'DAST Site Validations associated with the project.'
field :agent_configurations,
::Types::Kas::AgentConfigurationType.connection_type,
null: true,
description: 'Agent configurations defined by the project',
resolver: ::Resolvers::Kas::AgentConfigurationsResolver
field :cluster_agent, field :cluster_agent,
::Types::Clusters::AgentType, ::Types::Clusters::AgentType,
null: true, null: true,
......
# frozen_string_literal: true
module Resolvers
module Kas
class AgentConfigurationsResolver < BaseResolver
type Types::Kas::AgentConfigurationType, null: true
# Calls Gitaly via KAS
calls_gitaly!
alias_method :project, :object
def resolve
return [] unless can_read_agent_configuration?
kas_client.list_agent_config_files(project: project)
rescue GRPC::BadStatus => e
raise Gitlab::Graphql::Errors::ResourceNotAvailable, e.class.name
end
private
def can_read_agent_configuration?
project.licensed_feature_available?(:cluster_agents) && current_user.can?(:admin_cluster, project)
end
def kas_client
@kas_client ||= Gitlab::Kas::Client.new
end
end
end
end
# frozen_string_literal: true
module Types
module Kas
# rubocop: disable Graphql/AuthorizeTypes
class AgentConfigurationType < BaseObject
graphql_name 'AgentConfiguration'
description 'Configuration details for an Agent'
field :agent_name,
GraphQL::STRING_TYPE,
null: true,
description: 'Name of the agent.'
end
# rubocop: enable Graphql/AuthorizeTypes
end
end
# frozen_string_literal: true
require 'spec_helper'
RSpec.describe Resolvers::Kas::AgentConfigurationsResolver do
include GraphqlHelpers
it { expect(described_class.type).to eq(Types::Kas::AgentConfigurationType) }
it { expect(described_class.null).to be_truthy }
it { expect(described_class.field_options).to include(calls_gitaly: true) }
describe '#resolve' do
let_it_be(:project) { create(:project) }
let(:user) { create(:user, maintainer_projects: [project]) }
let(:ctx) { Hash(current_user: user) }
let(:feature_available) { true }
let(:agent1) { double }
let(:agent2) { double }
let(:kas_client) { instance_double(Gitlab::Kas::Client, list_agent_config_files: [agent1, agent2]) }
subject { resolve(described_class, obj: project, ctx: ctx) }
before do
stub_licensed_features(cluster_agents: feature_available)
allow(Gitlab::Kas::Client).to receive(:new).and_return(kas_client)
end
it 'returns agents configured for the project' do
expect(subject).to contain_exactly(agent1, agent2)
end
context 'an error is returned from the KAS client' do
before do
allow(kas_client).to receive(:list_agent_config_files).and_raise(GRPC::DeadlineExceeded)
end
it 'raises a graphql error' do
expect { subject }.to raise_error(Gitlab::Graphql::Errors::ResourceNotAvailable, 'GRPC::DeadlineExceeded')
end
end
context 'feature is not available' do
let(:feature_available) { false }
it { is_expected.to be_empty }
end
context 'user does not have permission' do
let(:user) { create(:user) }
it { is_expected.to be_empty }
end
end
end
# frozen_string_literal: true
require 'spec_helper'
RSpec.describe GitlabSchema.types['AgentConfiguration'] do
let(:fields) { %i[agent_name] }
it { expect(described_class.graphql_name).to eq('AgentConfiguration') }
it { expect(described_class.description).to eq('Configuration details for an Agent') }
it { expect(described_class).to have_graphql_fields(fields) }
end
...@@ -101,6 +101,41 @@ RSpec.describe GitlabSchema.types['Project'] do ...@@ -101,6 +101,41 @@ RSpec.describe GitlabSchema.types['Project'] do
end end
end end
describe 'agent_configurations' do
let_it_be(:query) do
%(
query {
project(fullPath: "#{project.full_path}") {
agentConfigurations {
nodes {
agentName
}
}
}
}
)
end
let(:agent_name) { 'example-agent-name' }
let(:kas_client) { instance_double(Gitlab::Kas::Client, list_agent_config_files: [double(agent_name: agent_name)]) }
subject { GitlabSchema.execute(query, context: { current_user: user }).as_json }
before do
stub_licensed_features(cluster_agents: true)
project.add_maintainer(user)
allow(Gitlab::Kas::Client).to receive(:new).and_return(kas_client)
end
it 'returns configured agents' do
agents = subject.dig('data', 'project', 'agentConfigurations', 'nodes')
expect(agents.count).to eq(1)
expect(agents.first['agentName']).to eq(agent_name)
end
end
describe 'cluster_agents' do describe 'cluster_agents' do
let_it_be(:cluster_agent) { create(:cluster_agent, project: project, name: 'agent-name') } let_it_be(:cluster_agent) { create(:cluster_agent, project: project, name: 'agent-name') }
let_it_be(:query) do let_it_be(:query) do
......
...@@ -45,6 +45,13 @@ module Gitlab ...@@ -45,6 +45,13 @@ module Gitlab
Gitlab.config.gitlab_kas.external_url Gitlab.config.gitlab_kas.external_url
end end
# Return GitLab KAS internal_url
#
# @return [String] internal_url
def internal_url
Gitlab.config.gitlab_kas.internal_url
end
# Return whether GitLab KAS is enabled # Return whether GitLab KAS is enabled
# #
# @return [Boolean] external_url # @return [Boolean] external_url
......
# frozen_string_literal: true
module Gitlab
module Kas
class Client
TIMEOUT = 2.seconds.freeze
JWT_AUDIENCE = 'gitlab-kas'
STUB_CLASSES = {
configuration_project: Gitlab::Agent::ConfigurationProject::Rpc::ConfigurationProject::Stub
}.freeze
ConfigurationError = Class.new(StandardError)
def initialize
raise ConfigurationError, 'GitLab KAS is not enabled' unless Gitlab::Kas.enabled?
raise ConfigurationError, 'KAS internal URL is not configured' unless Gitlab::Kas.internal_url.present?
end
def list_agent_config_files(project:)
request = Gitlab::Agent::ConfigurationProject::Rpc::ListAgentConfigFilesRequest.new(
repository: repository(project),
gitaly_address: gitaly_address(project)
)
stub_for(:configuration_project)
.list_agent_config_files(request, metadata: metadata)
.config_files
.to_a
end
private
def stub_for(service)
@stubs ||= {}
@stubs[service] ||= STUB_CLASSES.fetch(service).new(kas_endpoint_url, credentials, timeout: TIMEOUT)
end
def repository(project)
gitaly_repository = project.repository.gitaly_repository
Gitlab::Agent::Modserver::Repository.new(gitaly_repository.to_h)
end
def gitaly_address(project)
connection_data = Gitlab::GitalyClient.connection_data(project.repository_storage)
Gitlab::Agent::Modserver::GitalyAddress.new(connection_data)
end
def kas_endpoint_url
Gitlab::Kas.internal_url.delete_prefix('grpc://')
end
def credentials
if Rails.env.test? || Rails.env.development?
:this_channel_is_insecure
else
GRPC::Core::ChannelCredentials.new
end
end
def metadata
{ 'authorization' => "bearer #{token}" }
end
def token
JSONWebToken::HMACToken.new(Gitlab::Kas.secret).tap do |token|
token.issuer = Settings.gitlab.host
token.audience = JWT_AUDIENCE
end.encoded
end
end
end
end
# frozen_string_literal: true
require 'spec_helper'
RSpec.describe Gitlab::Kas::Client do
let_it_be(:project) { create(:project) }
describe '#initialize' do
context 'kas is not enabled' do
before do
allow(Gitlab::Kas).to receive(:enabled?).and_return(false)
end
it 'raises a configuration error' do
expect { described_class.new }.to raise_error(described_class::ConfigurationError, 'GitLab KAS is not enabled')
end
end
context 'internal url is not set' do
before do
allow(Gitlab::Kas).to receive(:enabled?).and_return(true)
allow(Gitlab::Kas).to receive(:internal_url).and_return(nil)
end
it 'raises a configuration error' do
expect { described_class.new }.to raise_error(described_class::ConfigurationError, 'KAS internal URL is not configured')
end
end
end
describe 'gRPC calls' do
let(:token) { instance_double(JSONWebToken::HMACToken, encoded: 'test-token') }
before do
allow(Gitlab::Kas).to receive(:enabled?).and_return(true)
allow(Gitlab::Kas).to receive(:internal_url).and_return('grpc://example.kas.internal')
expect(JSONWebToken::HMACToken).to receive(:new)
.with(Gitlab::Kas.secret)
.and_return(token)
expect(token).to receive(:issuer=).with(Settings.gitlab.host)
expect(token).to receive(:audience=).with(described_class::JWT_AUDIENCE)
end
describe '#list_agent_config_files' do
let(:stub) { instance_double(Gitlab::Agent::ConfigurationProject::Rpc::ConfigurationProject::Stub) }
let(:request) { instance_double(Gitlab::Agent::ConfigurationProject::Rpc::ListAgentConfigFilesRequest) }
let(:response) { double(Gitlab::Agent::ConfigurationProject::Rpc::ListAgentConfigFilesResponse, config_files: agent_configurations) }
let(:repository) { instance_double(Gitlab::Agent::Modserver::Repository) }
let(:gitaly_address) { instance_double(Gitlab::Agent::Modserver::GitalyAddress) }
let(:agent_configurations) { [double] }
subject { described_class.new.list_agent_config_files(project: project) }
before do
expect(Gitlab::Agent::ConfigurationProject::Rpc::ConfigurationProject::Stub).to receive(:new)
.with('example.kas.internal', :this_channel_is_insecure, timeout: described_class::TIMEOUT)
.and_return(stub)
expect(Gitlab::Agent::Modserver::Repository).to receive(:new)
.with(project.repository.gitaly_repository.to_h)
.and_return(repository)
expect(Gitlab::Agent::Modserver::GitalyAddress).to receive(:new)
.with(Gitlab::GitalyClient.connection_data(project.repository_storage))
.and_return(gitaly_address)
expect(Gitlab::Agent::ConfigurationProject::Rpc::ListAgentConfigFilesRequest).to receive(:new)
.with(repository: repository, gitaly_address: gitaly_address)
.and_return(request)
expect(stub).to receive(:list_agent_config_files)
.with(request, metadata: { 'authorization' => 'bearer test-token' })
.and_return(response)
end
it { expect(subject).to eq(agent_configurations) }
end
end
end
...@@ -65,6 +65,12 @@ RSpec.describe Gitlab::Kas do ...@@ -65,6 +65,12 @@ RSpec.describe Gitlab::Kas do
end end
end end
describe '.internal_url' do
it 'returns gitlab_kas internal_url config' do
expect(described_class.internal_url).to eq(Gitlab.config.gitlab_kas.internal_url)
end
end
describe '.version' do describe '.version' do
it 'returns gitlab_kas version config' do it 'returns gitlab_kas version config' do
version_file = Rails.root.join(described_class::VERSION_FILE) version_file = Rails.root.join(described_class::VERSION_FILE)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment