Commit 61f4d08b authored by Zeger-Jan van de Weg's avatar Zeger-Jan van de Weg

Allow testing on Gitaly call count

Previous efforts were aimed at detecting N + 1 queries, general
regressions are hard to find and mitigate.
parent 2087f121
...@@ -52,8 +52,8 @@ rm -rf tmp/tests/gitaly ...@@ -52,8 +52,8 @@ rm -rf tmp/tests/gitaly
## `TooManyInvocationsError` errors ## `TooManyInvocationsError` errors
During development and testing, you may experience `Gitlab::GitalyClient::TooManyInvocationsError` failures. During development and testing, you may experience `Gitlab::GitalyClient::TooManyInvocationsError` failures.
The `GitalyClient` will attempt to block against potential n+1 issues by raising this error The `GitalyClient` will attempt to block against potential n+1 issues by raising this error
when Gitaly is called more than 30 times in a single Rails request or Sidekiq execution. when Gitaly is called more than 30 times in a single Rails request or Sidekiq execution.
As a temporary measure, export `GITALY_DISABLE_REQUEST_LIMITS=1` to suppress the error. This will disable the n+1 detection As a temporary measure, export `GITALY_DISABLE_REQUEST_LIMITS=1` to suppress the error. This will disable the n+1 detection
...@@ -64,7 +64,7 @@ Please raise an issue in the GitLab CE or EE repositories to report the issue. I ...@@ -64,7 +64,7 @@ Please raise an issue in the GitLab CE or EE repositories to report the issue. I
`TooManyInvocationsError`. Also include any known failing tests if possible. `TooManyInvocationsError`. Also include any known failing tests if possible.
Isolate the source of the n+1 problem. This will normally be a loop that results in Gitaly being called for each Isolate the source of the n+1 problem. This will normally be a loop that results in Gitaly being called for each
element in an array. If you are unable to isolate the problem, please contact a member element in an array. If you are unable to isolate the problem, please contact a member
of the [Gitaly Team](https://gitlab.com/groups/gl-gitaly/group_members) for assistance. of the [Gitaly Team](https://gitlab.com/groups/gl-gitaly/group_members) for assistance.
Once the source has been found, wrap it in an `allow_n_plus_1_calls` block, as follows: Once the source has been found, wrap it in an `allow_n_plus_1_calls` block, as follows:
...@@ -79,6 +79,34 @@ end ...@@ -79,6 +79,34 @@ end
Once the code is wrapped in this block, this code-path will be excluded from n+1 detection. Once the code is wrapped in this block, this code-path will be excluded from n+1 detection.
## Request counts
Commits and other git data, is now fetched through Gitaly. These fetches can,
much like with a database, be batched. This improves performance for the client
and for Gitaly itself and therefore for the users too. To keep performance stable
and guard performance regressions, Gitaly calls can be counted and the call count
can be tested against:
```ruby
describe 'Gitaly Request count tests' do
context 'when the request store is activated', :request_store do
it 'correctly counts the gitaly requests made' do
count = gitaly_request_count { subject }
expect { subject }.to change { Gitlab::GitalyClient.get_request_count }.by(10)
end
end
context 'when the request store is disabled' do
it 'is always zero' do
subject
expect(Gitlab::GitalyClient.get_request_count).to be_zero
end
end
end
```
--- ---
[Return to Development documentation](README.md) [Return to Development documentation](README.md)
...@@ -3,32 +3,36 @@ require 'spec_helper' ...@@ -3,32 +3,36 @@ require 'spec_helper'
describe Projects::PipelinesController do describe Projects::PipelinesController do
include ApiHelpers include ApiHelpers
let(:user) { create(:user) } set(:user) { create(:user) }
let(:project) { create(:project, :public) } set(:project) { create(:project, :public, :repository) }
let(:feature) { ProjectFeature::DISABLED } let(:feature) { ProjectFeature::DISABLED }
before do before do
stub_not_protect_default_branch stub_not_protect_default_branch
project.add_developer(user) project.add_developer(user)
project.project_feature.update( project.project_feature.update(builds_access_level: feature)
builds_access_level: feature)
sign_in(user) sign_in(user)
end end
describe 'GET index.json' do describe 'GET index.json' do
before do before do
create(:ci_empty_pipeline, status: 'pending', project: project) branch_head = project.commit
create(:ci_empty_pipeline, status: 'running', project: project) parent = branch_head.parent
create(:ci_empty_pipeline, status: 'created', project: project)
create(:ci_empty_pipeline, status: 'success', project: project)
get :index, namespace_id: project.namespace, create(:ci_empty_pipeline, status: 'pending', project: project, sha: branch_head.id)
project_id: project, create(:ci_empty_pipeline, status: 'running', project: project, sha: branch_head.id)
format: :json create(:ci_empty_pipeline, status: 'created', project: project, sha: parent.id)
create(:ci_empty_pipeline, status: 'success', project: project, sha: parent.id)
end
subject do
get :index, namespace_id: project.namespace, project_id: project, format: :json
end end
it 'returns JSON with serialized pipelines' do it 'returns JSON with serialized pipelines' do
subject
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(response).to match_response_schema('pipeline') expect(response).to match_response_schema('pipeline')
...@@ -39,6 +43,12 @@ describe Projects::PipelinesController do ...@@ -39,6 +43,12 @@ describe Projects::PipelinesController do
expect(json_response['count']['pending']).to eq 1 expect(json_response['count']['pending']).to eq 1
expect(json_response['count']['finished']).to eq 1 expect(json_response['count']['finished']).to eq 1
end end
context 'when performing gitaly calls', :request_store do
it 'limits the Gitaly requests' do
expect { subject }.to change { Gitlab::GitalyClient.get_request_count }.by(10)
end
end
end end
describe 'GET show JSON' do describe 'GET show JSON' do
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment