Skip to content

G
gitlab-ce
Commit 0fb43b66 authored by GitLab Bot's avatar GitLab Bot
Browse files
Options

Automatic merge of gitlab-org/gitlab-ce master

parents 9a9e1e04 681fca60
Hide whitespace changes
Inline Side-by-side
Showing with 110 additions and 33 deletions
doc/administration/monitoring/prometheus/gitlab_metrics.md
View file @ 0fb43b66
...@@ -19,38 +19,106 @@ it, the client IP needs to be [included in a whitelist][whitelist]. ...@@ -19,38 +19,106 @@ it, the client IP needs to be [included in a whitelist][whitelist].
For Omnibus and Chart installations, these metrics are automatically enabled and collected as of [GitLab 9.4](https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1702). For source installations or earlier versions, these metrics will need to be enabled manually and collected by a Prometheus server. For Omnibus and Chart installations, these metrics are automatically enabled and collected as of [GitLab 9.4](https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1702). For source installations or earlier versions, these metrics will need to be enabled manually and collected by a Prometheus server.
## Unicorn Metrics available ## Metrics available
The following metrics are available: The following metrics are available:
| Metric | Type | Since | Description | | Metric | Type | Since | Description | Labels |
|:--------------------------------- |:--------- |:----- |:----------- | |:-------------------------------------------------------------|:----------|-----------------------:|:----------------------------------------------------------------------------------------------------|:----------------------------------------------------|
| db_ping_timeout | Gauge | 9.4 | Whether or not the last database ping timed out | | gitlab_banzai_cached_render_real_duration_seconds | Histogram | 9.4 | Duration of rendering markdown into HTML when cached output exists | controller, action |
| db_ping_success | Gauge | 9.4 | Whether or not the last database ping succeeded | | gitlab_banzai_cacheless_render_real_duration_seconds | Histogram | 9.4 | Duration of rendering markdown into HTML when cached outupt does not exist | controller, action |
| db_ping_latency_seconds | Gauge | 9.4 | Round trip time of the database ping | | gitlab_cache_misses_total | Counter | 10.2 | Cache read miss | controller, action |
| filesystem_access_latency_seconds | Gauge | 9.4 | Latency in accessing a specific filesystem | | gitlab_cache_operation_duration_seconds | Histogram | 10.2 | Cache access time | |
| filesystem_accessible | Gauge | 9.4 | Whether or not a specific filesystem is accessible | | gitlab_cache_operations_total | Counter | 12.2 | Cache operations by controller/action | controller, action, operation |
| filesystem_write_latency_seconds | Gauge | 9.4 | Write latency of a specific filesystem | | gitlab_database_transaction_seconds | Histogram | 12.1 | Time spent in database transactions, in seconds | |
| filesystem_writable | Gauge | 9.4 | Whether or not the filesystem is writable | | gitlab_method_call_duration_seconds | Histogram | 10.2 | Method calls real duration | controller, action, module, method |
| filesystem_read_latency_seconds | Gauge | 9.4 | Read latency of a specific filesystem | | gitlab_rails_queue_duration_seconds | Histogram | 9.4 | Measures latency between gitlab-workhorse forwarding a request to Rails | |
| filesystem_readable | Gauge | 9.4 | Whether or not the filesystem is readable | | gitlab_sql_duration_seconds | Histogram | 10.2 | SQL execution time, excluding SCHEMA operations and BEGIN / COMMIT | |
| gitlab_cache_misses_total | Counter | 10.2 | Cache read miss | | gitlab_transaction_allocated_memory_bytes | Histogram | 10.2 | Allocated memory for all transactions (gitlab_transaction_* metrics) | |
| gitlab_cache_operation_duration_seconds | Histogram | 10.2 | Cache access time | | gitlab_transaction_cache_<key>_count_total | Counter | 10.2 | Counter for total Rails cache calls (per key) | |
| gitlab_cache_operations_total | Counter | 12.2 | Cache operations by controller/action | | gitlab_transaction_cache_<key>_duration_total | Counter | 10.2 | Counter for total time (seconds) spent in Rails cache calls (per key) | |
| http_requests_total | Counter | 9.4 | Rack request count | | gitlab_transaction_cache_count_total | Counter | 10.2 | Counter for total Rails cache calls (aggregate) | |
| http_request_duration_seconds | Histogram | 9.4 | HTTP response time from rack middleware | | gitlab_transaction_cache_duration_total | Counter | 10.2 | Counter for total time (seconds) spent in Rails cache calls (aggregate) | |
| pipelines_created_total | Counter | 9.4 | Counter of pipelines created | | gitlab_transaction_cache_read_hit_count_total | Counter | 10.2 | Counter for cache hits for Rails cache calls | controller, action |
| rack_uncaught_errors_total | Counter | 9.4 | Rack connections handling uncaught errors count | | gitlab_transaction_cache_read_miss_count_total | Counter | 10.2 | Counter for cache misses for Rails cache calls | controller, action |
| redis_ping_timeout | Gauge | 9.4 | Whether or not the last redis ping timed out | | gitlab_transaction_duration_seconds | Histogram | 10.2 | Duration for all transactions (gitlab_transaction_* metrics) | controller, action |
| redis_ping_success | Gauge | 9.4 | Whether or not the last redis ping succeeded | | gitlab_transaction_event_build_found_total | Counter | 9.4 | Counter for build found for api /jobs/request | |
| redis_ping_latency_seconds | Gauge | 9.4 | Round trip time of the redis ping | | gitlab_transaction_event_build_invalid_total | Counter | 9.4 | Counter for build invalid due to concurrency conflict for api /jobs/request | |
| user_session_logins_total | Counter | 9.4 | Counter of how many users have logged in | | gitlab_transaction_event_build_not_found_cached_total | Counter | 9.4 | Counter for cached response of build not found for api /jobs/request | |
| upload_file_does_not_exist | Counter | 10.7 in EE, 11.5 in CE | Number of times an upload record could not find its file | | gitlab_transaction_event_build_not_found_total | Counter | 9.4 | Counter for build not found for api /jobs/request | |
| failed_login_captcha_total | Gauge | 11.0 | Counter of failed CAPTCHA attempts during login | | gitlab_transaction_event_change_default_branch_total | Counter | 9.4 | Counter when default branch is changed for any repository | |
| successful_login_captcha_total | Gauge | 11.0 | Counter of successful CAPTCHA attempts during login | | gitlab_transaction_event_create_repository_total | Counter | 9.4 | Counter when any repository is created | |
| unicorn_active_connections | Gauge | 11.0 | The number of active Unicorn connections (workers) | | gitlab_transaction_event_etag_caching_cache_hit_total | Counter | 9.4 | Counter for etag cache hit. | endpoint |
| unicorn_queued_connections | Gauge | 11.0 | The number of queued Unicorn connections | | gitlab_transaction_event_etag_caching_header_missing_total | Counter | 9.4 | Counter for etag cache miss - header missing | endpoint |
| unicorn_workers | Gauge | 12.0 | The number of Unicorn workers | | gitlab_transaction_event_etag_caching_key_not_found_total | Counter | 9.4 | Counter for etag cache miss - key not found | endpoint |
| gitlab_transaction_event_etag_caching_middleware_used_total | Counter | 9.4 | Counter for etag middleware accessed | endpoint |
| gitlab_transaction_event_etag_caching_resource_changed_total | Counter | 9.4 | Counter for etag cache miss - resource changed | endpoint |
| gitlab_transaction_event_fork_repository_total | Counter | 9.4 | Counter for repository forks (RepositoryForkWorker). Only incremented when source repository exists | |
| gitlab_transaction_event_import_repository_total | Counter | 9.4 | Counter for repository imports (RepositoryImportWorker) | |
| gitlab_transaction_event_push_branch_total | Counter | 9.4 | Counter for all branch pushes | |
| gitlab_transaction_event_push_commit_total | Counter | 9.4 | Counter for commits | branch |
| gitlab_transaction_event_push_tag_total | Counter | 9.4 | Counter for tag pushes | |
| gitlab_transaction_event_rails_exception_total | Counter | 9.4 | Counter for number of rails exceptions | |
| gitlab_transaction_event_receive_email_total | Counter | 9.4 | Counter for recieved emails | handler |
| gitlab_transaction_event_remote_mirrors_failed_total | Counter | 10.8 | Counter for failed remote mirrors | |
| gitlab_transaction_event_remote_mirrors_finished_total | Counter | 10.8 | Counter for finished remote mirrors | |
| gitlab_transaction_event_remote_mirrors_running_total | Counter | 10.8 | Counter for running remote mirrors | |
| gitlab_transaction_event_remove_branch_total | Counter | 9.4 | Counter when a branch is removed for any repository | |
| gitlab_transaction_event_remove_repository_total | Counter | 9.4 | Counter when a repository is removed | |
| gitlab_transaction_event_remove_tag_total | Counter | 9.4 | Counter when a tag is remove for any repository | |
| gitlab_transaction_event_sidekiq_exception_total | Counter | 9.4 | Counter of sidekiq exceptions | |
| gitlab_transaction_event_stuck_import_jobs_total | Counter | 9.4 | Count of stuck import jobs | projects_without_jid_count, projects_with_jid_count |
| gitlab_transaction_event_update_build_total | Counter | 9.4 | Counter for update build for api /jobs/request/:id | |
| gitlab_transaction_new_redis_connections_total | Counter | 9.4 | Counter for new redis connections | |
| gitlab_transaction_queue_duration_total | Counter | 9.4 | Duration jobs were enqueued before processing | |
| gitlab_transaction_rails_queue_duration_total | Counter | 9.4 | Measures latency between gitlab-workhorse forwarding a request to Rails | controller, action |
| gitlab_transaction_view_duration_total | Counter | 9.4 | Duration for views | controller, action, view |
| gitlab_view_rendering_duration_seconds | Histogram | 10.2 | Duration for views (histogram) | controller, action, view |
| http_requests_total | Counter | 9.4 | Rack request count | method |
| http_request_duration_seconds | Histogram | 9.4 | HTTP response time from rack middleware | method, status |
| pipelines_created_total | Counter | 9.4 | Counter of pipelines created | |
| rack_uncaught_errors_total | Counter | 9.4 | Rack connections handling uncaught errors count | |
| user_session_logins_total | Counter | 9.4 | Counter of how many users have logged in | |
| upload_file_does_not_exist | Counter | 10.7 in EE, 11.5 in CE | Number of times an upload record could not find its file | |
| failed_login_captcha_total | Gauge | 11.0 | Counter of failed CAPTCHA attempts during login | |
| successful_login_captcha_total | Gauge | 11.0 | Counter of successful CAPTCHA attempts during login | |
## Metrics controlled by a feature flag
The following metrics can be controlled by feature flags:
| Metric | Feature Flag |
|:-------------------------------------------------------------|:-----------------------------------------------------------------|
| gitlab_method_call_duration_seconds | prometheus_metrics_method_instrumentation |
| gitlab_transaction_allocated_memory_bytes | prometheus_metrics_transaction_allocated_memory |
| gitlab_transaction_event_build_found_total | prometheus_transaction_event_build_found_total |
| gitlab_transaction_event_build_invalid_total | prometheus_transaction_event_build_invalid_total |
| gitlab_transaction_event_build_not_found_cached_total | prometheus_transaction_event_build_not_found_cached_total |
| gitlab_transaction_event_build_not_found_total | prometheus_transaction_event_build_not_found_total |
| gitlab_transaction_event_change_default_branch_total | prometheus_transaction_event_change_default_branch_total |
| gitlab_transaction_event_create_repository_total | prometheus_transaction_event_create_repository_total |
| gitlab_transaction_event_etag_caching_cache_hit_total | prometheus_transaction_event_etag_caching_cache_hit_total |
| gitlab_transaction_event_etag_caching_header_missing_total | prometheus_transaction_event_etag_caching_header_missing_total |
| gitlab_transaction_event_etag_caching_key_not_found_total | prometheus_transaction_event_etag_caching_key_not_found_total |
| gitlab_transaction_event_etag_caching_middleware_used_total | prometheus_transaction_event_etag_caching_middleware_used_total |
| gitlab_transaction_event_etag_caching_resource_changed_total | prometheus_transaction_event_etag_caching_resource_changed_total |
| gitlab_transaction_event_fork_repository_total | prometheus_transaction_event_fork_repository_total |
| gitlab_transaction_event_import_repository_total | prometheus_transaction_event_import_repository_total |
| gitlab_transaction_event_push_branch_total | prometheus_transaction_event_push_branch_total |
| gitlab_transaction_event_push_commit_total | prometheus_transaction_event_push_commit_total |
| gitlab_transaction_event_push_tag_total | prometheus_transaction_event_push_tag_total |
| gitlab_transaction_event_rails_exception_total | prometheus_transaction_event_rails_exception_total |
| gitlab_transaction_event_receive_email_total | prometheus_transaction_event_receive_email_total |
| gitlab_transaction_event_remote_mirrors_failed_total | prometheus_transaction_event_remote_mirrors_failed_total |
| gitlab_transaction_event_remote_mirrors_finished_total | prometheus_transaction_event_remote_mirrors_finished_total |
| gitlab_transaction_event_remote_mirrors_running_total | prometheus_transaction_event_remote_mirrors_running_total |
| gitlab_transaction_event_remove_branch_total | prometheus_transaction_event_remove_branch_total |
| gitlab_transaction_event_remove_repository_total | prometheus_transaction_event_remove_repository_total |
| gitlab_transaction_event_remove_tag_total | prometheus_transaction_event_remove_tag_total |
| gitlab_transaction_event_sidekiq_exception_total | prometheus_transaction_event_sidekiq_exception_total |
| gitlab_transaction_event_stuck_import_jobs_total | prometheus_transaction_event_stuck_import_jobs_total |
| gitlab_transaction_event_update_build_total | prometheus_transaction_event_update_build_total |
| gitlab_view_rendering_duration_seconds | prometheus_metrics_view_instrumentation |
## Sidekiq Metrics available for Geo **(PREMIUM)** ## Sidekiq Metrics available for Geo **(PREMIUM)**
...@@ -99,17 +167,27 @@ Some basic Ruby runtime metrics are available: ...@@ -99,17 +167,27 @@ Some basic Ruby runtime metrics are available:
| Metric | Type | Since | Description | | Metric | Type | Since | Description |
|:-------------------------------------- |:--------- |:----- |:----------- | |:-------------------------------------- |:--------- |:----- |:----------- |
| ruby_gc_duration_seconds_total | Counter | 11.1 | Time spent by Ruby in GC | | ruby_gc_duration_seconds | Counter | 11.1 | Time spent by Ruby in GC |
| ruby_gc_stat_... | Gauge | 11.1 | Various metrics from [GC.stat] | | ruby_gc_stat_... | Gauge | 11.1 | Various metrics from [GC.stat] |
| ruby_file_descriptors | Gauge | 11.1 | File descriptors per process | | ruby_file_descriptors | Gauge | 11.1 | File descriptors per process |
| ruby_memory_bytes | Gauge | 11.1 | Memory usage by process | | ruby_memory_bytes | Gauge | 11.1 | Memory usage by process |
| ruby_sampler_duration_seconds_total | Counter | 11.1 | Time spent collecting stats | | ruby_sampler_duration_seconds | Counter | 11.1 | Time spent collecting stats |
| ruby_process_cpu_seconds_total | Gauge | 12.0 | Total amount of CPU time per process | | ruby_process_cpu_seconds_total | Gauge | 12.0 | Total amount of CPU time per process |
| ruby_process_max_fds | Gauge | 12.0 | Maximum number of open file descriptors per process | | ruby_process_max_fds | Gauge | 12.0 | Maximum number of open file descriptors per process |
| ruby_process_resident_memory_bytes | Gauge | 12.0 | Memory usage by process, measured in bytes | | ruby_process_resident_memory_bytes | Gauge | 12.0 | Memory usage by process, measured in bytes |
| ruby_process_start_time_seconds | Gauge | 12.0 | UNIX timestamp of process start time | | ruby_process_start_time_seconds | Gauge | 12.0 | UNIX timestamp of process start time |
[GC.stat]: https://ruby-doc.org/core-2.3.0/GC.html#method-c-stat [GC.stat]: https://ruby-doc.org/core-2.6.3/GC.html#method-c-stat
## Unicorn Metrics
Unicorn specific metrics, when Unicorn is used.
| Metric | Type | Since | Description |
|:---------------------------|:------|:------|:---------------------------------------------------|
| unicorn_active_connections | Gauge | 11.0 | The number of active Unicorn connections (workers) |
| unicorn_queued_connections | Gauge | 11.0 | The number of queued Unicorn connections |
| unicorn_workers | Gauge | 12.0 | The number of Unicorn workers |
## Puma Metrics **(EXPERIMENTAL)** ## Puma Metrics **(EXPERIMENTAL)**
...@@ -126,7 +204,6 @@ When Puma is used instead of Unicorn, following metrics are available: ...@@ -126,7 +204,6 @@ When Puma is used instead of Unicorn, following metrics are available:
| puma_pool_capacity | Gauge | 12.0 | Number of requests the worker is capable of taking right now | | puma_pool_capacity | Gauge | 12.0 | Number of requests the worker is capable of taking right now |
| puma_max_threads | Gauge | 12.0 | Maximum number of worker threads | | puma_max_threads | Gauge | 12.0 | Maximum number of worker threads |
| puma_idle_threads | Gauge | 12.0 | Number of spawned threads which are not processing a request | | puma_idle_threads | Gauge | 12.0 | Number of spawned threads which are not processing a request |
| rack_state_total | Gauge | 12.0 | Number of requests in a given rack state |
| puma_killer_terminations_total | Gauge | 12.0 | Number of workers terminated by PumaWorkerKiller | | puma_killer_terminations_total | Gauge | 12.0 | Number of workers terminated by PumaWorkerKiller |
## Metrics shared directory ## Metrics shared directory
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7