Commit d7c1a9d9 authored by Achilleas Pipinellis's avatar Achilleas Pipinellis

Merge branch 'docs-add-indexes-monitoring' into 'master'

Docs: add indexes for monitoring and performance monitoring

See merge request gitlab-org/gitlab-ce!15942
parents 2c3e6f42 c234913e
...@@ -106,20 +106,22 @@ server with IMAP authentication on Ubuntu, to be used with Reply by email. ...@@ -106,20 +106,22 @@ server with IMAP authentication on Ubuntu, to be used with Reply by email.
## Monitoring GitLab ## Monitoring GitLab
- [Monitoring uptime](../user/admin_area/monitoring/health_check.md): Check the server status using the health check endpoint. - [Monitoring GitLab](monitoring/index.md):
- [Monitoring uptime](../user/admin_area/monitoring/health_check.md): Check the server status using the health check endpoint.
- [IP whitelist](monitoring/ip_whitelist.md): Monitor endpoints that provide health check information when probed. - [IP whitelist](monitoring/ip_whitelist.md): Monitor endpoints that provide health check information when probed.
- [Monitoring GitHub imports](monitoring/github_imports.md): GitLab's GitHub Importer displays Prometheus metrics to monitor the health and progress of the importer. - [Monitoring GitHub imports](monitoring/github_imports.md): GitLab's GitHub Importer displays Prometheus metrics to monitor the health and progress of the importer.
- [Conversational Development (ConvDev) Index](../user/admin_area/monitoring/convdev.md): Provides an overview of your entire instance's feature usage. - [Conversational Development (ConvDev) Index](../user/admin_area/monitoring/convdev.md): Provides an overview of your entire instance's feature usage.
### Performance Monitoring ### Performance Monitoring
- [GitLab Performance Monitoring](monitoring/performance/gitlab_configuration.md): Enable GitLab Performance Monitoring. - [GitLab Performance Monitoring](monitoring/performance/index.md):
- [GitLab performance monitoring with InfluxDB](monitoring/performance/introduction.md): Configure GitLab and InfluxDB for measuring performance metrics. - [Enable Performance Monitoring](monitoring/performance/gitlab_configuration.md): Enable GitLab Performance Monitoring.
- [GitLab performance monitoring with InfluxDB](monitoring/performance/influxdb_configuration.md): Configure GitLab and InfluxDB for measuring performance metrics.
- [InfluxDB Schema](monitoring/performance/influxdb_schema.md): Measurements stored in InfluxDB. - [InfluxDB Schema](monitoring/performance/influxdb_schema.md): Measurements stored in InfluxDB.
- [GitLab performance monitoring with Prometheus](monitoring/prometheus/index.md): Configure GitLab and Prometheus for measuring performance metrics. - [GitLab performance monitoring with Prometheus](monitoring/prometheus/index.md): Configure GitLab and Prometheus for measuring performance metrics.
- [GitLab performance monitoring with Grafana](monitoring/prometheus/index.md): Configure GitLab to visualize time series metrics through graphs and dashboards. - [GitLab performance monitoring with Grafana](monitoring/performance/grafana_configuration.md): Configure GitLab to visualize time series metrics through graphs and dashboards.
- [Request Profiling](monitoring/performance/request_profiling.md): Get a detailed profile on slow requests. - [Request Profiling](monitoring/performance/request_profiling.md): Get a detailed profile on slow requests.
- [Performance Bar](monitoring/performance/performance_bar.md): Get performance information for the current page. - [Performance Bar](monitoring/performance/performance_bar.md): Get performance information for the current page.
## Troubleshooting ## Troubleshooting
......
# Monitoring GitLab
Explore our features to monitor your GitLab instance:
- [Performance monitoring](performance/index.md): GitLab Performance Monitoring makes it possible to measure a wide variety of statistics of your instance.
- [Prometheus](prometheus/index.md): Prometheus is a powerful time-series monitoring service, providing a flexible platform for monitoring GitLab and other software products.
- [GitHub imports](github_imports.md): Monitor the health and progress of GitLab's GitHub importer with various Prometheus metrics.
- [Monitoring uptime](../user/admin_area/monitoring/health_check.md): Check the server status using the health check endpoint.
- [IP whitelists](ip_whitelist.md): Configure GitLab for monitoring endpoints that provide health check information when probed.
# GitLab Performance Monitoring
GitLab comes with its own application performance measuring system as of GitLab
8.4, simply called "GitLab Performance Monitoring". GitLab Performance Monitoring is available in both the
Community and Enterprise editions.
Apart from this introduction, you are advised to read through the following
documents in order to understand and properly configure GitLab Performance Monitoring:
- [GitLab Configuration](gitlab_configuration.md)
- [InfluxDB Install/Configuration](influxdb_configuration.md)
- [InfluxDB Schema](influxdb_schema.md)
- [Grafana Install/Configuration](grafana_configuration.md)
- [Performance bar](performance_bar.md)
- [Request profiling](request_profiling.md)
>**Note:**
Omnibus GitLab 8.16 includes Prometheus as an additional tool to collect
metrics. It will eventually replace InfluxDB when their metrics collection is
on par. Read more in the [Prometheus documentation](../prometheus/index.md).
## Introduction to GitLab Performance Monitoring
GitLab Performance Monitoring makes it possible to measure a wide variety of statistics
including (but not limited to):
- The time it took to complete a transaction (a web request or Sidekiq job).
- The time spent in running SQL queries and rendering HAML views.
- The time spent executing (instrumented) Ruby methods.
- Ruby object allocations, and retained objects in particular.
- System statistics such as the process' memory usage and open file descriptors.
- Ruby garbage collection statistics.
Metrics data is written to [InfluxDB][influxdb] over [UDP][influxdb-udp]. Stored
data can be visualized using [Grafana][grafana] or any other application that
supports reading data from InfluxDB. Alternatively data can be queried using the
InfluxDB CLI.
## Metric Types
Two types of metrics are collected:
1. Transaction specific metrics.
1. Sampled metrics, collected at a certain interval in a separate thread.
### Transaction Metrics
Transaction metrics are metrics that can be associated with a single
transaction. This includes statistics such as the transaction duration, timings
of any executed SQL queries, time spent rendering HAML views, etc. These metrics
are collected for every Rack request and Sidekiq job processed.
### Sampled Metrics
Sampled metrics are metrics that can't be associated with a single transaction.
Examples include garbage collection statistics and retained Ruby objects. These
metrics are collected at a regular interval. This interval is made up out of two
parts:
1. A user defined interval.
1. A randomly generated offset added on top of the interval, the same offset
can't be used twice in a row.
The actual interval can be anywhere between a half of the defined interval and a
half above the interval. For example, for a user defined interval of 15 seconds
the actual interval can be anywhere between 7.5 and 22.5. The interval is
re-generated for every sampling run instead of being generated once and re-used
for the duration of the process' lifetime.
[influxdb]: https://influxdata.com/time-series-platform/influxdb/
[influxdb-udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/
[grafana]: http://grafana.org/
# GitLab Performance Monitoring This document was moved to [another location](index.md).
GitLab comes with its own application performance measuring system as of GitLab
8.4, simply called "GitLab Performance Monitoring". GitLab Performance Monitoring is available in both the
Community and Enterprise editions.
Apart from this introduction, you are advised to read through the following
documents in order to understand and properly configure GitLab Performance Monitoring:
- [GitLab Configuration](gitlab_configuration.md)
- [InfluxDB Install/Configuration](influxdb_configuration.md)
- [InfluxDB Schema](influxdb_schema.md)
- [Grafana Install/Configuration](grafana_configuration.md)
>**Note:**
Omnibus GitLab 8.16 includes Prometheus as an additional tool to collect
metrics. It will eventually replace InfluxDB when their metrics collection is
on par. Read more in the [Prometheus documentation](../prometheus/index.md).
## Introduction to GitLab Performance Monitoring
GitLab Performance Monitoring makes it possible to measure a wide variety of statistics
including (but not limited to):
- The time it took to complete a transaction (a web request or Sidekiq job).
- The time spent in running SQL queries and rendering HAML views.
- The time spent executing (instrumented) Ruby methods.
- Ruby object allocations, and retained objects in particular.
- System statistics such as the process' memory usage and open file descriptors.
- Ruby garbage collection statistics.
Metrics data is written to [InfluxDB][influxdb] over [UDP][influxdb-udp]. Stored
data can be visualized using [Grafana][grafana] or any other application that
supports reading data from InfluxDB. Alternatively data can be queried using the
InfluxDB CLI.
## Metric Types
Two types of metrics are collected:
1. Transaction specific metrics.
1. Sampled metrics, collected at a certain interval in a separate thread.
### Transaction Metrics
Transaction metrics are metrics that can be associated with a single
transaction. This includes statistics such as the transaction duration, timings
of any executed SQL queries, time spent rendering HAML views, etc. These metrics
are collected for every Rack request and Sidekiq job processed.
### Sampled Metrics
Sampled metrics are metrics that can't be associated with a single transaction.
Examples include garbage collection statistics and retained Ruby objects. These
metrics are collected at a regular interval. This interval is made up out of two
parts:
1. A user defined interval.
1. A randomly generated offset added on top of the interval, the same offset
can't be used twice in a row.
The actual interval can be anywhere between a half of the defined interval and a
half above the interval. For example, for a user defined interval of 15 seconds
the actual interval can be anywhere between 7.5 and 22.5. The interval is
re-generated for every sampling run instead of being generated once and re-used
for the duration of the process' lifetime.
[influxdb]: https://influxdata.com/time-series-platform/influxdb/
[influxdb-udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/
[grafana]: http://grafana.org/
...@@ -37,7 +37,7 @@ graphs/dashboards. ...@@ -37,7 +37,7 @@ graphs/dashboards.
GitLab provides built-in tools to aid the process of improving performance: GitLab provides built-in tools to aid the process of improving performance:
* [Sherlock](profiling.md#sherlock) * [Sherlock](profiling.md#sherlock)
* [GitLab Performance Monitoring](../administration/monitoring/performance/introduction.md) * [GitLab Performance Monitoring](../administration/monitoring/performance/index.md)
* [Request Profiling](../administration/monitoring/performance/request_profiling.md) * [Request Profiling](../administration/monitoring/performance/request_profiling.md)
* [QueryRecoder](query_recorder.md) for preventing `N+1` regressions * [QueryRecoder](query_recorder.md) for preventing `N+1` regressions
......
This document was moved to [administration/monitoring/performance/introduction](../../administration/monitoring/performance/introduction.md). This document was moved to [administration/monitoring/performance/introduction](../../administration/monitoring/performance/index.md).
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment