@@ -21,7 +21,7 @@ When you are optimizing your SQL queries, there are two dimensions to pay attent
| Queries in a migration | `100ms` | This is different than the total [migration time](database_review.md#timing-guidelines-for-migrations). |
| Concurrent operations in a migration | `5min` | Concurrent operations do not block the database, but they block the GitLab update. This includes operations such as `add_concurrent_index` and `add_concurrent_foreign_key`. |
| Background migrations | `1s` | |
| Service Ping | `1s` | See the [Service Ping docs](usage_ping/index.md#developing-and-testing-service-ping) for more details. |
| Service Ping | `1s` | See the [Service Ping docs](service_ping/index.md#developing-and-testing-service-ping) for more details. |
- When analyzing your query's performance, pay attention to if the time you are seeing is on a [cold or warm cache](#cold-and-warm-cache). These guidelines apply for both cache types.
- When working with batched queries, change the range and batch size to see how it effects the query timing and caching.
info:To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Metrics instrumentation guide
This guide describes how to develop Service Ping metrics using metrics instrumentation.
## Nomenclature
-**Instrumentation class**:
- Inherits one of the metric classes: `DatabaseMetric`, `RedisHLLMetric` or `GenericMetric`.
- Implements the logic that calculates the value for a Service Ping metric.
-**Metric definition**
The Service Data metric YAML definition.
-**Hardening**:
Hardening a method is the process that ensures the method fails safe, returning a fallback value like -1.
## How it works
A metric definition has the [`instrumentation_class`](metrics_dictionary.md) field, which can be set to a class.
The defined instrumentation class should have one of the existing metric classes: `DatabaseMetric`, `RedisHLLMetric`, or `GenericMetric`.
Using the instrumentation classes ensures that metrics can fail safe individually, without breaking the entire
process of Service Ping generation.
We have built a domain-specific language (DSL) to define the metrics instrumentation.
## Database metrics
[Example of a merge request that adds a database metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60022).
```ruby
moduleGitlab
moduleUsage
moduleMetrics
moduleInstrumentations
classCountBoardsMetric<DatabaseMetric
operation:count
relation{Board}
end
end
end
end
end
```
## Redis HyperLogLog metrics
[Example of a merge request that adds a `RedisHLL` metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/61685).
Count unique values for `i_quickactions_approve` event.
[Example of a merge request that adds a generic metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60256).
```ruby
moduleGitlab
moduleUsage
moduleMetrics
moduleInstrumentations
classUuidMetric<GenericMetric
valuedo
Gitlab::CurrentSettings.uuid
end
end
end
end
end
end
```
## Creating a new metric instrumentation class
To create a stub instrumentation for a Service Ping metric, you can use a dedicated [generator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/generators/gitlab/usage_metric_generator.rb):
The generator takes the class name as an argument and the following options:
-`--type=TYPE` Required. Indicates the metric type. It must be one of: `database`, `generic`, `redis_hll`.
info:To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
- Decide whether a Product Intelligence review is needed.
- If a Product Intelligence review is needed, add the labels
`~product intelligence` and `~product intelligence::review pending`.
- Assign an
[engineer](https://gitlab.com/groups/gitlab-org/growth/product-intelligence/engineers/-/group_members?with_inherited_permissions=exclude) from the Product Intelligence team for a review.
- Set the correct attributes in the metric's YAML definition:
[Metrics Dictionary](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/usage_ping/dictionary.md) if needed.
- Add a changelog [according to guidelines](../changelog.md).
#### The Product Intelligence **reviewer** should
- Perform a first-pass review on the merge request and suggest improvements to the author.
- Check the [metrics location](index.md#1-naming-and-placing-the-metrics) in
the Service Ping JSON payload.
- Add the `~database` label and ask for a [database review](../database_review.md) for
metrics that are based on Database.
- For tracking using Redis HLL (HyperLogLog):
- Check the Redis slot.
- Check if a [feature flag is needed](index.md#recommendations).
- For a metric's YAML definition:
- Check the metric's `description`.
- Check the metric's `key_path`.
- Check the `product_section`, `product_stage`, `product_group`, and `product_category` fields.
Read the [stages file](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml).
- Check the file location. Consider the time frame, and if the file should be under `ee`.
- Check the tiers.
- Approve the MR, and relabel the MR with `~"product intelligence::approved"`.
## Review workload distribution
[Danger bot](../dangerbot.md) adds the list of changed Product Intelligence files
and pings the
[`@gitlab-org/growth/product-intelligence/engineers`](https://gitlab.com/groups/gitlab-org/growth/product-intelligence/engineers/-/group_members?with_inherited_permissions=exclude) group for merge requests
that are not drafts.
Any of the Product Intelligence engineers can be assigned for the Product Intelligence review.
info:To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
[Example of a merge request that adds a generic metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60256).
```ruby
moduleGitlab
moduleUsage
moduleMetrics
moduleInstrumentations
classUuidMetric<GenericMetric
valuedo
Gitlab::CurrentSettings.uuid
end
end
end
end
end
end
```
## Creating a new metric instrumentation class
To create a stub instrumentation for a Service Ping metric, you can use a dedicated [generator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/generators/gitlab/usage_metric_generator.rb):
The generator takes the class name as an argument and the following options:
-`--type=TYPE` Required. Indicates the metric type. It must be one of: `database`, `generic`, `redis_hll`.
info:To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
- Decide whether a Product Intelligence review is needed.
- If a Product Intelligence review is needed, add the labels
`~product intelligence` and `~product intelligence::review pending`.
- Assign an
[engineer](https://gitlab.com/groups/gitlab-org/growth/product-intelligence/engineers/-/group_members?with_inherited_permissions=exclude) from the Product Intelligence team for a review.
- Set the correct attributes in the metric's YAML definition:
[Metrics Dictionary](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/development/usage_ping/dictionary.md) if needed.
- Add a changelog [according to guidelines](../changelog.md).
#### The Product Intelligence **reviewer** should
- Perform a first-pass review on the merge request and suggest improvements to the author.
- Check the [metrics location](index.md#1-naming-and-placing-the-metrics) in
the Service Ping JSON payload.
- Add the `~database` label and ask for a [database review](../database_review.md) for
metrics that are based on Database.
- For tracking using Redis HLL (HyperLogLog):
- Check the Redis slot.
- Check if a [feature flag is needed](index.md#recommendations).
- For a metric's YAML definition:
- Check the metric's `description`.
- Check the metric's `key_path`.
- Check the `product_section`, `product_stage`, `product_group`, and `product_category` fields.
Read the [stages file](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml).
- Check the file location. Consider the time frame, and if the file should be under `ee`.
- Check the tiers.
- Approve the MR, and relabel the MR with `~"product intelligence::approved"`.
## Review workload distribution
[Danger bot](../dangerbot.md) adds the list of changed Product Intelligence files
and pings the
[`@gitlab-org/growth/product-intelligence/engineers`](https://gitlab.com/groups/gitlab-org/growth/product-intelligence/engineers/-/group_members?with_inherited_permissions=exclude) group for merge requests
that are not drafts.
Any of the Product Intelligence engineers can be assigned for the Product Intelligence review.
<!-- This redirect file can be deleted after <2021-10-09>. -->
<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
@@ -46,7 +46,7 @@ The following Rake tasks are available for use with GitLab:
| [Repository storage](../administration/raketasks/storage.md) | List and migrate existing projects and attachments from legacy storage to hashed storage. |
| [Uploads migrate](../administration/raketasks/uploads/migrate.md) | Migrate uploads between local storage and object storage. |
| [Uploads sanitize](../administration/raketasks/uploads/sanitize.md) | Remove EXIF data from images uploaded to earlier versions of GitLab. |
| [Service Data](../administration/troubleshooting/gitlab_rails_cheat_sheet.md#generate-service-ping) | Generate and troubleshoot [Service Ping](../development/usage_ping/index.md). |
| [Service Data](../administration/troubleshooting/gitlab_rails_cheat_sheet.md#generate-service-ping) | Generate and troubleshoot [Service Ping](../development/service_ping/index.md). |
| [User management](user_management.md) | Perform user management tasks. |
