Commit 39e7e42d authored by Amy Qualls's avatar Amy Qualls

Remove information from the Prometheus page

Move more info from the Prometheus page to where it belongs in the
Operations section.
parent 08bc371d
...@@ -73,11 +73,11 @@ you should ...@@ -73,11 +73,11 @@ you should
## Taking action on Prometheus alerts **(ULTIMATE)** ## Taking action on Prometheus alerts **(ULTIMATE)**
You can [add a webhook](../../../operations/metrics/index.md#external-prometheus-instances) You can [add a webhook](../../../operations/metrics/alerts.md#external-prometheus-instances)
to the Prometheus configuration in order for GitLab to receive notifications of any alerts. to the Prometheus configuration in order for GitLab to receive notifications of any alerts.
Once the webhook is setup, you can Once the webhook is setup, you can
[take action on incoming alerts](../../../operations/metrics/index.md#taking-action-on-incidents-ultimate). [take action on incoming alerts](../../../operations/metrics/alerts.md#trigger-actions-from-alerts-ultimate).
## Adding custom metrics to the self monitoring project ## Adding custom metrics to the self monitoring project
......
---
stage: Monitor
group: APM
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
---
# Set up alerts for Prometheus metrics
After [configuring metrics for your CI/CD environment](index.md), you can set up
alerting for Prometheus metrics depending on the location of your instances, and
[trigger actions from alerts](#trigger-actions-from-alerts-ultimate) to notify
your team when environment performance falls outside of the boundaries you set.
## Managed Prometheus instances
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/6590) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.2 for [custom metrics](index.md#adding-custom-metrics), and 11.3 for [library metrics](../../user/project/integrations/prometheus_library/metrics.md).
For managed Prometheus instances using auto configuration, you can
[configure alerts for metrics](index.md#adding-custom-metrics) directly in the
[metrics dashboard](index.md). To set an alert:
1. In your project, navigate to **{cloud-gear}** **Operations > Metrics**,
1. Identify the metric you want to create the alert for, and click the
**ellipsis** **{ellipsis_v}** icon in the top right corner of the metric.
1. Choose **Alerts**.
1. Set threshold and operator.
1. Click **Add** to save and activate the alert.
![Adding an alert](../../user/project/integrations/img/prometheus_alert.png)
To remove the alert, click back on the alert icon for the desired metric, and click **Delete**.
## External Prometheus instances
>- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9258) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.8.
>- [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/42640) to [GitLab Core](https://about.gitlab.com/pricing/) in 12.10.
For manually configured Prometheus servers, GitLab provides a notify endpoint for
use with Prometheus webhooks. If you have manual configuration enabled, an
**Alerts** section is added to **{settings}** **Settings > Integrations > Prometheus**.
This section contains the **URL** and **Authorization Key** you will need. The
**Reset Key** button will invalidate the key and generate a new one.
![Prometheus service configuration of Alerts](../../user/project/integrations/img/prometheus_service_alerts.png)
To send GitLab alert notifications, copy the **URL** and **Authorization Key** into the
[`webhook_configs`](https://prometheus.io/docs/alerting/latest/configuration/#webhook_config)
section of your Prometheus Alertmanager configuration:
```yaml
receivers:
name: gitlab
webhook_configs:
- http_config:
bearer_token: 9e1cbfcd546896a9ea8be557caf13a76
send_resolved: true
url: http://192.168.178.31:3001/root/manual_prometheus/prometheus/alerts/notify.json
...
```
For GitLab to associate your alerts with an [environment](../../ci/environments/index.md),
you must configure a `gitlab_environment_name` label on the alerts you set up in
Prometheus. The value of this should match the name of your environment in GitLab.
NOTE: **Note:**
In GitLab versions 13.1 and greater, you can configure your manually configured
Prometheus server to use the
[Generic alerts integration](../../user/project/integrations/generic_alerts.md).
## Trigger actions from alerts **(ULTIMATE)**
>- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/4925) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.11.
>- [From GitLab Ultimate 12.5](https://gitlab.com/gitlab-org/gitlab/-/issues/13401), when GitLab receives a recovery alert, it will automatically close the associated issue.
Alerts can be used to trigger actions, like opening an issue automatically
(disabled by default since `13.1`). To configure the actions:
1. Navigate to your project's **{settings}** **Settings > Operations > Incidents**.
1. Enable the option to create issues.
1. Choose the [issue template](../../user/project/description_templates.md) to create the issue from.
1. Optionally, select whether to send an email notification to the developers of the project.
1. Click **Save changes**.
After enabling, GitLab automatically opens an issue when an alert is triggered containing
values extracted from [alert's payload](https://prometheus.io/docs/alerting/latest/configuration/#webhook_config):
- Issue author: `GitLab Alert Bot`
- Issue title: Extract from `annotations/title`, `annotations/summary` or `labels/alertname`
- Alert `Summary`: A list of properties
- `starts_at`: Alert start time via `startsAt`
- `full_query`: Alert query extracted from `generatorURL`
- Optional list of attached annotations extracted from `annotations/*`
- Alert [GFM](../../user/markdown.md): GitLab Flavored Markdown from `annotations/gitlab_incident_markdown`
When GitLab receives a **Recovery Alert**, it closes the associated issue.
This action is recorded as a system message on the issue indicating that it
was closed automatically by the GitLab Alert bot.
To further customize the issue, you can add labels, mentions, or any other supported
[quick action](../../user/project/quick_actions.md) in the selected issue template,
which applies to all incidents. To limit quick actions or other information to
only specific types of alerts, use the `annotations/gitlab_incident_markdown` field.
Since [version 12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/63373),
GitLab tags each incident issue with the `incident` label automatically. If the label
does not yet exist, it is also created automatically.
If the metric exceeds the threshold of the alert for over 5 minutes, GitLab sends
an email to all [Maintainers and Owners](../../user/permissions.md#project-members-permissions)
of the project.
...@@ -6,32 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w ...@@ -6,32 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Using the Metrics Dashboard # Using the Metrics Dashboard
## Select an environment
The **Environment** dropdown box above the dashboard displays the list of all [environments](../index.md).
It enables you to search as you type through all environments and select the one you're looking for.
![Monitoring Dashboard Environments](../../../user/project/integrations/img/prometheus_dashboard_environments_v12_8.png)
## Select a dashboard
The **dashboard** dropdown box above the dashboard displays the list of all dashboards available for the project.
It enables you to search as you type through all dashboards and select the one you're looking for.
![Monitoring Dashboard select](../../../user/project/integrations/img/prometheus_dashboard_select_v_13_0.png)
## Mark a dashboard as favorite
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214582) in GitLab 13.0.
When viewing a dashboard, click the empty **Star dashboard** **{star-o}** button to mark a
dashboard as a favorite. Starred dashboards display a solid star **{star}** button,
and appear at the top of the dashboard select list.
To remove dashboard from the favorites list, click the solid **Unstar Dashboard** **{star}** button.
![Monitoring Dashboard favorite state toggle](../../../user/project/integrations/img/toggle_metrics_user_starred_dashboard_v13_0.png)
## Manage the metrics dashboard settings ## Manage the metrics dashboard settings
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223204) in GitLab 13.2. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223204) in GitLab 13.2.
...@@ -57,7 +31,7 @@ The options are: ...@@ -57,7 +31,7 @@ The options are:
- [View logs](#view-logs-ultimate) - [View logs](#view-logs-ultimate)
- [Download CSV](#downloading-data-as-csv) - [Download CSV](#downloading-data-as-csv)
- [Copy link to chart](../embed.md#embedding-gitlab-managed-kubernetes-metrics) - [Copy link to chart](../embed.md#embedding-gitlab-managed-kubernetes-metrics)
- [Alerts](../index.md#setting-up-alerts-for-prometheus-metrics) - [Alerts](../alerts.md)
### View and edit the source file of a custom dashboard ### View and edit the source file of a custom dashboard
......
...@@ -76,7 +76,7 @@ Panels in a panel group are laid out in rows consisting of two panels per row. A ...@@ -76,7 +76,7 @@ Panels in a panel group are laid out in rows consisting of two panels per row. A
| Property | Type | Required | Description | | Property | Type | Required | Description |
| ------ | ------ | ------ | ------ | | ------ | ------ | ------ | ------ |
| `id` | string | no | Used for associating dashboard metrics with database records. Must be unique across dashboard configuration files. Required for [alerting](../index.md#setting-up-alerts-for-prometheus-metrics) (support not yet enabled, see [relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/27980)). | | `id` | string | no | Used for associating dashboard metrics with database records. Must be unique across dashboard configuration files. Required for [alerting](../alerts.md) (support not yet enabled, see [relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/27980)). |
| `unit` | string | yes | Defines the unit of the query's return data. | | `unit` | string | yes | Defines the unit of the query's return data. |
| `label` | string | no, but highly encouraged | Defines the legend-label for the query. Should be unique within the panel's metrics. Can contain time series labels as interpolated variables. | | `label` | string | no, but highly encouraged | Defines the legend-label for the query. Should be unique within the panel's metrics. Can contain time series labels as interpolated variables. |
| `query` | string | yes if `query_range` is not defined | Defines the Prometheus query to be used to populate the chart/panel. If defined, the `query` endpoint of the [Prometheus API](https://prometheus.io/docs/prometheus/latest/querying/api/) will be utilized. | | `query` | string | yes if `query_range` is not defined | Defines the Prometheus query to be used to populate the chart/panel. If defined, the `query` endpoint of the [Prometheus API](https://prometheus.io/docs/prometheus/latest/querying/api/) will be utilized. |
......
...@@ -58,7 +58,7 @@ It is also possible to embed either the default dashboard metrics or individual ...@@ -58,7 +58,7 @@ It is also possible to embed either the default dashboard metrics or individual
## Embedding metrics based on alerts in incident issues ## Embedding metrics based on alerts in incident issues
For [GitLab-managed alerting rules](index.md#setting-up-alerts-for-prometheus-metrics), the issue will include an embedded chart for the query corresponding to the alert. The chart displays an hour of data surrounding the starting point of the incident, 30 minutes before and after. For [GitLab-managed alerting rules](alerts.md), the issue will include an embedded chart for the query corresponding to the alert. The chart displays an hour of data surrounding the starting point of the incident, 30 minutes before and after.
For [manually configured Prometheus instances](../../user/project/integrations/prometheus.md#manual-configuration-of-prometheus), a chart corresponding to the query can be included if these requirements are met: For [manually configured Prometheus instances](../../user/project/integrations/prometheus.md#manual-configuration-of-prometheus), a chart corresponding to the query can be included if these requirements are met:
......
...@@ -4,14 +4,81 @@ group: APM ...@@ -4,14 +4,81 @@ group: APM
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
--- ---
# Monitor CI/CD Environment metrics # Monitor metrics for your CI/CD environment
Once [configured](../../user/project/integrations/prometheus.md), GitLab will attempt to retrieve performance metrics for any After [configuring Prometheus for a cluster](../../user/project/integrations/prometheus.md),
environment which has had a successful deployment. GitLab attempts to retrieve performance metrics for any [environment](../../ci/environments/index.md) with
a successful deployment.
GitLab will automatically scan the Prometheus server for metrics from known servers like Kubernetes and NGINX, and attempt to identify individual environments. The supported metrics and scan process is detailed in our [Prometheus Metrics Library documentation](../../user/project/integrations/prometheus_library/index.md).
GitLab scans the Prometheus server for metrics from known servers like Kubernetes
You can view the performance dashboard for an environment by [clicking on the monitoring button](../../ci/environments/index.md#monitoring-environments). and NGINX, and attempts to identify individual environments. To learn more about
the supported metrics and scan processes, see the
[Prometheus Metrics Library documentation](../../user/project/integrations/prometheus_library/index.md).
To view the metrics dashboard for an environment that has
[completed at least one deployment](#populate-your-metrics-dashboard):
1. *If the metrics dashboard is only visible to project members,* sign in to
GitLab as a member of a project. Learn more about [metrics dashboard visibility](#metrics-dashboard-visibility).
1. In your project, navigate to **{cloud-gear}** **Operations > Metrics**.
GitLab displays the default metrics dashboard for the environment, like the
following example:
![Example of metrics dashboard](img/example-dashboard_v13_1.png)
The top of the dashboard contains a navigation bar. From left to right, the
navigation bar contains:
- **Dashboard** - A dropdown list of all dashboards available for the project,
with starred dashboards listed first.
- **Environment** - A dropdown list of all [environments](../index.md) that searches
through all environments as you type.
- **Range** - The time period of data to display.
- **Refresh dashboard** **{retry}** - Reload the dashboard with current data.
- **Set refresh rate** - Set a time frame for refreshing the data displayed.
- **Star dashboard** **{star-o}** - Click to mark a dashboard as a favorite.
Starred dashboards display a solid star **{star}** button, and display first
in the **Dashboard** dropdown list.
([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214582) in GitLab 13.0.)
- **Create dashboard** **{file-addition-solid}** - Create a
[new custom dashboard for your project](dashboards/index.md#adding-a-new-dashboard-to-your-project).
- **Metrics settings** **{settings}** - Configure the
[settings for this dashboard](dashboards/index.md#manage-the-metrics-dashboard-settings).
## Populate your metrics dashboard
After [configuring Prometheus for a cluster](../../user/project/integrations/prometheus.md),
you must also deploy code for the **{cloud-gear}** **Operations > Metrics** page
to contain data. Setting up [Auto DevOps](../../topics/autodevops/index.md)
helps quickly create a deployment:
1. Navigate to your project's **{cloud-gear}** **Operations > Kubernetes** page.
1. Ensure that, in addition to Prometheus, you also have Runner and Ingress
installed.
1. After installing Ingress, copy its endpoint.
1. Navigate to your project's **{settings}** **Settings > CI/CD** page. In the
**Auto DevOps** section, select a deployment strategy and save your changes.
1. On the same page, in the **Variables** section, add a variable named
`KUBE_INGRESS_BASE_DOMAIN` with the value of the Ingress endpoint you
copied previously. Leave the type as **Variable**.
1. Navigate to your project's **{rocket}** **CI/CD > Pipelines** page, and run a
pipeline on any branch.
1. When the pipeline has run successfully, graphs are available on the
**{cloud-gear}** **Operations > Metrics** page.
![Monitoring Dashboard](../../user/project/integrations/img/prometheus_monitoring_dashboard_v13_1.png)
## Customize your metrics dashboard
After creating your dashboard, you can customize it to meet your needs:
- **Add custom metrics**: In addition to the GitLab default metrics, you can
[create custom metrics](#adding-custom-metrics) and display them on your metrics dashboard.
- **Configure alerts for metrics**: [Configure custom alerts](alerts.md) for your team when
environment performance falls outside of the boundaries you set.
- **Trigger actions from alerts**: [Open new issues for your team](alerts.md#trigger-actions-from-alerts-ultimate) **(ULTIMATE)**
when environment performance falls outside of the boundaries you set.
## Metrics dashboard visibility ## Metrics dashboard visibility
...@@ -26,9 +93,12 @@ dashboard is visible to authenticated and non-authenticated users. ...@@ -26,9 +93,12 @@ dashboard is visible to authenticated and non-authenticated users.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3799) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.6. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3799) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.6.
> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28527) to [GitLab Core](https://about.gitlab.com/pricing/) 12.10. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28527) to [GitLab Core](https://about.gitlab.com/pricing/) 12.10.
Custom metrics can be monitored by adding them on the monitoring dashboard page. Once saved, they will be displayed on the environment performance dashboard provided that either: Custom metrics can be monitored by adding them on the monitoring dashboard page.
After saving them, they display on the environment metrics dashboard provided that either:
- A [connected Kubernetes cluster](../../user/project/clusters/add_remove_clusters.md) with the environment scope of `*` is used and [Prometheus installed on the cluster](../../user/project/integrations/prometheus.md#enabling-prometheus-integration) - A [connected Kubernetes cluster](../../user/project/clusters/add_remove_clusters.md)
with the matching [environment scope](../../ci/environments/index.md#scoping-environments-with-specs) is used and
[Prometheus installed on the cluster](../../user/project/integrations/prometheus.md#enabling-prometheus-integration).
- Prometheus is [manually configured](../../user/project/integrations/prometheus.md#manual-configuration-of-prometheus). - Prometheus is [manually configured](../../user/project/integrations/prometheus.md#manual-configuration-of-prometheus).
![Add New Metric](../../user/project/integrations/img/prometheus_add_metric.png) ![Add New Metric](../../user/project/integrations/img/prometheus_add_metric.png)
...@@ -41,98 +111,29 @@ A few fields are required: ...@@ -41,98 +111,29 @@ A few fields are required:
- **Y-axis label**: Y axis title to display on the dashboard. - **Y-axis label**: Y axis title to display on the dashboard.
- **Unit label**: Query units, for example `req / sec`. Shown next to the value. - **Unit label**: Query units, for example `req / sec`. Shown next to the value.
Multiple metrics can be displayed on the same chart if the fields **Name**, **Type**, and **Y-axis label** match between metrics. For example, a metric with **Name** `Requests Rate`, **Type** `Business`, and **Y-axis label** `rec / sec` would display on the same chart as a second metric with the same values. A **Legend label** is suggested if this feature is used. Multiple metrics can be displayed on the same chart if the fields **Name**, **Type**,
and **Y-axis label** match between metrics. For example, a metric with **Name**
`Requests Rate`, **Type** `Business`, and **Y-axis label** `rec / sec` would display
on the same chart as a second metric with the same values. A **Legend label** is
suggested if this feature is used.
## Editing additional metrics from the dashboard ## Editing additional metrics from the dashboard
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208976) in GitLab 12.9. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208976) in GitLab 12.9.
You can edit existing additional custom metrics by clicking the **{ellipsis_v}** **More actions** dropdown and selecting **Edit metric**. You can edit existing additional custom metrics for your dashboard by clicking the
**{ellipsis_v}** **More actions** dropdown and selecting **Edit metric**.
![Edit metric](../../user/project/integrations/img/prometheus_dashboard_edit_metric_link_v_12_9.png) ![Edit metric](../../user/project/integrations/img/prometheus_dashboard_edit_metric_link_v_12_9.png)
## Setting up alerts for Prometheus metrics
### Managed Prometheus instances
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/6590) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.2 for [custom metrics](#adding-custom-metrics), and 11.3 for [library metrics](../../user/project/integrations/prometheus_library/metrics.md).
For managed Prometheus instances using auto configuration, alerts for metrics [can be configured](#adding-custom-metrics) directly in the performance dashboard.
To set an alert:
1. Click on the ellipsis icon in the top right corner of the metric you want to create the alert for.
1. Choose **Alerts**
1. Set threshold and operator.
1. Click **Add** to save and activate the alert.
![Adding an alert](../../user/project/integrations/img/prometheus_alert.png)
To remove the alert, click back on the alert icon for the desired metric, and click **Delete**.
### External Prometheus instances
>- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9258) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.8.
>- [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/42640) to [GitLab Core](https://about.gitlab.com/pricing/) in 12.10.
For manually configured Prometheus servers, a notify endpoint is provided to use with Prometheus webhooks. If you have manual configuration enabled, an **Alerts** section is added to **Settings > Integrations > Prometheus**. This contains the *URL* and *Authorization Key*. The **Reset Key** button will invalidate the key and generate a new one.
![Prometheus service configuration of Alerts](../../user/project/integrations/img/prometheus_service_alerts.png)
To send GitLab alert notifications, copy the *URL* and *Authorization Key* into the [`webhook_configs`](https://prometheus.io/docs/alerting/latest/configuration/#webhook_config) section of your Prometheus Alertmanager configuration:
```yaml
receivers:
name: gitlab
webhook_configs:
- http_config:
bearer_token: 9e1cbfcd546896a9ea8be557caf13a76
send_resolved: true
url: http://192.168.178.31:3001/root/manual_prometheus/prometheus/alerts/notify.json
...
```
In order for GitLab to associate your alerts with an [environment](../../ci/environments/index.md), you need to configure a `gitlab_environment_name` label on the alerts you set up in Prometheus. The value of this should match the name of your Environment in GitLab.
NOTE: **Note:**
In GitLab versions 13.1 and greater, you can configure your manually configured Prometheus server to use the [Generic alerts integration](../../user/project/integrations/generic_alerts.md).
## Taking action on incidents **(ULTIMATE)**
>- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/4925) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.11.
>- [From GitLab Ultimate 12.5](https://gitlab.com/gitlab-org/gitlab/-/issues/13401), when GitLab receives a recovery alert, it will automatically close the associated issue.
Alerts can be used to trigger actions, like opening an issue automatically (disabled by default since `13.1`). To configure the actions:
1. Navigate to your project's **Settings > Operations > Incidents**.
1. Enable the option to create issues.
1. Choose the [issue template](../../user/project/description_templates.md) to create the issue from.
1. Optionally, select whether to send an email notification to the developers of the project.
1. Click **Save changes**.
Once enabled, an issue will be opened automatically when an alert is triggered which contains values extracted from [alert's payload](https://prometheus.io/docs/alerting/latest/configuration/#webhook_config):
- Issue author: `GitLab Alert Bot`
- Issue title: Extract from `annotations/title`, `annotations/summary` or `labels/alertname`
- Alert `Summary`: A list of properties
- `starts_at`: Alert start time via `startsAt`
- `full_query`: Alert query extracted from `generatorURL`
- Optional list of attached annotations extracted from `annotations/*`
- Alert [GFM](../../user/markdown.md): GitLab Flavored Markdown from `annotations/gitlab_incident_markdown`
When GitLab receives a **Recovery Alert**, it will automatically close the associated issue. This action will be recorded as a system message on the issue indicating that it was closed automatically by the GitLab Alert bot.
To further customize the issue, you can add labels, mentions, or any other supported [quick action](../../user/project/quick_actions.md) in the selected issue template, which will apply to all incidents. To limit quick actions or other information to only specific types of alerts, use the `annotations/gitlab_incident_markdown` field.
Since [version 12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/63373), GitLab will tag each incident issue with the `incident` label automatically. If the label does not yet exist, it will be created automatically as well.
If the metric exceeds the threshold of the alert for over 5 minutes, an email will be sent to all [Maintainers and Owners](../../user/permissions.md#project-members-permissions) of the project.
## Keyboard shortcuts for charts ## Keyboard shortcuts for charts
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202146) in GitLab 13.2. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202146) in GitLab 13.2.
You can use keyboard shortcuts to interact more quickly with your currently-focused chartpanel. To activate keyboard shortcuts, use keyboard tabs to highlight the**{ellipsis_v}** **More actions** dropdown menu, or hover over the dropdown menu with your mouse, then press the key corresponding to your desired action: You can use keyboard shortcuts to interact more quickly with your currently-focused
chart panel. To activate keyboard shortcuts, use keyboard tabs to highlight the
**{ellipsis_v}** **More actions** dropdown menu, or hover over the dropdown menu
with your mouse, then press the key corresponding to your desired action:
- **Expand panel** - <kbd>e</kbd> - **Expand panel** - <kbd>e</kbd>
- **View logs** - <kbd>l</kbd> (lowercase 'L') - **View logs** - <kbd>l</kbd> (lowercase 'L')
......
...@@ -25,7 +25,7 @@ to create issues when alerts are triggered: ...@@ -25,7 +25,7 @@ to create issues when alerts are triggered:
checkbox to create an issue based on your own checkbox to create an issue based on your own
[issue templates](../project/description_templates.md#creating-issue-templates). [issue templates](../project/description_templates.md#creating-issue-templates).
For more information, see For more information, see
[Taking Action on Incidents](../../operations/metrics/index.md#taking-action-on-incidents-ultimate) **(ULTIMATE)**. [Trigger actions from alerts](../../operations/metrics/alerts.md#trigger-actions-from-alerts-ultimate) **(ULTIMATE)**.
1. To create issues from alerts, select the template in the **Issue Template** 1. To create issues from alerts, select the template in the **Issue Template**
select box. select box.
1. To send [separate email notifications](#notify-developers-of-alerts) to users 1. To send [separate email notifications](#notify-developers-of-alerts) to users
...@@ -36,7 +36,7 @@ to create issues when alerts are triggered: ...@@ -36,7 +36,7 @@ to create issues when alerts are triggered:
Appropriately configured alerts include an Appropriately configured alerts include an
[embedded chart](../../operations/metrics/embed.md#embedding-metrics-based-on-alerts-in-incident-issues) [embedded chart](../../operations/metrics/embed.md#embedding-metrics-based-on-alerts-in-incident-issues)
for the query corresponding to the alert. You can also configure GitLab to for the query corresponding to the alert. You can also configure GitLab to
[close issues](../../operations/metrics/index.md#taking-action-on-incidents-ultimate) [close issues](../../operations/metrics/alerts.md#trigger-actions-from-alerts-ultimate)
when you receive notification that the alert is resolved. when you receive notification that the alert is resolved.
### Notify developers of alerts ### Notify developers of alerts
...@@ -53,8 +53,8 @@ To send separate email notifications to users with ...@@ -53,8 +53,8 @@ To send separate email notifications to users with
You can set up Prometheus alerts in: You can set up Prometheus alerts in:
- [GitLab-managed Prometheus](../../operations/metrics/index.md#setting-up-alerts-for-prometheus-metrics) installations. - [GitLab-managed Prometheus](../../operations/metrics/alerts.md) installations.
- [Self-managed Prometheus](../../operations/metrics/index.md#external-prometheus-instances) installations. - [Self-managed Prometheus](../../operations/metrics/alerts.md#external-prometheus-instances) installations.
Prometheus alerts are created by the special Alert Bot user. You can't remove this Prometheus alerts are created by the special Alert Bot user. You can't remove this
user, but it does not count toward your license limit. user, but it does not count toward your license limit.
......
...@@ -19,7 +19,9 @@ The entire payload will be posted in the issue discussion as a comment ...@@ -19,7 +19,9 @@ The entire payload will be posted in the issue discussion as a comment
authored by the GitLab Alert Bot. authored by the GitLab Alert Bot.
NOTE: **Note:** NOTE: **Note:**
In GitLab versions 13.1 and greater, you can configure [External Prometheus instances](../../../operations/metrics/index.md#external-prometheus-instances) to use this endpoint. In GitLab versions 13.1 and greater, you can configure
[External Prometheus instances](../../../operations/metrics/alerts.md#external-prometheus-instances)
to use this endpoint.
## Setting up generic alerts ## Setting up generic alerts
......
...@@ -19,7 +19,8 @@ There are two ways to set up Prometheus integration, depending on where your app ...@@ -19,7 +19,8 @@ There are two ways to set up Prometheus integration, depending on where your app
- For deployments on Kubernetes, GitLab can automatically [deploy and manage Prometheus](#managed-prometheus-on-kubernetes). - For deployments on Kubernetes, GitLab can automatically [deploy and manage Prometheus](#managed-prometheus-on-kubernetes).
- For other deployment targets, simply [specify the Prometheus server](#manual-configuration-of-prometheus). - For other deployment targets, simply [specify the Prometheus server](#manual-configuration-of-prometheus).
Once enabled, GitLab will automatically detect metrics from known services in the [metric library](prometheus_library/index.md). You can also [add your own metrics](../../../operations/metrics/index.md#adding-custom-metrics). Once enabled, GitLab will automatically detect metrics from known services in the [metric library](prometheus_library/index.md). You can also [add your own metrics](../../../operations/metrics/index.md#adding-custom-metrics) and create
[custom dashboards](../../../operations/metrics/dashboards/index.md).
## Enabling Prometheus Integration ## Enabling Prometheus Integration
...@@ -43,25 +44,6 @@ Once you have a connected Kubernetes cluster, deploying a managed Prometheus is ...@@ -43,25 +44,6 @@ Once you have a connected Kubernetes cluster, deploying a managed Prometheus is
![Managed Prometheus Deploy](img/prometheus_deploy.png) ![Managed Prometheus Deploy](img/prometheus_deploy.png)
#### Getting metrics to display on the Metrics Dashboard
After completing the steps above, you will also need deployments in order to view the
**Operations > Metrics** page. Setting up [Auto DevOps](../../../topics/autodevops/index.md)
will help you to quickly create a deployment:
1. Navigate to your project's **Operations > Kubernetes** page, and ensure that,
in addition to "Prometheus", you also have "Runner" and "Ingress"
installed. Once "Ingress" is installed, copy its endpoint.
1. Navigate to your project's **Settings > CI/CD** page. In the Auto DevOps section,
select a deployment strategy and save your changes.
1. On the same page, in the Variables section, add a variable named `KUBE_INGRESS_BASE_DOMAIN`
with the value of the Ingress endpoint you have copied in the previous step. Leave the type
as "Variable".
1. Navigate to your project's **CI/CD > Pipelines** page, and run a pipeline on any branch.
1. When the pipeline has run successfully, graphs will be available on the **Operations > Metrics** page.
![Monitoring Dashboard](img/prometheus_monitoring_dashboard_v13_1.png)
#### About managed Prometheus deployments #### About managed Prometheus deployments
Prometheus is deployed into the `gitlab-managed-apps` namespace, using the [official Helm chart](https://github.com/helm/charts/tree/master/stable/prometheus). Prometheus is only accessible within the cluster, with GitLab communicating through the [Kubernetes API](https://kubernetes.io/docs/concepts/overview/kubernetes-api/). Prometheus is deployed into the `gitlab-managed-apps` namespace, using the [official Helm chart](https://github.com/helm/charts/tree/master/stable/prometheus). Prometheus is only accessible within the cluster, with GitLab communicating through the [Kubernetes API](https://kubernetes.io/docs/concepts/overview/kubernetes-api/).
......
...@@ -65,16 +65,16 @@ When GitLab-managed Prometheus is installed, the [Alerts list](#alert-management ...@@ -65,16 +65,16 @@ When GitLab-managed Prometheus is installed, the [Alerts list](#alert-management
is also enabled. is also enabled.
To populate the alerts with data, see To populate the alerts with data, see
[GitLab-Managed Prometheus instances](../../../operations/metrics/index.md#managed-prometheus-instances). [GitLab-Managed Prometheus instances](../../../operations/metrics/alerts.md#managed-prometheus-instances).
### Enable external Prometheus alerts ### Enable external Prometheus alerts
You can configure an externally-managed Prometheus instance to send alerts You can configure an externally-managed Prometheus instance to send alerts
to GitLab. To set up this configuration, see the [configuring Prometheus](../../../operations/metrics/index.md#external-prometheus-instances) documentation. Activating the external Prometheus to GitLab. To set up this configuration, see the [configuring Prometheus](../../../operations/metrics/alerts.md#external-prometheus-instances) documentation. Activating the external Prometheus
configuration also enables the [Alerts list](#alert-management-list). configuration also enables the [Alerts list](#alert-management-list).
To populate the alerts with data, see To populate the alerts with data, see
[External Prometheus instances](../../../operations/metrics/index.md#external-prometheus-instances). [External Prometheus instances](../../../operations/metrics/alerts.md#external-prometheus-instances).
## Alert Management severity ## Alert Management severity
...@@ -240,14 +240,14 @@ To view the metrics for an alert: ...@@ -240,14 +240,14 @@ To view the metrics for an alert:
For GitLab-managed Prometheus instances, metrics data is automatically available For GitLab-managed Prometheus instances, metrics data is automatically available
for the alert, making it easy to see surrounding behavior. See for the alert, making it easy to see surrounding behavior. See
[Managed Prometheus instances](../../../operations/metrics/index.md#managed-prometheus-instances) [Managed Prometheus instances](../../../operations/metrics/alerts.md#managed-prometheus-instances)
for information on setting up alerts. for information on setting up alerts.
For externally-managed Prometheus instances, you can configure your alerting rules to For externally-managed Prometheus instances, you can configure your alerting rules to
display a chart in the alert. See display a chart in the alert. See
[Embedding metrics based on alerts in incident issues](../../../operations/metrics/embed.md#embedding-metrics-based-on-alerts-in-incident-issues) [Embedding metrics based on alerts in incident issues](../../../operations/metrics/embed.md#embedding-metrics-based-on-alerts-in-incident-issues)
for information on how to appropriately configure your alerting rules. See for information on how to appropriately configure your alerting rules. See
[External Prometheus instances](../../../operations/metrics/index.md#external-prometheus-instances) [External Prometheus instances](../../../operations/metrics/alerts.md#external-prometheus-instances)
for information on setting up alerts for your self-managed Prometheus instance. for information on setting up alerts for your self-managed Prometheus instance.
## Use cases for assigning alerts ## Use cases for assigning alerts
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment