health_check.md 4.35 KB
Newer Older
1 2
# Health Check

3 4 5 6 7
>**Notes:**
  - Liveness and readiness probes were [introduced][ce-10416] in GitLab 9.1.
  - The `health_check` endpoint was [introduced][ce-3888] in GitLab 8.8 and will
    be deprecated in GitLab 9.1. Read more in the [old behavior](#old-behavior)
    section.
Pawel Chojnacki's avatar
Pawel Chojnacki committed
8 9
  - [Access token](#access-token) has been deprecated in GitLab 9.4
    in favor of [IP Whitelist](#ip-whitelist)
10 11 12 13 14 15

GitLab provides liveness and readiness probes to indicate service health and
reachability to required services. These probes report on the status of the
database connection, Redis connection, and access to the filesystem. These
endpoints [can be provided to schedulers like Kubernetes][kubernetes] to hold
traffic until the system is ready or restart the container as needed.
16

17 18
## IP Whitelist

19 20
To access monitoring resources the client IP needs to be included in the whitelist.
To add or remove hosts or IP ranges from the list you can edit `gitlab.rb` or `gitlab.yml`.
21 22 23 24 25 26 27 28 29

Example whitelist configuration:
```yaml
monitoring:
  ip_whitelist:
    - 127.0.0.0/8 # by default only local IPs are allowed to access monitoring resources
```

## Access Token (Deprecated)
30

31 32 33
An access token needs to be provided while accessing the probe endpoints. The current
accepted token can be found under the **Admin area ➔ Monitoring ➔ Health check**
(`admin/health_check`) page of your GitLab instance.
34 35 36 37 38 39

![access token](img/health_check_token.png)

The access token can be passed as a URL parameter:

```
40
https://gitlab.example.com/-/readiness?token=ACCESS_TOKEN
41 42
```

43
which will then provide a report of system health in JSON format:
44

45 46 47 48 49 50 51 52 53 54 55 56 57 58 59
```
{
  "db_check": {
    "status": "ok"
  },
  "redis_check": {
    "status": "ok"
  },
  "fs_shards_check": {
    "status": "ok",
    "labels": {
      "shard": "default"
    }
  }
}
60 61 62 63
```

## Using the Endpoint

64
With default whitelist settings, the probes can be accessed from localhost:
65

66 67
- `http://localhost/-/readiness`
- `http://localhost/-/liveness`
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87

## Status

On failure, the endpoint will return a `500` HTTP status code. On success, the endpoint
will return a valid successful HTTP status code, and a `success` message.

## Old behavior

>**Notes:**
  - Liveness and readiness probes were [introduced][ce-10416] in GitLab 9.1.
  - The `health_check` endpoint was [introduced][ce-3888] in GitLab 8.8 and will
    be deprecated in GitLab 9.1. Read more in the [old behavior](#old-behavior)
    section.

GitLab provides a health check endpoint for uptime monitoring on the `health_check` web
endpoint. The health check reports on the overall system status based on the status of
the database connection, the state of the database migrations, and the ability to write
and access the cache. This endpoint can be provided to uptime monitoring services like
[Pingdom][pingdom], [Nagios][nagios-health], and [NewRelic][newrelic-health].

Pawel Chojnacki's avatar
Pawel Chojnacki committed
88 89
Once you have the [access token](#access-token) or your client IP is [whitelisted](#ip-whitelist),
health information can be retrieved as plain text, JSON, or XML using the `health_check` endpoint:
90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116

- `https://gitlab.example.com/health_check?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check.json?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check.xml?token=ACCESS_TOKEN`

You can also ask for the status of specific services:

- `https://gitlab.example.com/health_check/cache.json?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check/database.json?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check/migrations.json?token=ACCESS_TOKEN`

For example, the JSON output of the following health check:

```bash
curl --header "TOKEN: ACCESS_TOKEN" https://gitlab.example.com/health_check.json
```

would be like:

```
{"healthy":true,"message":"success"}
```

On failure, the endpoint will return a `500` HTTP status code. On success, the endpoint
will return a valid successful HTTP status code, and a `success` message. Ideally your
uptime monitoring should look for the success message.

Ben Bodenmiller's avatar
Ben Bodenmiller committed
117
[ce-10416]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10416
118 119 120 121
[ce-3888]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3888
[pingdom]: https://www.pingdom.com
[nagios-health]: https://nagios-plugins.org/doc/man/check_http.html
[newrelic-health]: https://docs.newrelic.com/docs/alerts/alert-policies/downtime-alerts/availability-monitoring
122
[kubernetes]: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/