info:To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Add an existing Kubernetes cluster
If you have an existing Kubernetes cluster, you can add it to a project, group,
or instance and benefit from the integration with GitLab.
## Prerequisites
See the prerequisites below to add existing clusters to GitLab.
### All clusters
To add any cluster to GitLab, you need:
- Either a GitLab.com account or an account for a self-managed installation
running GitLab 12.5 or later.
- Maintainer permissions for group-level and project-level clusters.
- Access to the Admin area for instance-level clusters. **(FREE SELF)**
- A Kubernetes cluster.
- Cluster administration access to the cluster with `kubectl`.
You can host your cluster in [EKS](#eks-clusters), [GKE](#gke-clusters),
on premises, and with other providers.
To host them on premises and with other providers,
use either the EKS or GKE method to guide you through and enter your cluster's
settings manually.
WARNING:
GitLab doesn't support `arm64` clusters. See the issue
[Helm Tiller fails to install on `arm64` cluster](https://gitlab.com/gitlab-org/gitlab/-/issues/29838)
for details.
### EKS clusters
To add an existing **EKS** cluster, you need:
- An Amazon EKS cluster with worker nodes properly configured.
-`kubectl`[installed and configured](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html#get-started-kubectl)
for access to the EKS cluster.
- Ensure the token of the account has administrator privileges for the cluster.
### GKE clusters
To add an existing **GKE** cluster, you need:
- The `container.clusterRoleBindings.create` permission to create a cluster
role binding. You can follow the [Google Cloud documentation](https://cloud.google.com/iam/docs/granting-changing-revoking-access)
to grant access.
## How to add an existing cluster
<!-- (REVISE - BREAK INTO SMALLER STEPS) -->
To add a Kubernetes cluster to your project, group, or instance:
1. Navigate to your:
1. Project's **{cloud-gear}****Infrastructure > Kubernetes clusters** page, for a project-level cluster.
1. Group's **{cloud-gear}****Kubernetes** page, for a group-level cluster.
1.**Menu >****{admin}****Admin >****{cloud-gear}****Kubernetes** page, for an instance-level cluster.
1. Click **Add Kubernetes cluster**.
1. Click the **Add existing cluster** tab and fill in the details:
1.**Kubernetes cluster name** (required) - The name you wish to give the cluster.
1.**Environment scope** (required) - The
[associated environment](index.md#setting-the-environment-scope) to this cluster.
1.**API URL** (required) -
It's the URL that GitLab uses to access the Kubernetes API. Kubernetes
exposes several APIs, we want the "base" URL that is common to all of them.
For example, `https://kubernetes.example.com` rather than `https://kubernetes.example.com/api/v1`.
info:To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Adding GKE clusters **(FREE)**
# GKE clusters (DEPRECATED) **(FREE)**
> - [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/6049) in GitLab 14.0.
WARNING:
Use [Infrastrucure as Code](../../infrastructure/index.md) to create new clusters. The method described in this document is deprecated as of GitLab 14.0.
Through GitLab, you can create new clusters and add existing clusters hosted on Amazon Elastic
Kubernetes Service (EKS).
GitLab supports adding new and existing GKE clusters.
...
...
@@ -19,7 +27,12 @@ requirements are met:
take up to 10 minutes after you create a project. For more information see the
["Before you begin" section of the Kubernetes Engine docs](https://cloud.google.com/kubernetes-engine/docs/quickstart#before-you-begin).
## New GKE cluster
## Add an existing GKE cluster
If you already have a GKE cluster and want to integrate it with GitLab,
see how to [add an existing cluster](add_existing_cluster.md).
## Create new GKE cluster
Starting from [GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/issues/25925), all the GKE clusters
provisioned by GitLab are [VPC-native](https://cloud.google.com/kubernetes-engine/docs/how-to/alias-ips).
...
...
@@ -30,13 +43,13 @@ Note the following:
at the instance level. If that's not the case, ask your GitLab administrator to enable it. On
GitLab.com, this is enabled.
- Starting from [GitLab 12.1](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/55902), all GKE clusters
created by GitLab are RBAC-enabled. Take a look at the [RBAC section](add_remove_clusters.md#rbac-cluster-resources) for
created by GitLab are RBAC-enabled. Take a look at the [RBAC section](cluster_access.md#rbac-cluster-resources) for
more information.
- Starting from [GitLab 12.5](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18341), the
cluster's pod address IP range is set to `/16` instead of the regular `/14`. `/16` is a CIDR
notation.
- GitLab requires basic authentication enabled and a client certificate issued for the cluster to
set up an [initial service account](add_remove_clusters.md#access-controls). In [GitLab versions
set up an [initial service account](cluster_access.md). In [GitLab versions
11.10 and later](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/58208), the cluster creation process
explicitly requests GKE to create clusters with basic authentication enabled and a client
certificate.
...
...
@@ -49,7 +62,7 @@ To create and add a new Kubernetes cluster to your project, group, or instance:
- Project's **{cloud-gear}****Infrastructure > Kubernetes clusters** page, for a project-level
cluster.
- Group's **{cloud-gear}****Kubernetes** page, for a group-level cluster.
-**Admin Area >****{cloud-gear}****Kubernetes** page, for an instance-level cluster.
-**Menu >****{admin}****Admin >****{cloud-gear}****Kubernetes** page, for an instance-level cluster.
1. Click **Integrate with a cluster certificate**.
1. Under the **Create new cluster** tab, click **Google GKE**.
1. Connect your Google account if you haven't done already by clicking the
...
...
@@ -81,8 +94,3 @@ You can choose to use Cloud Run for Anthos in place of installing Knative and Is
separately after the cluster has been created. This means that Cloud Run
(Knative), Istio, and HTTP Load Balancing are enabled on the cluster
from the start, and cannot be installed or uninstalled.
## Existing GKE cluster
For information on adding an existing GKE cluster, see
info:To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Cluster access controls (RBAC or ABAC)
> Restricted service account for deployment was [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51716) in GitLab 11.5.
When creating a cluster in GitLab, you are asked if you would like to create either:
- A [Role-based access control (RBAC)](https://kubernetes.io/docs/reference/access-authn-authz/rbac/)
cluster, which is the GitLab default and recommended option.
- An [Attribute-based access control (ABAC)](https://kubernetes.io/docs/reference/access-authn-authz/abac/) cluster.
When GitLab creates the cluster,
a `gitlab` service account with `cluster-admin` privileges is created in the `default` namespace
to manage the newly created cluster.
Helm also creates additional service accounts and other resources for each
installed application. Consult the documentation of the Helm charts for each application
for details.
If you are [adding an existing Kubernetes cluster](add_remove_clusters.md#add-existing-cluster),
ensure the token of the account has administrator privileges for the cluster.
The resources created by GitLab differ depending on the type of cluster.
## Important notes
Note the following about access controls:
- Environment-specific resources are only created if your cluster is
[managed by GitLab](index.md#gitlab-managed-clusters).
- If your cluster was created before GitLab 12.2, it uses a single namespace for all project
environments.
## RBAC cluster resources
GitLab creates the following resources for RBAC clusters.
To view your project level Kubernetes clusters, navigate to **Infrastructure > Kubernetes clusters**
from your project. On this page, you can [add a new cluster](#adding-and-removing-clusters)
and view information about your existing clusters, such as:
- Nodes count.
- Rough estimates of memory and CPU usage.
## Setting up
### Supported cluster versions
## Supported cluster versions
GitLab is committed to support at least two production-ready Kubernetes minor
versions at any given time. We regularly review the versions we support, and
...
...
@@ -61,19 +57,31 @@ Kubernetes version to any supported version at any time:
Some GitLab features may support versions outside the range provided here.
NOTE:
[GKE Cluster creation](add_remove_clusters.md#create-new-cluster) by GitLab is currently not supported for Kubernetes 1.19+. For these versions you can create the cluster through GCP, then [Add existing cluster](add_remove_clusters.md#add-existing-cluster). See [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/331922) for more information.
## Add and remove clusters
You can create new or add existing clusters to GitLab:
### Adding and removing clusters
- On the project-level, to have a cluster dedicated to a project.
- On the [group level](../../group/clusters/index.md), to use the same cluster across multiple projects within your group.
- On the [instance level](../../instance/clusters/index.md), to use the same cluster across multiple groups and projects. **(FREE SELF)**
See [Adding and removing Kubernetes clusters](add_remove_clusters.md) for details on how
to:
To create new clusters, use one of the following methods:
- Create a cluster in Google Cloud Platform (GCP) or Amazon Elastic Kubernetes Service
(EKS) using the GitLab UI.
- Add an integration to an existing cluster from any Kubernetes platform.
-[Infrastructure as Code](../../infrastructure/index.md)(**recommended**).
You can also [add existing clusters](add_existing_cluster.md) to GitLab.
## View your clusters
To view your project-level Kubernetes clusters, to go **Infrastructure > Kubernetes clusters**
from your project. On this page, you can add a new cluster
and view information about your existing clusters, such as:
- Nodes count.
- Rough estimates of memory and CPU usage.
## Multiple Kubernetes clusters
> - Introduced in [GitLab Premium](https://about.gitlab.com/pricing/) 10.3
> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35094) to GitLab Free in 13.2.
...
...
@@ -85,7 +93,7 @@ Add another cluster, like you did the first time, and make sure to
[set an environment scope](#setting-the-environment-scope) that
differentiates the new cluster from the rest.
#### Setting the environment scope
### Setting the environment scope
When adding more than one Kubernetes cluster to your project, you need to differentiate
them with an environment scope. The environment scope associates clusters with [environments](../../../ci/environments/index.md) similar to how the
...
...
@@ -141,8 +149,8 @@ The results:
## Configuring your Kubernetes cluster
After [adding a Kubernetes cluster](add_remove_clusters.md) to GitLab, read this section that covers
important considerations for configuring Kubernetes clusters with GitLab.
Use the [GitLab Kubernetes Agent](../../clusters/agent/index.md) to safely
configure your clusters. Otherwise, there are [security implications](#security-implications).
### Security implications
...
...
@@ -172,6 +180,8 @@ for your deployment jobs to use. Otherwise, a namespace is created for you.
#### Important notes
Note the following with GitLab and clusters:
Be aware that manually managing resources that have been created by GitLab, like
namespaces and service accounts, can cause unexpected errors. If this occurs, try
[clearing the cluster cache](#clearing-the-cluster-cache).
...
...
@@ -253,6 +263,11 @@ This list provides a generic solution, and some GitLab-specific approaches:
If you see a trailing `%` on some Kubernetes versions, do not include it.
## Cluster integrations
See the available [cluster integrations](../../clusters/integrations.md)
to integrate third-party applications with your clusters through GitLab.
## Cluster management project
Attach a [Cluster management project](../../clusters/management_project.md)
...
...
@@ -261,14 +276,8 @@ installation, such as an Ingress controller.
## Auto DevOps
Auto DevOps automatically detects, builds, tests, deploys, and monitors your
applications.
To make full use of Auto DevOps (Auto Deploy, Auto Review Apps, and
Auto Monitoring) the Kubernetes project integration must be enabled. However,
Kubernetes clusters can be used without Auto DevOps.
[Read more about Auto DevOps](../../../topics/autodevops/index.md).
You can use [Auto DevOps](../../../topics/autodevops/index.md) to automatically
detect, build, test, deploy, and monitor your applications.
## Deploying to a Kubernetes cluster
...
...
@@ -309,7 +318,7 @@ GitLab CI/CD build environment to deployment jobs. Deployment jobs have
| Deployment Variable | Description |
|----------------------------|-------------|
| `KUBE_URL` | Equal to the API URL. |
| `KUBE_TOKEN` | The Kubernetes token of the [environment service account](add_remove_clusters.md#access-controls). Prior to GitLab 11.5, `KUBE_TOKEN` was the Kubernetes token of the main service account of the cluster integration. |
| `KUBE_TOKEN` | The Kubernetes token of the [environment service account](cluster_access.md). Prior to GitLab 11.5, `KUBE_TOKEN` was the Kubernetes token of the main service account of the cluster integration. |
| `KUBE_NAMESPACE` | The namespace associated with the project's deployment service account. In the format `<project_name>-<project_id>-<environment>`. For GitLab-managed clusters, a matching namespace is automatically created by GitLab in the cluster. If your cluster was created before GitLab 12.2, the default `KUBE_NAMESPACE` is set to `<project_name>-<project_id>`. |
| `KUBE_CA_PEM_FILE` | Path to a file containing PEM data. Only present if a custom CA bundle was specified. |
| `KUBE_CA_PEM` | (**deprecated**) Raw PEM data. Only if a custom CA bundle was specified. |