Skip to content

G
gitlab-ce
Commit bca153e5 authored by Achilleas Pipinellis's avatar Achilleas Pipinellis
Browse files
Options

Address feedback for AWS Runner autoscale article

parent 19821250
Hide whitespace changes
Inline Side-by-side
Showing with 142 additions and 118 deletions
doc/articles/index.md
View file @ bca153e5
...@@ -26,7 +26,7 @@ Build, test, and deploy the software you develop with [GitLab CI/CD](../ci/READM ...@@ -26,7 +26,7 @@ Build, test, and deploy the software you develop with [GitLab CI/CD](../ci/READM
| Article title | Category | Publishing date | | Article title | Category | Publishing date |
| :------------ | :------: | --------------: | | :------------ | :------: | --------------: |
| [Autoscaling GitLab Runners on AWS](runner_autoscale_aws/index.md) | Tutorial | 2017-11-10 | | [Autoscaling GitLab Runners on AWS](runner_autoscale_aws/index.md) | Admin guide | 2017-11-24 |
| [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md) | Tutorial | 2017-08-31 | | [How to test and deploy Laravel/PHP applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md) | Tutorial | 2017-08-31 |
| [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md) | Tutorial | 2017-08-15 | | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md) | Tutorial | 2017-08-15 |
| [Making CI Easier with GitLab](https://about.gitlab.com/2017/07/13/making-ci-easier-with-gitlab/) | Concepts | 2017-07-13 | | [Making CI Easier with GitLab](https://about.gitlab.com/2017/07/13/making-ci-easier-with-gitlab/) | Concepts | 2017-07-13 |
......
doc/articles/runner_autoscale_aws/index.md
View file @ bca153e5
--- ---
last_updated: 2017-11-10 last_updated: 2017-11-24
--- ---
> **[Article Type](../../development/writing_documentation.html#types-of-technical-articles):** Admin guide ||
> **Level:** intermediary ||
> **Author:** [Achilleas Pipinellis](https://gitlab.com/axil) ||
> **Publication date:** 2017/11/24
# Autoscaling GitLab Runner on AWS # Autoscaling GitLab Runner on AWS
One of the biggest advantages of GitLab Runner is its ability to automatically One of the biggest advantages of GitLab Runner is its ability to automatically
...@@ -16,38 +21,43 @@ In this tutorial, we'll explore how to properly configure a GitLab Runner in ...@@ -16,38 +21,43 @@ In this tutorial, we'll explore how to properly configure a GitLab Runner in
AWS that will serve as the bastion where it will spawn new Docker machines on AWS that will serve as the bastion where it will spawn new Docker machines on
demand. demand.
In addition, we'll make use of [Amazon's EC2 Spot instances](https://aws.amazon.com/ec2/spot/) In addition, we'll make use of [Amazon's EC2 Spot instances][spot] which will
which will greatly reduce the costs of the Runner instances while still using greatly reduce the costs of the Runner instances while still using quite
quite powerful autoscaling machines. powerful autoscaling machines.
## Prerequisites ## Prerequisites
NOTE: **Note:**
A familiarity with Amazon Web Services (AWS) is required as this is where most
of the configuration will take place.
Your GitLab instance is going to need to talk to the Runners over the network, Your GitLab instance is going to need to talk to the Runners over the network,
so consider the sensitivity of your projects and communication between nodes and that is something you need think about when configuring any AWS security
when moving forward. This has implications on the AWS security groups you'll groups or when setting up your DNS configuration.
need, your DNS configuration, and other networking factors.
For example, you can keep the EC2 resources segmented away from public traffic For example, you can keep the EC2 resources segmented away from public traffic
in a different VPC. Your environment is likely different, so consider what works in a different VPC to better strengthen your network security. Your environment
best for your situation. is likely different, so consider what works best for your situation.
### AWS security groups ### AWS security groups
Docker Machine will attempt to use a Docker Machine will attempt to use a
[default security group](https://docs.docker.com/machine/drivers/aws/#security-group) [default security group](https://docs.docker.com/machine/drivers/aws/#security-group)
with rules for port 2376, which is required for communication with the Docker with rules for port `2376`, which is required for communication with the Docker
daemon. Instead of relying on Docker, you can create a security group with the daemon. Instead of relying on Docker, you can create a security group with the
rules you need and provide that in the Runner options as we will see below. rules you need and provide that in the Runner options as we will
This way, you can customize it to your liking ahead of time based on your [see below](#the-runners-machine-section). This way, you can customize it to your
networking environment. liking ahead of time based on your networking environment.
### AWS credentials ### AWS credentials
You'll need an AWS Access Key tied to a user with permission to scale (EC2) and You'll need an [AWS Access Key](https://docs.aws.amazon.com/general/latest/gr/managing-aws-access-keys.html)
update the cache (via S3). Create a new user with policies for EC2 tied to a user with permission to scale (EC2) and update the cache (via S3).
(AmazonEC2FullAccess) and S3 (AmazonS3FullAccess). To be more secure, you can Create a new user with [policies](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-policies-for-amazon-ec2.html)
disable console login for that user. Grab the security credentials as we'll use for EC2 (AmazonEC2FullAccess) and S3 (AmazonS3FullAccess). To be more secure,
them later during the Runner configuration. you can disable console login for that user. Keep the tab open or copy paste the
security credentials in an editor as we'll use them later during the
[Runner configuration](#the-runners-machine-section).
## Prepare the bastion instance ## Prepare the bastion instance
...@@ -70,11 +80,7 @@ Install the prerequisites: ...@@ -70,11 +80,7 @@ Install the prerequisites:
Now that the Runner is installed, it's time to register it. Now that the Runner is installed, it's time to register it.
## Registering GitLab Runner ## Registering the GitLab Runner
TIP: **Tip:**
If you want every user in your instance to be able to use the autoscaled Runners,
register the Runner as a shared one.
Before configuring the GitLab Runner, you need to first register it, so that Before configuring the GitLab Runner, you need to first register it, so that
it connects with your GitLab instance: it connects with your GitLab instance:
...@@ -83,75 +89,29 @@ it connects with your GitLab instance: ...@@ -83,75 +89,29 @@ it connects with your GitLab instance:
1. [Register the Runner](https://docs.gitlab.com/runner/register/index.html#gnu-linux) 1. [Register the Runner](https://docs.gitlab.com/runner/register/index.html#gnu-linux)
1. When asked the executor type, enter `docker+machine` 1. When asked the executor type, enter `docker+machine`
You can now move on to the most important part, configuring GitLab Runner. You can now move on to the most important part, configuring the GitLab Runner.
## Configuring GitLab Runner to use the AWS machine driver TIP: **Tip:**
If you want every user in your instance to be able to use the autoscaled Runners,
register the Runner as a shared one.
## Configuring the GitLab Runner
Now that the Runner is registered, you need to edit its configuration file and Now that the Runner is registered, you need to edit its configuration file and
add the required options for the AWS machine driver. add the required options for the AWS machine driver.
Here's a full example of `/etc/gitlab-runner/config.toml`. Open it with your Let's first break it down to pieces.
editor and edit as you see fit:
```toml ### The global section
concurrent = 10
check_interval = 0
[[runners]]
name = "gitlab-aws-autoscaler"
url = "<URL of your GitLab instance>"
token = "<Runner's token>"
executor = "docker+machine"
limit = 20
[runners.docker]
image = "alpine"
privileged = true
disable_cache = true
[runners.cache]
Type = "s3"
ServerAddress = "s3.amazonaws.com"
AccessKey = "<your AWS Access Key ID>"
SecretKey = "<your AWS Secret Access Key>"
BucketName = "<the bucket where your cache should be kept>"
BucketLocation = "us-east-1"
Shared = true
[runners.machine]
IdleCount = 1
IdleTime = 1800
MaxBuilds = 100
OffPeakPeriods = [
"* * 0-9,18-23 * * mon-fri *",
"* * * * * sat,sun *"
]
OffPeakIdleCount = 0
OffPeakIdleTime = 1200
MachineDriver = "amazonec2"
MachineName = "gitlab-docker-machine-%s"
MachineOptions = [
"amazonec2-access-key=XXXX",
"amazonec2-secret-key=XXXX",
"amazonec2-region=us-east-1",
"amazonec2-vpc-id=vpc-xxxxx",
"amazonec2-subnet-id=subnet-xxxxx",
"amazonec2-use-private-address=true",
"amazonec2-tags=runner-manager-name,GitLab Runner autoscale,gitlab,true,gitlab-runner-autoscale,true",
"amazonec2-security-group=docker-machine-scaler",
"amazonec2-instance-type=m4.2xlarge",
]
```
Let's break it down to pieces.
### Global section
In the global section, you can define the limit of the jobs that can be run In the global section, you can define the limit of the jobs that can be run
concurrently across all Runners (`concurrent`). This heavily depends on your concurrently across all Runners (`concurrent`). This heavily depends on your
needs, like how many users your Runners will accommodate, how much time your needs, like how many users your Runners will accommodate, how much time your
builds take, etc. You can start with something low, and increase its value going builds take, etc. You can start with something low like `10`, and increase or
forward. decrease its value going forward.
The `check_interval` setting defines in seconds how often the Runner should The `check_interval` option defines how often the Runner should check GitLab
check GitLab for new jobs. for new jobs, in seconds.
Example: Example:
...@@ -163,14 +123,14 @@ check_interval = 0 ...@@ -163,14 +123,14 @@ check_interval = 0
[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section) [Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section)
about all the options you can use. about all the options you can use.
### `[[runners]]` section ### The `runners` section
From the `[[runners]]` section, the most important part is the `executor` which From the `[[runners]]` section, the most important part is the `executor` which
must be set to `docker+machine`. Most of those settings are taken care of when must be set to `docker+machine`. Most of those settings are taken care of when
you register the Runner for the first time. you register the Runner for the first time.
`limit` sets the maximum number of machines (running and idle) that this Runner `limit` sets the maximum number of machines (running and idle) that this Runner
will start. For more info check the [relationship between `limit`, `concurrent` will spawn. For more info check the [relationship between `limit`, `concurrent`
and `IdleCount`](https://docs.gitlab.com/runner/configuration/autoscale.html#how-concurrent-limit-and-idlecount-generate-the-upper-limit-of-running-machines). and `IdleCount`](https://docs.gitlab.com/runner/configuration/autoscale.html#how-concurrent-limit-and-idlecount-generate-the-upper-limit-of-running-machines).
Example: Example:
...@@ -187,16 +147,17 @@ Example: ...@@ -187,16 +147,17 @@ Example:
[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section) [Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section)
about all the options you can use under `[[runners]]`. about all the options you can use under `[[runners]]`.
### `[runners.docker]` section ### The `runners.docker` section
In the `[runners.docker]` section you can define the default Docker image to In the `[runners.docker]` section you can define the default Docker image to
be used by the child Runners if it's not defined in [`.gitlab-ci.yml`](../../ci/yaml/README.md). be used by the child Runners if it's not defined in [`.gitlab-ci.yml`](../../ci/yaml/README.md).
Using `privileged = true`, all Runners will be able to run Docker in Docker By using `privileged = true`, all Runners will be able to run
[Docker in Docker](../../ci/docker/using_docker_build.md#use-docker-in-docker-executor)
which is useful if you plan to build your own Docker images via GitLab CI/CD. which is useful if you plan to build your own Docker images via GitLab CI/CD.
Next, we use `disable_cache = true` to disable the Docker executor's inner Next, we use `disable_cache = true` to disable the Docker executor's inner
cache mechanism since we will use the distributed cache mode as described cache mechanism since we will use the distributed cache mode as described
below. in the following section.
Example: Example:
...@@ -210,13 +171,13 @@ Example: ...@@ -210,13 +171,13 @@ Example:
[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-docker-section) [Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-docker-section)
about all the options you can use under `[runners.docker]`. about all the options you can use under `[runners.docker]`.
### `[runners.cache]` section ### The `runners.cache` section
To speed up your jobs, GitLab Runner provides a cache mechanism where selected To speed up your jobs, GitLab Runner provides a cache mechanism where selected
directories and/or files are saved and shared between subsequent jobs. directories and/or files are saved and shared between subsequent jobs.
While not required for this setup, it is recommended to use the distributed cache While not required for this setup, it is recommended to use the distributed cache
mechanism that GitLab Runner provides. Since new instances will be created on mechanism that GitLab Runner provides. Since new instances will be created on
demand, it is essential to have a common place where cache is stored. demand, it is essential to have a common place where the cache is stored.
In the following example, we use Amazon S3: In the following example, we use Amazon S3:
...@@ -231,13 +192,13 @@ In the following example, we use Amazon S3: ...@@ -231,13 +192,13 @@ In the following example, we use Amazon S3:
Shared = true Shared = true
``` ```
Here's some more info to get you started: Here's some more info to further explore the cache mechanism:
- [The `[runners.cache]` section reference](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-cache-section) - [Reference for `runners.cache`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-cache-section)
- [Deploying and using a cache server for GitLab Runner](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) - [Deploying and using a cache server for GitLab Runner](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching)
- [How cache works](../../ci/yaml/README.md#cache) - [How cache works](../../ci/yaml/README.md#cache)
### `[runners.machine]` section ### The `runners.machine` section
This is the most important part of the configuration and it's the one that This is the most important part of the configuration and it's the one that
tells GitLab Runner how and when to spawn new or remove old Docker Machine tells GitLab Runner how and when to spawn new or remove old Docker Machine
...@@ -246,10 +207,10 @@ instances. ...@@ -246,10 +207,10 @@ instances.
We will focus on the AWS machine options, for the rest of the settings read We will focus on the AWS machine options, for the rest of the settings read
about the: about the:
- [autoscaling algorithm and the parameters it's based on](https://docs.gitlab.com/runner/configuration/autoscale.html#autoscaling-algorithm-and-parameters) - depends on the needs of your organization - [Autoscaling algorithm and the parameters it's based on](https://docs.gitlab.com/runner/configuration/autoscale.html#autoscaling-algorithm-and-parameters) - depends on the needs of your organization
- [off peak time configuration](https://docs.gitlab.com/runner/configuration/autoscale.html#off-peak-time-mode-configuration) - useful when there are regular time periods in your organization when no work is done, for example weekends - [Off peak time configuration](https://docs.gitlab.com/runner/configuration/autoscale.html#off-peak-time-mode-configuration) - useful when there are regular time periods in your organization when no work is done, for example weekends
Example: Here's an example of the `runners.machine` section:
```toml ```toml
[runners.machine] [runners.machine]
...@@ -267,11 +228,11 @@ Example: ...@@ -267,11 +228,11 @@ Example:
MachineOptions = [ MachineOptions = [
"amazonec2-access-key=XXXX", "amazonec2-access-key=XXXX",
"amazonec2-secret-key=XXXX", "amazonec2-secret-key=XXXX",
"amazonec2-region=us-east-1", "amazonec2-region=us-central-1",
"amazonec2-vpc-id=vpc-xxxxx", "amazonec2-vpc-id=vpc-xxxxx",
"amazonec2-subnet-id=subnet-xxxxx", "amazonec2-subnet-id=subnet-xxxxx",
"amazonec2-use-private-address=true", "amazonec2-use-private-address=true",
"amazonec2-tags=runner-manager-name,GitLab Runner autoscale,gitlab,true,gitlab-runner-autoscale,true", "amazonec2-tags=runner-manager-name,gitlab-aws-autoscaler,gitlab,true,gitlab-runner-autoscale,true",
"amazonec2-security-group=docker-machine-scaler", "amazonec2-security-group=docker-machine-scaler",
"amazonec2-instance-type=m4.2xlarge", "amazonec2-instance-type=m4.2xlarge",
] ]
...@@ -282,17 +243,19 @@ standard prefix followed by `%s` (required) that is replaced by the ID of the ...@@ -282,17 +243,19 @@ standard prefix followed by `%s` (required) that is replaced by the ID of the
child Runner: `gitlab-docker-machine-%s`. child Runner: `gitlab-docker-machine-%s`.
Now, depending on your AWS infrastructure, there are many options you can set up Now, depending on your AWS infrastructure, there are many options you can set up
under `MachineOptions`. Let's see the most common ones: under `MachineOptions`. Below you can see the most common ones.
- `amazonec2-access-key=XXXX` - The AWS access key of the user that has permissions to create EC2 instances, see [AWS credentials](#aws-credentials). | Machine option | Description |
- `amazonec2-secret-key=XXXX` - The AWS secret key of the user that has permissions to create EC2 instances, see [AWS credentials](#aws-credentials). | -------------- | ----------- |
- `amazonec2-region=eu-central-1` - The region to use when launching the instance. You can omit this entirely and the default `us-east-1` will be used. | `amazonec2-access-key=XXXX` | The AWS access key of the user that has permissions to create EC2 instances, see [AWS credentials](#aws-credentials). |
- `amazonec2-vpc-id=vpc-xxxxx` - Your VPC ID to launch the instance in, read more in [Docker docs about the VPC ID](https://docs.docker.com/machine/drivers/aws/#vpc-id). | `amazonec2-secret-key=XXXX` | The AWS secret key of the user that has permissions to create EC2 instances, see [AWS credentials](#aws-credentials). |
- `amazonec2-subnet-id=subnet-xxxx` - AWS VPC subnet ID. | `amazonec2-region=eu-central-1` | The region to use when launching the instance. You can omit this entirely and the default `us-east-1` will be used. |
- `amazonec2-use-private-address=true` - Use the private IP address for docker-machine, but still create a public IP address. Useful to keep the traffic internal and avoid extra costs. | `amazonec2-vpc-id=vpc-xxxxx` | Your [VPC ID](https://docs.docker.com/machine/drivers/aws/#vpc-id) to launch the instance in. |
- `amazonec2-tags=runner-manager-name,GitLab Runner autoscale,gitlab,true,gitlab-runner-autoscale,true` - AWS extra tag key-value pairs, useful to identify the instances on the AWS console. Read more about [using tags in AWS](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html). | `amazonec2-subnet-id=subnet-xxxx` | The AWS VPC subnet ID. |
- `amazonec2-security-group=docker-machine-scaler` - AWS VPC security group name, see [AWS security groups](#aws-security-groups). | `amazonec2-use-private-address=true` | Use the private IP address of Docker Machines, but still create a public IP address. Useful to keep the traffic internal and avoid extra costs.|
- `amazonec2-instance-type=m4.2xlarge` - The instance type that the child Runners will run on. | `amazonec2-tags=runner-manager-name,gitlab-aws-autoscaler,gitlab,true,gitlab-runner-autoscale,true` | AWS extra tag key-value pairs, useful to identify the instances on the AWS console. The "Name" tag is set to the machine name by default. We set the "runner-manager-name" to match the Runner name set in `[[runners]]`, so that we can filter all the EC2 instances created by a specific manager setup. Read more about [using tags in AWS](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html). |
| `amazonec2-security-group=docker-machine-scaler` | AWS VPC security group name, see [AWS security groups](#aws-security-groups). |
| `amazonec2-instance-type=m4.2xlarge` | The instance type that the child Runners will run on. |
TIP: **Tip:** TIP: **Tip:**
Under `MachineOptions` you can add anything that the [AWS Docker Machine driver Under `MachineOptions` you can add anything that the [AWS Docker Machine driver
...@@ -306,17 +269,68 @@ different AMI ID by setting `amazonec2-ami`. ...@@ -306,17 +269,68 @@ different AMI ID by setting `amazonec2-ami`.
NOTE: **Note:** NOTE: **Note:**
If you specify `amazonec2-private-address-only=true` as one of the machine If you specify `amazonec2-private-address-only=true` as one of the machine
options, your EC2 instance won't get assigned a public IP. This is fine if your options, your EC2 instance won't get assigned a public IP. This is ok if your
VPC is configured correctly with an Internet Gateway (IGW) and routing is fine, VPC is configured correctly with an Internet Gateway (IGW) and routing is fine,
but it’s something to consider if you've got a more exotic configuration. Read but it’s something to consider if you've got a more complex configuration. Read
more in [Docker docs about VPC connectivity](https://docs.docker.com/machine/drivers/aws/#vpc-connectivity). more in [Docker docs about VPC connectivity](https://docs.docker.com/machine/drivers/aws/#vpc-connectivity).
[Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-machine-section) [Read more](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-machine-section)
about all the options you can use under `[runners.machine]`. about all the options you can use under `[runners.machine]`.
### Getting it all together
Here's the full example of `/etc/gitlab-runner/config.toml`:
```toml
concurrent = 10
check_interval = 0
[[runners]]
name = "gitlab-aws-autoscaler"
url = "<URL of your GitLab instance>"
token = "<Runner's token>"
executor = "docker+machine"
limit = 20
[runners.docker]
image = "alpine"
privileged = true
disable_cache = true
[runners.cache]
Type = "s3"
ServerAddress = "s3.amazonaws.com"
AccessKey = "<your AWS Access Key ID>"
SecretKey = "<your AWS Secret Access Key>"
BucketName = "<the bucket where your cache should be kept>"
BucketLocation = "us-east-1"
Shared = true
[runners.machine]
IdleCount = 1
IdleTime = 1800
MaxBuilds = 100
OffPeakPeriods = [
"* * 0-9,18-23 * * mon-fri *",
"* * * * * sat,sun *"
]
OffPeakIdleCount = 0
OffPeakIdleTime = 1200
MachineDriver = "amazonec2"
MachineName = "gitlab-docker-machine-%s"
MachineOptions = [
"amazonec2-access-key=XXXX",
"amazonec2-secret-key=XXXX",
"amazonec2-region=us-central-1",
"amazonec2-vpc-id=vpc-xxxxx",
"amazonec2-subnet-id=subnet-xxxxx",
"amazonec2-use-private-address=true",
"amazonec2-tags=runner-manager-name,gitlab-aws-autoscaler,gitlab,true,gitlab-runner-autoscale,true",
"amazonec2-security-group=docker-machine-scaler",
"amazonec2-instance-type=m4.2xlarge",
]
```
## Cutting down costs with Amazon EC2 Spot instances ## Cutting down costs with Amazon EC2 Spot instances
As described by Amazon: As [described by][spot] Amazon:
> >
Amazon EC2 Spot instances allow you to bid on spare Amazon EC2 computing capacity. Amazon EC2 Spot instances allow you to bid on spare Amazon EC2 computing capacity.
...@@ -325,7 +339,7 @@ pricing, you can significantly reduce the cost of running your applications, ...@@ -325,7 +339,7 @@ pricing, you can significantly reduce the cost of running your applications,
grow your application’s compute capacity and throughput for the same budget, grow your application’s compute capacity and throughput for the same budget,
and enable new types of cloud computing applications. and enable new types of cloud computing applications.
In addition to the [`[runners.machine]`](#runners-machine-section) options In addition to the [`runners.machine`](#the-runners-machine-section) options
you picked above, in `/etc/gitlab-runner/config.toml` under the `MachineOptions` you picked above, in `/etc/gitlab-runner/config.toml` under the `MachineOptions`
section, add the following: section, add the following:
...@@ -339,8 +353,8 @@ section, add the following: ...@@ -339,8 +353,8 @@ section, add the following:
With this configuration, Docker Machines are created on Spot instances with a With this configuration, Docker Machines are created on Spot instances with a
maximum bid price of $0.03 per hour and the duration of the Spot instance is maximum bid price of $0.03 per hour and the duration of the Spot instance is
capped at 60 minutes. Be sure to check on the current pricing based on the capped at 60 minutes. The `0.03` number mentioned above is just an example, so
region you picked. be sure to check on the current pricing based on the region you picked.
To learn more about Amazon EC2 Spot instances, visit the following links: To learn more about Amazon EC2 Spot instances, visit the following links:
...@@ -370,17 +384,27 @@ docker-machine ls -q --filter state=Error --format "{{.NAME}}" ...@@ -370,17 +384,27 @@ docker-machine ls -q --filter state=Error --format "{{.NAME}}"
``` ```
NOTE: **Note:** NOTE: **Note:**
In [issue 2771](https://gitlab.com/gitlab-org/gitlab-runner/issues/2771) There are some issues regarding making GitLab Runner gracefully handle Spot
there is a discussion to make GitLab Runner gracefully handle Spot price changes. price changes, and there are reports of `docker-machine` attempting to
continually remove a Docker Machine. GitLab has provided patches for both cases
in the upstream project. For more information, see issues
[#2771](https://gitlab.com/gitlab-org/gitlab-runner/issues/2771) and
[#2772](https://gitlab.com/gitlab-org/gitlab-runner/issues/2772).
## Conclusion ## Conclusion
In this guide we learned how to install and configure a GitLab Runner in
autoscale mode on AWS.
Using the autoscale feature of GitLab Runner can save you both time and money. Using the autoscale feature of GitLab Runner can save you both time and money.
Using the Spot instances that AWS provides can save you even more, but you must Using the Spot instances that AWS provides can save you even more, but you must
be aware of the implications. As long as your bid is high enough, there won't be aware of the implications. As long as your bid is high enough, there shouldn't
be an issue. be an issue.
You can read the following use cases from which this tutorial was influenced: You can read the following use cases from which this tutorial was (heavily)
influenced:
- [HumanGeo - Scaling GitLab CI](http://blog.thehumangeo.com/gitlab-autoscale-runners.html) - [HumanGeo - Scaling GitLab CI](http://blog.thehumangeo.com/gitlab-autoscale-runners.html)
- [subtrakt Health - Autoscale GitLab CI Runners and save 90% on EC2 costs](https://substrakthealth.com/news/gitlab-ci-cost-savings/) - [subtrakt Health - Autoscale GitLab CI Runners and save 90% on EC2 costs](https://substrakthealth.com/news/gitlab-ci-cost-savings/)
[spot]: https://aws.amazon.com/ec2/spot/
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7