@@ -6,8 +6,8 @@ The requirements are listed [on the index page](index.md#requirements-for-runnin
...
@@ -6,8 +6,8 @@ The requirements are listed [on the index page](index.md#requirements-for-runnin
## How does Geo know which projects to sync?
## How does Geo know which projects to sync?
On each **secondary** node, there is a read-only replicated copy of the GitLab database.
On each **secondary** node, there is a read-only replicated copy of the GitLab database.
A **secondary** node also has a tracking database where it stores which projects have been synced.
A **secondary** node also has a tracking database where it stores which projects have been synced.
Geo compares the two databases to find projects that are not yet tracked.
Geo compares the two databases to find projects that are not yet tracked.
At the start, this tracking database is empty, so Geo will start trying to update from every project that it can see in the GitLab database.
At the start, this tracking database is empty, so Geo will start trying to update from every project that it can see in the GitLab database.
...
@@ -15,19 +15,19 @@ At the start, this tracking database is empty, so Geo will start trying to updat
...
@@ -15,19 +15,19 @@ At the start, this tracking database is empty, so Geo will start trying to updat
For each project to sync:
For each project to sync:
1. Geo will issue a `git fetch geo --mirror` to get the latest information from the **primary** node.
1. Geo will issue a `git fetch geo --mirror` to get the latest information from the **primary** node.
If there are no changes, the sync will be fast and end quickly. Otherwise, it will pull the latest commits.
If there are no changes, the sync will be fast and end quickly. Otherwise, it will pull the latest commits.
1. The **secondary** node will update the tracking database to store the fact that it has synced projects A, B, C, etc.
1. The **secondary** node will update the tracking database to store the fact that it has synced projects A, B, C, etc.
1. Repeat until all projects are synced.
1. Repeat until all projects are synced.
When someone pushes a commit to the **primary** node, it generates an event in the GitLab database that the repository has changed.
When someone pushes a commit to the **primary** node, it generates an event in the GitLab database that the repository has changed.
The **secondary** node sees this event, marks the project in question as dirty, and schedules the project to be resynced.
The **secondary** node sees this event, marks the project in question as dirty, and schedules the project to be resynced.
To ensure that problems with pipelines (for example, syncs failing too many times or jobs being lost) don't permanently stop projects syncing, Geo also periodically checks the tracking database for projects that are marked as dirty. This check happens when
To ensure that problems with pipelines (for example, syncs failing too many times or jobs being lost) don't permanently stop projects syncing, Geo also periodically checks the tracking database for projects that are marked as dirty. This check happens when
the number of concurrent syncs falls below `repos_max_capacity` and there are no new projects waiting to be synced.
the number of concurrent syncs falls below `repos_max_capacity` and there are no new projects waiting to be synced.
Geo also has a checksum feature which runs a SHA256 sum across all the Git references to the SHA values.
Geo also has a checksum feature which runs a SHA256 sum across all the Git references to the SHA values.
If the refs don't match between the **primary** node and the **secondary** node, then the **secondary** node will mark that project as dirty and try to resync it.
If the refs don't match between the **primary** node and the **secondary** node, then the **secondary** node will mark that project as dirty and try to resync it.
So even if we have an outdated tracking database, the validation should activate and find discrepancies in the repository state and resync.
So even if we have an outdated tracking database, the validation should activate and find discrepancies in the repository state and resync.
## Can I use Geo in a disaster recovery situation?
## Can I use Geo in a disaster recovery situation?
@@ -909,11 +909,12 @@ import bundle from 'ee_else_ce/protected_branches/protected_branches_bundle.js';
...
@@ -909,11 +909,12 @@ import bundle from 'ee_else_ce/protected_branches/protected_branches_bundle.js';
See the frontend guide [performance section](fe_guide/performance.md) for
See the frontend guide [performance section](fe_guide/performance.md) for
information on managing page-specific javascript within EE.
information on managing page-specific javascript within EE.
## Vue code in `assets/javascript`
## Vue code in `assets/javascript`
### script tag
### script tag
#### Child Component only used in EE
#### Child Component only used in EE
To separate Vue template differences we should [async import the components](https://vuejs.org/v2/guide/components-dynamic-async.html#Async-Components).
To separate Vue template differences we should [async import the components](https://vuejs.org/v2/guide/components-dynamic-async.html#Async-Components).
Doing this allows for us to load the correct component in EE whilst in CE
Doing this allows for us to load the correct component in EE whilst in CE
...
@@ -937,10 +938,12 @@ export default {
...
@@ -937,10 +938,12 @@ export default {
```
```
#### For JS code that is EE only, like props, computed properties, methods, etc, we will keep the current approach
#### For JS code that is EE only, like props, computed properties, methods, etc, we will keep the current approach
- Since we [can't async load a mixin](https://github.com/vuejs/vue-loader/issues/418#issuecomment-254032223) we will use the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) alias we already have for webpack.
- Since we [can't async load a mixin](https://github.com/vuejs/vue-loader/issues/418#issuecomment-254032223) we will use the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) alias we already have for webpack.
- This means all the EE specific props, computed properties, methods, etc that are EE only should be in a mixin in the `ee/` folder and we need to create a CE counterpart of the mixin
- This means all the EE specific props, computed properties, methods, etc that are EE only should be in a mixin in the `ee/` folder and we need to create a CE counterpart of the mixin
##### Example:
##### Example:
```javascript
```javascript
importmixinfrom'ee_else_ce/path/mixin';
importmixinfrom'ee_else_ce/path/mixin';
...
@@ -955,6 +958,7 @@ import mixin from 'ee_else_ce/path/mixin';
...
@@ -955,6 +958,7 @@ import mixin from 'ee_else_ce/path/mixin';
- You can see an MR with an example [here](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/9762)
- You can see an MR with an example [here](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/9762)
#### `template` tag
#### `template` tag
***EE Child components**
***EE Child components**
- Since we are using the async loading to check which component to load, we'd still use the component's name, check [this example](#child-component-only-used-in-ee).
- Since we are using the async loading to check which component to load, we'd still use the component's name, check [this example](#child-component-only-used-in-ee).
...
@@ -962,11 +966,12 @@ import mixin from 'ee_else_ce/path/mixin';
...
@@ -962,11 +966,12 @@ import mixin from 'ee_else_ce/path/mixin';
- For the templates that have extra HTML in EE we should move it into a new component and use the `ee_else_ce` dynamic import
- For the templates that have extra HTML in EE we should move it into a new component and use the `ee_else_ce` dynamic import
### Non Vue Files
### Non Vue Files
For regular JS files, the approach is similar.
For regular JS files, the approach is similar.
1. We will keep using the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) helper, this means that EE only code should be inside the `ee/` folder.
1. We will keep using the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) helper, this means that EE only code should be inside the `ee/` folder.
1. An EE file should be created with the EE only code, and it should extend the CE counterpart.
1. An EE file should be created with the EE only code, and it should extend the CE counterpart.
1. For code inside functions that can't be extended, the code should be moved into a new file and we should use `ee_else_ce` helper:
1. For code inside functions that can't be extended, the code should be moved into a new file and we should use `ee_else_ce` helper:
##### Example:
##### Example:
...
@@ -996,6 +1001,7 @@ to isolate such ruleset from rest of CE rules (along with adding comment describ
...
@@ -996,6 +1001,7 @@ to isolate such ruleset from rest of CE rules (along with adding comment describ
to avoid conflicts during CE to EE merge.
to avoid conflicts during CE to EE merge.
#### Bad
#### Bad
```scss
```scss
.section-body{
.section-body{
.section-title{
.section-title{
...
@@ -1011,6 +1017,7 @@ to avoid conflicts during CE to EE merge.
...
@@ -1011,6 +1017,7 @@ to avoid conflicts during CE to EE merge.
@@ -64,20 +64,25 @@ All indexing after the initial one is done via `ElasticIndexerWorker` (sidekiq j
...
@@ -64,20 +64,25 @@ All indexing after the initial one is done via `ElasticIndexerWorker` (sidekiq j
Search queries are generated by the concerns found in [ee/app/models/concerns/elastic](https://gitlab.com/gitlab-org/gitlab-ee/tree/master/ee/app/models/concerns/elastic). These concerns are also in charge of access control, and have been a historic source of security bugs so please pay close attention to them!
Search queries are generated by the concerns found in [ee/app/models/concerns/elastic](https://gitlab.com/gitlab-org/gitlab-ee/tree/master/ee/app/models/concerns/elastic). These concerns are also in charge of access control, and have been a historic source of security bugs so please pay close attention to them!
## Existing Analyzers/Tokenizers/Filters
## Existing Analyzers/Tokenizers/Filters
These are all defined in https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/elasticsearch/git/model.rb
These are all defined in https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/elasticsearch/git/model.rb
### Analyzers
### Analyzers
#### `path_analyzer`
#### `path_analyzer`
Used when indexing blobs' paths. Uses the `path_tokenizer` and the `lowercase` and `asciifolding` filters.
Used when indexing blobs' paths. Uses the `path_tokenizer` and the `lowercase` and `asciifolding` filters.
Please see the `path_tokenizer` explanation below for an example.
Please see the `path_tokenizer` explanation below for an example.
#### `sha_analyzer`
#### `sha_analyzer`
Used in blobs and commits. Uses the `sha_tokenizer` and the `lowercase` and `asciifolding` filters.
Used in blobs and commits. Uses the `sha_tokenizer` and the `lowercase` and `asciifolding` filters.
Please see the `sha_tokenizer` explanation later below for an example.
Please see the `sha_tokenizer` explanation later below for an example.
#### `code_analyzer`
#### `code_analyzer`
Used when indexing a blob's filename and content. Uses the `whitespace` tokenizer and the filters: `code`, `edgeNGram_filter`, `lowercase`, and `asciifolding`
Used when indexing a blob's filename and content. Uses the `whitespace` tokenizer and the filters: `code`, `edgeNGram_filter`, `lowercase`, and `asciifolding`
The `whitespace` tokenizer was selected in order to have more control over how tokens are split. For example the string `Foo::bar(4)` needs to generate tokens like `Foo` and `bar(4)` in order to be properly searched.
The `whitespace` tokenizer was selected in order to have more control over how tokens are split. For example the string `Foo::bar(4)` needs to generate tokens like `Foo` and `bar(4)` in order to be properly searched.
...
@@ -85,15 +90,19 @@ The `whitespace` tokenizer was selected in order to have more control over how t
...
@@ -85,15 +90,19 @@ The `whitespace` tokenizer was selected in order to have more control over how t
Please see the `code` filter for an explanation on how tokens are split.
Please see the `code` filter for an explanation on how tokens are split.
#### `code_search_analyzer`
#### `code_search_analyzer`
Not directly used for indexing, but rather used to transform a search input. Uses the `whitespace` tokenizer and the `lowercase` and `asciifolding` filters.
Not directly used for indexing, but rather used to transform a search input. Uses the `whitespace` tokenizer and the `lowercase` and `asciifolding` filters.
### Tokenizers
### Tokenizers
#### `sha_tokenizer`
#### `sha_tokenizer`
This is a custom tokenizer that uses the [`edgeNGram` tokenizer](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-edgengram-tokenizer.html) to allow SHAs to be searcheable by any sub-set of it (minimum of 5 chars).
This is a custom tokenizer that uses the [`edgeNGram` tokenizer](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-edgengram-tokenizer.html) to allow SHAs to be searcheable by any sub-set of it (minimum of 5 chars).
example:
Example:
`240c29dc7e` becomes:
`240c29dc7e` becomes:
-`240c2`
-`240c2`
-`240c29`
-`240c29`
-`240c29d`
-`240c29d`
...
@@ -102,21 +111,26 @@ example:
...
@@ -102,21 +111,26 @@ example:
-`240c29dc7e`
-`240c29dc7e`
#### `path_tokenizer`
#### `path_tokenizer`
This is a custom tokenizer that uses the [`path_hierarchy` tokenizer](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-pathhierarchy-tokenizer.html) with `reverse: true` in order to allow searches to find paths no matter how much or how little of the path is given as input.
This is a custom tokenizer that uses the [`path_hierarchy` tokenizer](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-pathhierarchy-tokenizer.html) with `reverse: true` in order to allow searches to find paths no matter how much or how little of the path is given as input.
example:
Example:
`'/some/path/application.js'` becomes:
`'/some/path/application.js'` becomes:
-`'/some/path/application.js'`
-`'/some/path/application.js'`
-`'some/path/application.js'`
-`'some/path/application.js'`
-`'path/application.js'`
-`'path/application.js'`
-`'application.js'`
-`'application.js'`
### Filters
### Filters
#### `code`
#### `code`
Uses a [Pattern Capture token filter](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-pattern-capture-tokenfilter.html) to split tokens into more easily searched versions of themselves.
Uses a [Pattern Capture token filter](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-pattern-capture-tokenfilter.html) to split tokens into more easily searched versions of themselves.
Patterns:
Patterns:
-`"(\\p{Ll}+|\\p{Lu}\\p{Ll}+|\\p{Lu}+)"`: captures CamelCased and lowedCameCased strings as separate tokens
-`"(\\p{Ll}+|\\p{Lu}\\p{Ll}+|\\p{Lu}+)"`: captures CamelCased and lowedCameCased strings as separate tokens
-`'\/?([^\/]+)(?=\/|\b)'`: separate path terms `like/this/one`
-`'\/?([^\/]+)(?=\/|\b)'`: separate path terms `like/this/one`
#### `edgeNGram_filter`
#### `edgeNGram_filter`
Uses an [Edge NGram token filter](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-edgengram-tokenfilter.html) to allow inputs with only parts of a token to find the token. For example it would turn `glasses` into permutations starting with `gl` and ending with `glasses`, which would allow a search for "`glass`" to find the original token `glasses`
Uses an [Edge NGram token filter](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-edgengram-tokenfilter.html) to allow inputs with only parts of a token to find the token. For example it would turn `glasses` into permutations starting with `gl` and ending with `glasses`, which would allow a search for "`glass`" to find the original token `glasses`
## Gotchas
## Gotchas
...
@@ -140,13 +155,13 @@ Uses an [Edge NGram token filter](https://www.elastic.co/guide/en/elasticsearch/
...
@@ -140,13 +155,13 @@ Uses an [Edge NGram token filter](https://www.elastic.co/guide/en/elasticsearch/
In our case, `qa-login-field`, `qa-password-field` and `qa-sign-in-button`
In our case, `qa-login-field`, `qa-password-field` and `qa-sign-in-button`
**app/views/my/view.html.haml**
**app/views/my/view.html.haml**
```haml
```haml
=f.text_field:login,class: "form-control top qa-login-field",autofocus: "autofocus",autocapitalize: "off",autocorrect: "off",required: true,title: "This field is required."
=f.text_field:login,class: "form-control top qa-login-field",autofocus: "autofocus",autocapitalize: "off",autocorrect: "off",required: true,title: "This field is required."
...
@@ -146,7 +146,7 @@ Things to note:
...
@@ -146,7 +146,7 @@ Things to note:
- The CSS class must be `kebab-cased` (separated with hyphens "`-`")
- The CSS class must be `kebab-cased` (separated with hyphens "`-`")
- If the element appears on the page unconditionally, add `required: true` to the element. See
- If the element appears on the page unconditionally, add `required: true` to the element. See
[Dynamic element validation](dynamic_element_validation.md)
[Dynamic element validation](dynamic_element_validation.md)
@@ -25,22 +25,22 @@ and [Migrating from Jenkins to GitLab](https://www.youtube.com/watch?v=RlEVGOpYF
...
@@ -25,22 +25,22 @@ and [Migrating from Jenkins to GitLab](https://www.youtube.com/watch?v=RlEVGOpYF
## Use cases
## Use cases
- Suppose you are new to GitLab, and want to keep using Jenkins until you prepare
- Suppose you are new to GitLab, and want to keep using Jenkins until you prepare
your projects to build with [GitLab CI/CD](../ci/README.md). You set up the
your projects to build with [GitLab CI/CD](../ci/README.md). You set up the
integration between GitLab and Jenkins, then you migrate to GitLab CI later. While
integration between GitLab and Jenkins, then you migrate to GitLab CI later. While
you organize yourself and your team to onboard GitLab, you keep your pipelines
you organize yourself and your team to onboard GitLab, you keep your pipelines
running with Jenkins, but view the results in your project's repository in GitLab.
running with Jenkins, but view the results in your project's repository in GitLab.
- Your team uses [Jenkins Plugins](https://plugins.jenkins.io/) for other proceedings,
- Your team uses [Jenkins Plugins](https://plugins.jenkins.io/) for other proceedings,
therefore, you opt for keep using Jenkins to build your apps. Show the results of your
therefore, you opt for keep using Jenkins to build your apps. Show the results of your
pipelines directly in GitLab.
pipelines directly in GitLab.
For a real use case, read the blog post [Continuous integration: From Jenkins to GitLab using Docker](https://about.gitlab.com/2017/07/27/docker-my-precious/).
For a real use case, read the blog post [Continuous integration: From Jenkins to GitLab using Docker](https://about.gitlab.com/2017/07/27/docker-my-precious/).
@@ -14,7 +14,7 @@ Learn how GitLab helps you in the stages of the DevOps lifecycle by learning mor
...
@@ -14,7 +14,7 @@ Learn how GitLab helps you in the stages of the DevOps lifecycle by learning mor
### Self-managed: Install GitLab
### Self-managed: Install GitLab
Take a look at [installing GitLab](https://about.gitlab.com/install/) and our [administrator documentation](../administration/index.md). Then, follow the instructions below under [Your subscription](#your-subscription) to apply your license file.
Take a look at [installing GitLab](https://about.gitlab.com/install/) and our [administrator documentation](../administration/index.md). Then, follow the instructions below under [Your subscription](#your-subscription) to apply your license file.
### GitLab.com: Create a user and group
### GitLab.com: Create a user and group
...
@@ -74,11 +74,11 @@ Please note that you need to be a group owner to associate a group to your subsc
...
@@ -74,11 +74,11 @@ Please note that you need to be a group owner to associate a group to your subsc
To see the status of your GitLab.com subscription, you can click on the Billings
To see the status of your GitLab.com subscription, you can click on the Billings
section of the relevant namespace:
section of the relevant namespace:
* For individuals, this is located at https://gitlab.com/profile/billings under
- For individuals, this is located at https://gitlab.com/profile/billings under
in your Settings,
in your Settings,
* For groups, this is located under the group's Settings dropdown, under Billing.
- For groups, this is located under the group's Settings dropdown, under Billing.
For groups, you can see details of your subscription - including your current
For groups, you can see details of your subscription - including your current
plan - in the included table:
plan - in the included table:
![Billing table](billing_table.png)
![Billing table](billing_table.png)
...
@@ -86,11 +86,11 @@ plan - in the included table:
...
@@ -86,11 +86,11 @@ plan - in the included table:
| Field | Description |
| Field | Description |
| ------ | ------ |
| ------ | ------ |
| Seats in subscription | If this is a paid plan, this represents the number of seats you've paid to support in your group. |
| Seats in subscription | If this is a paid plan, this represents the number of seats you've paid to support in your group. |
| Seats currently in use | The number of active seats currently in use. |
| Seats currently in use | The number of active seats currently in use. |
| Max seats used | The highest number of seats you've used. If this exceeds the seats in subscription, you may owe an additional fee for the additional users. |
| Max seats used | The highest number of seats you've used. If this exceeds the seats in subscription, you may owe an additional fee for the additional users. |
| Seats owed | If your max seats used exceeds the seats in your subscription, you'll owe an additional fee for the users you've added. |
| Seats owed | If your max seats used exceeds the seats in your subscription, you'll owe an additional fee for the users you've added. |
| Subscription start date | The date your subscription started. If this is for a Free plan, this is the date you transitioned off your group's paid plan. |
| Subscription start date | The date your subscription started. If this is for a Free plan, this is the date you transitioned off your group's paid plan. |
| Subscription end date | The date your current subscription will end. This does not apply to Free plans. |
| Subscription end date | The date your current subscription will end. This does not apply to Free plans. |
@@ -41,12 +41,13 @@ You can create groups for numerous reasons. To name a couple:
...
@@ -41,12 +41,13 @@ You can create groups for numerous reasons. To name a couple:
- Make it easier to `@mention` all of your team at once in issues and merge requests by creating a group and including the appropriate members.
- Make it easier to `@mention` all of your team at once in issues and merge requests by creating a group and including the appropriate members.
For example, you could create a group for your company members, and create a [subgroup](subgroups/index.md) for each individual team. Let's say you create a group called `company-team`, and you create subgroups in this group for the individual teams `backend-team`, `frontend-team`, and `production-team`.
For example, you could create a group for your company members, and create a [subgroup](subgroups/index.md) for each individual team. Let's say you create a group called `company-team`, and you create subgroups in this group for the individual teams `backend-team`, `frontend-team`, and `production-team`.
- When you start a new implementation from an issue, you add a comment:
_"`@company-team`, let's do it! `@company-team/backend-team` you're good to go!"_
- When you start a new implementation from an issue, you add a comment:
- When your backend team needs help from frontend, they add a comment:
_"`@company-team`, let's do it! `@company-team/backend-team` you're good to go!"_
_"`@company-team/frontend-team` could you help us here please?"_
- When your backend team needs help from frontend, they add a comment:
- When the frontend team completes their implementation, they comment:
_"`@company-team/frontend-team` could you help us here please?"_
_"`@company-team/backend-team`, it's done! Let's ship it `@company-team/production-team`!"_
- When the frontend team completes their implementation, they comment:
_"`@company-team/backend-team`, it's done! Let's ship it `@company-team/production-team`!"_
@@ -12,9 +12,9 @@ By displaying the logs directly in GitLab, developers can avoid having to manage
...
@@ -12,9 +12,9 @@ By displaying the logs directly in GitLab, developers can avoid having to manage
1. Go to **Operations > Environments** and find the environment which contains the desired pod, like `production`.
1. Go to **Operations > Environments** and find the environment which contains the desired pod, like `production`.
1. On the **Environments** page, you should see the status of the environment's pods with [Deploy Boards](../deploy_boards.md).
1. On the **Environments** page, you should see the status of the environment's pods with [Deploy Boards](../deploy_boards.md).
1. When mousing over the list of pods, a tooltip will appear with the exact pod name and status.
1. When mousing over the list of pods, a tooltip will appear with the exact pod name and status.
![Deploy Boards pod list](img/pod_logs_deploy_board.png)
![Deploy Boards pod list](img/pod_logs_deploy_board.png)
1. Click on the desired pod to bring up the logs view, which will contain the last 500 lines for that pod. Support for pods with multiple containers is coming [in a future release](https://gitlab.com/gitlab-org/gitlab-ee/issues/6502).
1. Click on the desired pod to bring up the logs view, which will contain the last 500 lines for that pod. Support for pods with multiple containers is coming [in a future release](https://gitlab.com/gitlab-org/gitlab-ee/issues/6502).
![Deploy Boards pod list](img/kubernetes_pod_logs.png)
![Deploy Boards pod list](img/kubernetes_pod_logs.png)