@@ -6,8 +6,8 @@ The requirements are listed [on the index page](index.md#requirements-for-runnin
## How does Geo know which projects to sync?
On each **secondary** node, there is a read-only replicated copy of the GitLab database.
A **secondary** node also has a tracking database where it stores which projects have been synced.
On each **secondary** node, there is a read-only replicated copy of the GitLab database.
A **secondary** node also has a tracking database where it stores which projects have been synced.
Geo compares the two databases to find projects that are not yet tracked.
At the start, this tracking database is empty, so Geo will start trying to update from every project that it can see in the GitLab database.
...
...
@@ -15,19 +15,19 @@ At the start, this tracking database is empty, so Geo will start trying to updat
For each project to sync:
1. Geo will issue a `git fetch geo --mirror` to get the latest information from the **primary** node.
If there are no changes, the sync will be fast and end quickly. Otherwise, it will pull the latest commits.
If there are no changes, the sync will be fast and end quickly. Otherwise, it will pull the latest commits.
1. The **secondary** node will update the tracking database to store the fact that it has synced projects A, B, C, etc.
1. Repeat until all projects are synced.
When someone pushes a commit to the **primary** node, it generates an event in the GitLab database that the repository has changed.
When someone pushes a commit to the **primary** node, it generates an event in the GitLab database that the repository has changed.
The **secondary** node sees this event, marks the project in question as dirty, and schedules the project to be resynced.
To ensure that problems with pipelines (for example, syncs failing too many times or jobs being lost) don't permanently stop projects syncing, Geo also periodically checks the tracking database for projects that are marked as dirty. This check happens when
the number of concurrent syncs falls below `repos_max_capacity` and there are no new projects waiting to be synced.
the number of concurrent syncs falls below `repos_max_capacity` and there are no new projects waiting to be synced.
Geo also has a checksum feature which runs a SHA256 sum across all the Git references to the SHA values.
If the refs don't match between the **primary** node and the **secondary** node, then the **secondary** node will mark that project as dirty and try to resync it.
So even if we have an outdated tracking database, the validation should activate and find discrepancies in the repository state and resync.
Geo also has a checksum feature which runs a SHA256 sum across all the Git references to the SHA values.
If the refs don't match between the **primary** node and the **secondary** node, then the **secondary** node will mark that project as dirty and try to resync it.
So even if we have an outdated tracking database, the validation should activate and find discrepancies in the repository state and resync.
## Can I use Geo in a disaster recovery situation?
@@ -909,11 +909,12 @@ import bundle from 'ee_else_ce/protected_branches/protected_branches_bundle.js';
See the frontend guide [performance section](fe_guide/performance.md) for
information on managing page-specific javascript within EE.
## Vue code in `assets/javascript`
### script tag
#### Child Component only used in EE
To separate Vue template differences we should [async import the components](https://vuejs.org/v2/guide/components-dynamic-async.html#Async-Components).
Doing this allows for us to load the correct component in EE whilst in CE
...
...
@@ -937,10 +938,12 @@ export default {
```
#### For JS code that is EE only, like props, computed properties, methods, etc, we will keep the current approach
- Since we [can't async load a mixin](https://github.com/vuejs/vue-loader/issues/418#issuecomment-254032223) we will use the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) alias we already have for webpack.
- Since we [can't async load a mixin](https://github.com/vuejs/vue-loader/issues/418#issuecomment-254032223) we will use the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) alias we already have for webpack.
- This means all the EE specific props, computed properties, methods, etc that are EE only should be in a mixin in the `ee/` folder and we need to create a CE counterpart of the mixin
##### Example:
```javascript
importmixinfrom'ee_else_ce/path/mixin';
...
...
@@ -955,6 +958,7 @@ import mixin from 'ee_else_ce/path/mixin';
- You can see an MR with an example [here](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/9762)
#### `template` tag
***EE Child components**
- Since we are using the async loading to check which component to load, we'd still use the component's name, check [this example](#child-component-only-used-in-ee).
...
...
@@ -962,11 +966,12 @@ import mixin from 'ee_else_ce/path/mixin';
- For the templates that have extra HTML in EE we should move it into a new component and use the `ee_else_ce` dynamic import
### Non Vue Files
For regular JS files, the approach is similar.
1. We will keep using the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) helper, this means that EE only code should be inside the `ee/` folder.
1. An EE file should be created with the EE only code, and it should extend the CE counterpart.
1. For code inside functions that can't be extended, the code should be moved into a new file and we should use `ee_else_ce` helper:
1. An EE file should be created with the EE only code, and it should extend the CE counterpart.
1. For code inside functions that can't be extended, the code should be moved into a new file and we should use `ee_else_ce` helper:
##### Example:
...
...
@@ -996,6 +1001,7 @@ to isolate such ruleset from rest of CE rules (along with adding comment describ
to avoid conflicts during CE to EE merge.
#### Bad
```scss
.section-body{
.section-title{
...
...
@@ -1011,6 +1017,7 @@ to avoid conflicts during CE to EE merge.
@@ -64,20 +64,25 @@ All indexing after the initial one is done via `ElasticIndexerWorker` (sidekiq j
Search queries are generated by the concerns found in [ee/app/models/concerns/elastic](https://gitlab.com/gitlab-org/gitlab-ee/tree/master/ee/app/models/concerns/elastic). These concerns are also in charge of access control, and have been a historic source of security bugs so please pay close attention to them!
## Existing Analyzers/Tokenizers/Filters
These are all defined in https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/elasticsearch/git/model.rb
### Analyzers
#### `path_analyzer`
Used when indexing blobs' paths. Uses the `path_tokenizer` and the `lowercase` and `asciifolding` filters.
Please see the `path_tokenizer` explanation below for an example.
#### `sha_analyzer`
Used in blobs and commits. Uses the `sha_tokenizer` and the `lowercase` and `asciifolding` filters.
Please see the `sha_tokenizer` explanation later below for an example.
#### `code_analyzer`
Used when indexing a blob's filename and content. Uses the `whitespace` tokenizer and the filters: `code`, `edgeNGram_filter`, `lowercase`, and `asciifolding`
The `whitespace` tokenizer was selected in order to have more control over how tokens are split. For example the string `Foo::bar(4)` needs to generate tokens like `Foo` and `bar(4)` in order to be properly searched.
...
...
@@ -85,15 +90,19 @@ The `whitespace` tokenizer was selected in order to have more control over how t
Please see the `code` filter for an explanation on how tokens are split.
#### `code_search_analyzer`
Not directly used for indexing, but rather used to transform a search input. Uses the `whitespace` tokenizer and the `lowercase` and `asciifolding` filters.
### Tokenizers
#### `sha_tokenizer`
This is a custom tokenizer that uses the [`edgeNGram` tokenizer](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-edgengram-tokenizer.html) to allow SHAs to be searcheable by any sub-set of it (minimum of 5 chars).
example:
Example:
`240c29dc7e` becomes:
-`240c2`
-`240c29`
-`240c29d`
...
...
@@ -102,21 +111,26 @@ example:
-`240c29dc7e`
#### `path_tokenizer`
This is a custom tokenizer that uses the [`path_hierarchy` tokenizer](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-pathhierarchy-tokenizer.html) with `reverse: true` in order to allow searches to find paths no matter how much or how little of the path is given as input.
example:
Example:
`'/some/path/application.js'` becomes:
-`'/some/path/application.js'`
-`'some/path/application.js'`
-`'path/application.js'`
-`'application.js'`
### Filters
#### `code`
Uses a [Pattern Capture token filter](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-pattern-capture-tokenfilter.html) to split tokens into more easily searched versions of themselves.
Uses a [Pattern Capture token filter](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-pattern-capture-tokenfilter.html) to split tokens into more easily searched versions of themselves.
Patterns:
-`"(\\p{Ll}+|\\p{Lu}\\p{Ll}+|\\p{Lu}+)"`: captures CamelCased and lowedCameCased strings as separate tokens
-`'\/?([^\/]+)(?=\/|\b)'`: separate path terms `like/this/one`
#### `edgeNGram_filter`
Uses an [Edge NGram token filter](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-edgengram-tokenfilter.html) to allow inputs with only parts of a token to find the token. For example it would turn `glasses` into permutations starting with `gl` and ending with `glasses`, which would allow a search for "`glass`" to find the original token `glasses`
## Gotchas
...
...
@@ -140,13 +155,13 @@ Uses an [Edge NGram token filter](https://www.elastic.co/guide/en/elasticsearch/
In our case, `qa-login-field`, `qa-password-field` and `qa-sign-in-button`
**app/views/my/view.html.haml**
**app/views/my/view.html.haml**
```haml
=f.text_field:login,class: "form-control top qa-login-field",autofocus: "autofocus",autocapitalize: "off",autocorrect: "off",required: true,title: "This field is required."
...
...
@@ -146,7 +146,7 @@ Things to note:
- The CSS class must be `kebab-cased` (separated with hyphens "`-`")
- If the element appears on the page unconditionally, add `required: true` to the element. See
[Dynamic element validation](dynamic_element_validation.md)
[Dynamic element validation](dynamic_element_validation.md)
@@ -25,22 +25,22 @@ and [Migrating from Jenkins to GitLab](https://www.youtube.com/watch?v=RlEVGOpYF
## Use cases
- Suppose you are new to GitLab, and want to keep using Jenkins until you prepare
your projects to build with [GitLab CI/CD](../ci/README.md). You set up the
integration between GitLab and Jenkins, then you migrate to GitLab CI later. While
you organize yourself and your team to onboard GitLab, you keep your pipelines
running with Jenkins, but view the results in your project's repository in GitLab.
your projects to build with [GitLab CI/CD](../ci/README.md). You set up the
integration between GitLab and Jenkins, then you migrate to GitLab CI later. While
you organize yourself and your team to onboard GitLab, you keep your pipelines
running with Jenkins, but view the results in your project's repository in GitLab.
- Your team uses [Jenkins Plugins](https://plugins.jenkins.io/) for other proceedings,
therefore, you opt for keep using Jenkins to build your apps. Show the results of your
pipelines directly in GitLab.
therefore, you opt for keep using Jenkins to build your apps. Show the results of your
pipelines directly in GitLab.
For a real use case, read the blog post [Continuous integration: From Jenkins to GitLab using Docker](https://about.gitlab.com/2017/07/27/docker-my-precious/).
@@ -14,7 +14,7 @@ Learn how GitLab helps you in the stages of the DevOps lifecycle by learning mor
### Self-managed: Install GitLab
Take a look at [installing GitLab](https://about.gitlab.com/install/) and our [administrator documentation](../administration/index.md). Then, follow the instructions below under [Your subscription](#your-subscription) to apply your license file.
Take a look at [installing GitLab](https://about.gitlab.com/install/) and our [administrator documentation](../administration/index.md). Then, follow the instructions below under [Your subscription](#your-subscription) to apply your license file.
### GitLab.com: Create a user and group
...
...
@@ -74,11 +74,11 @@ Please note that you need to be a group owner to associate a group to your subsc
To see the status of your GitLab.com subscription, you can click on the Billings
section of the relevant namespace:
* For individuals, this is located at https://gitlab.com/profile/billings under
in your Settings,
* For groups, this is located under the group's Settings dropdown, under Billing.
- For individuals, this is located at https://gitlab.com/profile/billings under
in your Settings,
- For groups, this is located under the group's Settings dropdown, under Billing.
For groups, you can see details of your subscription - including your current
For groups, you can see details of your subscription - including your current
plan - in the included table:
![Billing table](billing_table.png)
...
...
@@ -86,11 +86,11 @@ plan - in the included table:
| Field | Description |
| ------ | ------ |
| Seats in subscription | If this is a paid plan, this represents the number of seats you've paid to support in your group. |
| Seats currently in use | The number of active seats currently in use. |
| Max seats used | The highest number of seats you've used. If this exceeds the seats in subscription, you may owe an additional fee for the additional users. |
| Seats owed | If your max seats used exceeds the seats in your subscription, you'll owe an additional fee for the users you've added. |
| Subscription start date | The date your subscription started. If this is for a Free plan, this is the date you transitioned off your group's paid plan. |
| Subscription end date | The date your current subscription will end. This does not apply to Free plans. |
| Seats currently in use | The number of active seats currently in use. |
| Max seats used | The highest number of seats you've used. If this exceeds the seats in subscription, you may owe an additional fee for the additional users. |
| Seats owed | If your max seats used exceeds the seats in your subscription, you'll owe an additional fee for the users you've added. |
| Subscription start date | The date your subscription started. If this is for a Free plan, this is the date you transitioned off your group's paid plan. |
| Subscription end date | The date your current subscription will end. This does not apply to Free plans. |
@@ -41,12 +41,13 @@ You can create groups for numerous reasons. To name a couple:
- Make it easier to `@mention` all of your team at once in issues and merge requests by creating a group and including the appropriate members.
For example, you could create a group for your company members, and create a [subgroup](subgroups/index.md) for each individual team. Let's say you create a group called `company-team`, and you create subgroups in this group for the individual teams `backend-team`, `frontend-team`, and `production-team`.
- When you start a new implementation from an issue, you add a comment:
_"`@company-team`, let's do it! `@company-team/backend-team` you're good to go!"_
- When your backend team needs help from frontend, they add a comment:
_"`@company-team/frontend-team` could you help us here please?"_
- When the frontend team completes their implementation, they comment:
_"`@company-team/backend-team`, it's done! Let's ship it `@company-team/production-team`!"_
- When you start a new implementation from an issue, you add a comment:
_"`@company-team`, let's do it! `@company-team/backend-team` you're good to go!"_
- When your backend team needs help from frontend, they add a comment:
_"`@company-team/frontend-team` could you help us here please?"_
- When the frontend team completes their implementation, they comment:
_"`@company-team/backend-team`, it's done! Let's ship it `@company-team/production-team`!"_
@@ -12,9 +12,9 @@ By displaying the logs directly in GitLab, developers can avoid having to manage
1. Go to **Operations > Environments** and find the environment which contains the desired pod, like `production`.
1. On the **Environments** page, you should see the status of the environment's pods with [Deploy Boards](../deploy_boards.md).
1. When mousing over the list of pods, a tooltip will appear with the exact pod name and status.
![Deploy Boards pod list](img/pod_logs_deploy_board.png)
![Deploy Boards pod list](img/pod_logs_deploy_board.png)
1. Click on the desired pod to bring up the logs view, which will contain the last 500 lines for that pod. Support for pods with multiple containers is coming [in a future release](https://gitlab.com/gitlab-org/gitlab-ee/issues/6502).
![Deploy Boards pod list](img/kubernetes_pod_logs.png)
![Deploy Boards pod list](img/kubernetes_pod_logs.png)