| `Elasticsearch indexing` | Enables or disables Elasticsearch indexing and creates an empty index if one does not already exist. You may want to enable indexing but disable search in order to give the index time to be fully completed, for example. Also, keep in mind that this option doesn't have any impact on existing data, this only enables/disables the background indexer which tracks data changes and ensures new data is indexed. |
| `Pause Elasticsearch indexing` | Enables or disables temporary indexing pause. This is useful for cluster migration/reindexing. All changes are still tracked, but they are not committed to the Elasticsearch index until unpaused. |
| `Pause Elasticsearch indexing` | Enables or disables temporary indexing pause. This is useful for cluster migration/reindexing. All changes are still tracked, but they are not committed to the Elasticsearch index until resumed. |
| `Search with Elasticsearch enabled` | Enables or disables using Elasticsearch in search. |
| `URL` | The URL to use for connecting to Elasticsearch. Use a comma-separated list to support clustering (e.g., `http://host1, https://host2:9200`). If your Elasticsearch instance is password protected, pass the `username:password` in the URL (e.g., `http://<username>:<password>@<elastic_host>:9200/`). |
| `Number of Elasticsearch shards` | Elasticsearch indexes are split into multiple shards for performance reasons. In general, larger indexes need to have more shards. Changes to this value do not take effect until the index is recreated. You can read more about tradeoffs in the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/scalability.html). |
...
...
@@ -260,7 +260,7 @@ from the Elasticsearch index as expected.
## Enabling custom language analyzers
You can improve the language support for Chinese and Japanese languages by utilizing [smartcn](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-smartcn.html) and/or [kuromoji](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-kuromoji.html) analysis plugins from Elastic.
You can improve the language support for Chinese and Japanese languages by utilizing [`smartcn`](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-smartcn.html) and/or [`kuromoji`](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-kuromoji.html) analysis plugins from Elastic.
To enable language(s) support:
...
...
@@ -276,10 +276,10 @@ For guidance on what to install, see the following Elasticsearch language plugin
| `Enable Chinese (smartcn) custom analyzer: Indexing` | Enables or disables Chinese language support using [smartcn](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-smartcn.html) custom analyzer for newly created indices.|
| `Enable Chinese (smartcn) custom analyzer: Search` | Enables or disables using [smartcn](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-smartcn.html) fields for Advanced Search. Please only enable this after [installing the plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-smartcn.html), enabling custom analyzer indexing and recreating the index.|
| `Enable Japanese (kuromoji) custom analyzer: Indexing` | Enables or disables Japanese language support using [kuromoji](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-kuromoji.html) custom analyzer for newly created indices.|
| `Enable Japanese (kuromoji) custom analyzer: Search` | Enables or disables using [kuromoji](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-kuromoji.html) fields for Advanced Search. Please only enable this after [installing the plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-kuromoji.html), enabling custom analyzer indexing and recreating the index.|
| `Enable Chinese (smartcn) custom analyzer: Indexing` | Enables or disables Chinese language support using [`smartcn`](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-smartcn.html) custom analyzer for newly created indices.|
| `Enable Chinese (smartcn) custom analyzer: Search` | Enables or disables using [`smartcn`](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-smartcn.html) fields for Advanced Search. Please only enable this after [installing the plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-smartcn.html), enabling custom analyzer indexing and recreating the index.|
| `Enable Japanese (kuromoji) custom analyzer: Indexing` | Enables or disables Japanese language support using [`kuromoji`](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-kuromoji.html) custom analyzer for newly created indices.|
| `Enable Japanese (kuromoji) custom analyzer: Search` | Enables or disables using [`kuromoji`](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-kuromoji.html) fields for Advanced Search. Please only enable this after [installing the plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-kuromoji.html), enabling custom analyzer indexing and recreating the index.|
## Disabling Advanced Search
...
...
@@ -317,9 +317,9 @@ used by the GitLab Advanced Search integration.
In the **Admin Area > Settings > Advanced Search** section, select the
**Pause Elasticsearch Indexing** setting, and then save your change.
With this, all updates that should happen on your Elasticsearch index will be
buffered and caught up once unpaused.
buffered and caught up after resuming.
The indexing will also be automatically paused when the [**Trigger cluster reindexing**](#trigger-the-reindex-via-the-advanced-search-administration) button is used, and unpaused when the reindexing completes or aborts.
The indexing will also be automatically paused when the [**Trigger cluster reindexing**](#trigger-the-reindex-via-the-advanced-search-administration) button is used, and resumes when the reindexing completes or aborts.
### Setup
...
...
@@ -430,7 +430,7 @@ To trigger the re-index from `primary` index:
The reindexing is now completed. Your GitLab instance is now ready to use the [automated in-cluster reindexing](#trigger-the-reindex-via-the-advanced-search-administration) feature for future reindexing.
1.Unpause the indexing
1.Resume the indexing
Under **Admin Area > Settings > Advanced Search**, uncheck the **Pause Elasticsearch Indexing** setting and save.
...
...
@@ -448,9 +448,9 @@ After the reindexing is completed, the original index will be scheduled to be de
While the reindexing is running, you will be able to follow its progress under that same section.
### Mark the most recent reindex job as failed and unpause the indexing
### Mark the most recent reindex job as failed and resume the indexing
Sometimes, you might want to abandon the unfinished reindex job and unpause the indexing. You can achieve this via the following steps:
Sometimes, you might want to abandon the unfinished reindex job and resume the indexing. You can achieve this via the following steps:
1. Mark the most recent reindex job as failed:
...
...
@@ -518,8 +518,8 @@ In order to debug issues with the migrations you can check the [`elasticsearch.l
Some migrations are built with a retry limit. If the migration cannot finish within the retry limit,
it will be halted and a notification will be displayed in the Advanced Search integration settings.
It is recommended to check the [`elasticsearch.log` file](../administration/logs.md#elasticsearchlog) to
debug why the migration was halted and make any changes before retrying the migration. Once you believe you've
fixed the cause of the failure, click "Retry migration", and the migration will be scheduled to be retried
debug why the migration was halted and make any changes before retrying the migration. Once you believe you've
fixed the cause of the failure, click "Retry migration", and the migration will be scheduled to be retried
in the background.
## GitLab Advanced Search Rake tasks
...
...
@@ -598,7 +598,7 @@ For basic guidance on choosing a cluster configuration you may refer to [Elastic
- Number of CPUs (CPU cores) per node usually corresponds to the `Number of Elasticsearch shards` setting described below.
- A good guideline is to ensure you keep the number of shards per node below 20 per GB heap it has configured. A node with a 30GB heap should therefore have a maximum of 600 shards, but the further below this limit you can keep it the better. This will generally help the cluster stay in good health.
- Small shards result in small segments, which increases overhead. Aim to keep the average shard size between at least a few GB and a few tens of GB. Another consideration is the number of documents, you should aim for this simple formula for the number of shards: `number of expected documents / 5M +1`.
- `refresh_interval` is a per index setting. You may want to adjust that from default `1s` to a bigger value if you don't need data in realtime. This will change how soon you will see fresh results. If that's important for you, you should leave it as close as possible to the default value.
- `refresh_interval` is a per index setting. You may want to adjust that from default `1s` to a bigger value if you don't need data in real-time. This will change how soon you will see fresh results. If that's important for you, you should leave it as close as possible to the default value.
- You might want to raise [`indices.memory.index_buffer_size`](https://www.elastic.co/guide/en/elasticsearch/reference/current/indexing-buffer.html) to 30% or 40% if you have a lot of heavy indexing operations.
### Advanced Search integration settings guidance
...
...
@@ -993,7 +993,7 @@ Sometimes there may be issues with your Elasticsearch index data and as such
GitLab will allow you to revert to "basic search" when there are no search
results and assuming that basic search is supported in that scope. This "basic
search" will behave as though you don't have Advanced Search enabled at all for
your instance and search using other data sources (ie. PostgreSQL data and Git
your instance and search using other data sources (such as PostgreSQL data and Git
data).
### Data recovery: Elasticsearch is a secondary data store only
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/37238) in GitLab 12.7.
> - From [GitLab 12.8 onwards](https://gitlab.com/gitlab-org/gitlab/-/issues/39505), custom metrics are also duplicated when you duplicate a dashboard.
> - [GitLab versions 12.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/39505), custom metrics are also duplicated when you duplicate a dashboard.
You can save a complete copy of a GitLab-defined dashboard along with all custom metrics added to it.
The resulting `.yml` file can be customized and adapted to your project.
...
...
@@ -128,7 +128,7 @@ any chart on a dashboard:
The options are:
-**Expand panel** - Displays a larger version of a visualization. To return to
the dashboard, click the **Back** button in your browser, or press the <kbd>Esc</kbd> key.
the dashboard, click the **Back** button in your browser, or press the <kbd>Escape</kbd> key.
([Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3100) in GitLab 13.0.)
@@ -185,6 +192,8 @@ The GitLab University curriculum is composed of GitLab videos, screencasts, pres
### 3.9. Integrations
<!-- vale gitlab.Spelling = NO -->
1.[How to Integrate Jira and Jenkins with GitLab - Video](https://gitlabmeetings.webex.com/gitlabmeetings/ldr.php?RCID=44b548147a67ab4d8a62274047146415)
1.[How to Integrate Jira with GitLab](../user/project/integrations/jira.md)
1.[How to Integrate Jenkins with GitLab](../integration/jenkins.md)
...
...
@@ -193,9 +202,11 @@ The GitLab University curriculum is composed of GitLab videos, screencasts, pres
1.[How to Integrate Convox with GitLab](https://about.gitlab.com/blog/2016/06/09/continuous-delivery-with-gitlab-and-convox/)
1.[Getting Started with GitLab and Shippable CI](https://about.gitlab.com/blog/2016/05/05/getting-started-gitlab-and-shippable/)
<!-- vale gitlab.Spelling = YES -->
## 4. External Articles
1.[2011 WSJ article by Marc Andreessen - Software is Eating the World](https://www.wsj.com/articles/SB10001424053111903480904576512250915629460)
1.[2011 Wall Street Journal article - Software is Eating the World](https://www.wsj.com/articles/SB10001424053111903480904576512250915629460)
1.[2014 Blog post by Chris Dixon - Software eats software development](https://cdixon.org/2014/04/13/software-eats-software-development/)
1.[2015 Venture Beat article - Actually, Open Source is Eating the World](https://venturebeat.com/2015/12/06/its-actually-open-source-software-thats-eating-the-world/)
