@@ -201,8 +201,8 @@ estimated to keep migration timing to a minimum.
NOTE: **Note:**
Keep in mind that all runtimes should be measured against GitLab.com.
|Migration Type | Execution Time Recommended | Notes |
|Migration Type | Execution Time Recommended | Notes |
|----|----|---|
| Regular migrations on `db/migrate` | `3 minutes` | A valid exception are index creation as this can take a long time. |
| Post migrations on `db/post_migrate`| `10 minutes` | |
| Background migrations | --- | Since these are suitable for larger tables, it's not possible to set a precise timing guideline, however, any single query must stay below `1 second` execution time with cold caches. |
| Post migrations on `db/post_migrate`| `10 minutes` | |
| Background migrations | --- | Since these are suitable for larger tables, it's not possible to set a precise timing guideline, however, any single query must stay below `1 second` execution time with cold caches. |
@@ -100,7 +100,7 @@ To propose additions to the glossary please
### Inclusive language in French
<!-- vale gitlab.Spelling = NO -->
In French, the "écritureinclusive" is now over (see on [Legifrance](https://www.legifrance.gouv.fr/affichTexte.do?cidTexte=JORFTEXT000036068906&categorieLien=id)).
In French, the "écritureinclusive" is now over (see on [Legifrance](https://www.legifrance.gouv.fr/affichTexte.do?cidTexte=JORFTEXT000036068906&categorieLien=id)).
So, to include both genders, write “Utilisateurs et utilisatrices” instead of “Utilisateur·rice·s”.
When space is missing, the male gender should be used alone.
Model can be updated through a refresh strategy based on a project routes SQL and a [materialized view](https://www.postgresql.org/docs/11/rules-materializedviews.html):
...
...
@@ -71,7 +71,7 @@ While this implied a single query update (and probably a fast one), it has some
- Materialized views syntax varies from PostgreSQL and MySQL. While this feature was worked on, MySQL was still supported by GitLab.
- Rails does not have native support for materialized views. We'd need to use a specialized gem to take care of the management of the database views, which implies additional work.
###Attempt B: An update through a CTE
###Attempt B: An update through a CTE
Similar to Attempt A: Model update done through a refresh strategy with a [Common Table Expression](https://www.postgresql.org/docs/9.1/queries-with.html)
...
...
@@ -140,7 +140,7 @@ Even though this approach would make aggregating much easier, it has some major
- We'd have to migrate **all namespaces** by adding and filling a new column. Because of the size of the table, dealing with time/cost will not be great. The background migration will take approximately `153h`, see <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/29772>.
- Background migration has to be shipped one release before, delaying the functionality by another milestone.
###Attempt E (final): Update the namespace storage statistics in async way
###Attempt E (final): Update the namespace storage statistics in async way
This approach consists of keep using the incremental statistics updates we currently already have,
but we refresh them through Sidekiq jobs and in different transactions:
...
...
@@ -170,7 +170,7 @@ The only downside of this approach is that namespaces' statistics are updated up
which means there's a time window in which the statistics are inaccurate. Because we're still not
[enforcing storage limits](https://gitlab.com/gitlab-org/gitlab/-/issues/17664), this is not a major problem.
##Conclusion
##Conclusion
Updating the storage statistics asynchronously, was the less problematic and
performant approach of aggregating the root namespaces.
@@ -74,7 +74,7 @@ A step-by-step guide to [upgrading the Omnibus-bundled PostgreSQL is documented
## Upgrading major versions
Backward-incompatible changes and migrations are reserved for major versions. See the [upgrade guide](../update/README.md#upgrading-to-a-new-major-version).
Backward-incompatible changes and migrations are reserved for major versions. See the [upgrade guide](../update/README.md#upgrading-to-a-new-major-version).
@@ -53,7 +53,7 @@ is **not** `19.03.0`. See [troubleshooting information](#error-response-from-dae
## Supported languages and package managers
GitLab relies on [`rules`](../../../ci/yaml/README.md#rules) to start relevant analyzers depending on the languages detected in the repository.
The current detection logic limits the maximum search depth to two levels. For example, the `gemnasium-dependency_scanning` job is enabled if a repository contains either a `Gemfile`or `api/Gemfile` file, but not if the only supported dependency file is `api/client/Gemfile`.
The current detection logic limits the maximum search depth to two levels. For example, the `gemnasium-dependency_scanning` job is enabled if a repository contains either a `Gemfile`or `api/Gemfile` file, but not if the only supported dependency file is `api/client/Gemfile`.
The following languages and dependency managers are supported:
| [Container Scanning](container_scanning/index.md) | Uses `clair`. The latest `clair-db` version is used for each job by running the [`latest` Docker image tag](https://gitlab.com/gitlab-org/gitlab/blob/438a0a56dc0882f22bdd82e700554525f552d91b/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L37). The `clair-db` database [is updated daily according to the author](https://github.com/arminc/clair-local-scan#clair-server-or-local). |
| [Dependency Scanning](dependency_scanning/index.md) | Relies on `bundler-audit` (for Ruby gems), `retire.js` (for NPM packages), and `gemnasium` (GitLab's own tool for all libraries). Both `bundler-audit` and `retire.js` fetch their vulnerabilities data from GitHub repositories, so vulnerabilities added to `ruby-advisory-db` and`retire.js` are immediately available. The tools themselves are updated once per month if there's a new version. The [Gemnasium DB](https://gitlab.com/gitlab-org/security-products/gemnasium-db) is updated at least once a week. See our [current measurement of time from CVE being issued to our product being updated](https://about.gitlab.com/handbook/engineering/development/performance-indicators/#cve-issue-to-update). |
| [Dependency Scanning](dependency_scanning/index.md) | Relies on `bundler-audit` (for Ruby gems), `retire.js` (for NPM packages), and `gemnasium` (GitLab's own tool for all libraries). Both `bundler-audit` and `retire.js` fetch their vulnerabilities data from GitHub repositories, so vulnerabilities added to `ruby-advisory-db` and`retire.js` are immediately available. The tools themselves are updated once per month if there's a new version. The [Gemnasium DB](https://gitlab.com/gitlab-org/security-products/gemnasium-db) is updated at least once a week. See our [current measurement of time from CVE being issued to our product being updated](https://about.gitlab.com/handbook/engineering/development/performance-indicators/#cve-issue-to-update). |
| [Dynamic Application Security Testing (DAST)](dast/index.md) | The scanning engine is updated on a periodic basis. See the [version of the underlying tool `zaproxy`](https://gitlab.com/gitlab-org/security-products/dast/blob/master/Dockerfile#L1). The scanning rules are downloaded at scan runtime. |
| [Static Application Security Testing (SAST)](sast/index.md) | Relies exclusively on [the tools GitLab wraps](sast/index.md#supported-languages-and-frameworks). The underlying analyzers are updated at least once per month if a relevant update is available. The vulnerabilities database is updated by the upstream tools. |
@@ -348,7 +348,7 @@ This can be due to multiple reasons:
nothing will be displayed.
- The [`artifacts:expire_in`](../../../ci/yaml/README.md#artifactsexpire_in) CI/CD
setting can cause the Code Quality artifact(s) to expire faster than desired.
- Large `codeclimate.json` files (esp. >10MB) are [known to prevent the report from being displayed](https://gitlab.com/gitlab-org/gitlab/-/issues/2737).
- Large `codeclimate.json` files (esp. >10MB) are [known to prevent the report from being displayed](https://gitlab.com/gitlab-org/gitlab/-/issues/2737).
As a work-around, try removing [properties](https://github.com/codeclimate/platform/blob/master/spec/analyzers/SPEC.md#data-types)
that are [ignored by GitLab](#implementing-a-custom-tool). You can:
- Configure the Code Quality tool to not output those types.
