Merge branch 'eread/move-concepts-to-main-page' into 'master'

Move some conceptual material to main page and other minor improvements

See merge request gitlab-org/gitlab!66137

doc/administration/gitaly/faq.md

@@ -25,7 +25,7 @@ The following table outlines the major differences between Gitaly Cluster and Ge
 
 | Tool           | Nodes    | Locations | Latency tolerance  | Failover                                                                    | Consistency                              | Provides redundancy for |
 |:---------------|:---------|:----------|:-------------------|:----------------------------------------------------------------------------|:-----------------------------------------|:------------------------|
-| Gitaly Cluster | Multiple | Single    | Approximately 1 ms | [Automatic](praefect.md#automatic-failover-and-primary-election-strategies) | [Strong](praefect.md#strong-consistency) | Data storage in Git     |
+| Gitaly Cluster | Multiple | Single    | Approximately 1 ms | [Automatic](praefect.md#automatic-failover-and-primary-election-strategies) | [Strong](index.md#strong-consistency)    | Data storage in Git     |
 | Geo            | Multiple | Multiple  | Up to one minute   | [Manual](../geo/disaster_recovery/index.md)                                 | Eventual                                 | Entire GitLab instance  |
 
 For more information, see:
@@ -40,7 +40,7 @@ Yes! For more information, see [Migrate to Gitaly Cluster](praefect.md#migrate-t
 ## What are some repository storage recommendations?
 
 The size of the required storage can vary between instances and depends on the set
-[replication factor](praefect.md#replication-factor). You might want to include implementing
+[replication factor](index.md#replication-factor). You might want to include implementing
 repository storage redundancy.
 
 For a replication factor:

doc/administration/gitaly/praefect.md

@@ -24,7 +24,7 @@ NOTE:
 Upgrade instructions for Omnibus GitLab installations
 [are available](https://docs.gitlab.com/omnibus/update/#gitaly-cluster).
 
-## Requirements for configuring a Gitaly Cluster
+## Requirements
 
 The minimum recommended configuration for a Gitaly Cluster requires:
 
@@ -33,14 +33,33 @@ The minimum recommended configuration for a Gitaly Cluster requires:
 - 3 Praefect nodes
 - 3 Gitaly nodes (1 primary, 2 secondary)
 
-See the [design
-document](https://gitlab.com/gitlab-org/gitaly/-/blob/master/doc/design_ha.md)
+See the [design document](https://gitlab.com/gitlab-org/gitaly/-/blob/master/doc/design_ha.md)
 for implementation details.
 
 NOTE:
 If not set in GitLab, feature flags are read as false from the console and Praefect uses their
 default value. The default value depends on the GitLab version.
 
+### Network connectivity
+
+Gitaly Cluster [components](index.md#components) need to communicate with each other over many
+routes. Your firewall rules must allow the following for Gitaly Cluster to function properly:
+
+| From                   | To                      | Default port / TLS port |
+|:-----------------------|:------------------------|:------------------------|
+| GitLab                 | Praefect load balancer  | `2305` / `3305`         |
+| Praefect load balancer | Praefect                | `2305` / `3305`         |
+| Praefect               | Gitaly                  | `8075` / `9999`         |
+| Gitaly                 | GitLab (internal API)   | `80` / `443`            |
+| Gitaly                 | Praefect load balancer  | `2305` / `3305`         |
+| Gitaly                 | Praefect                | `2305` / `3305`         |
+| Gitaly                 | Gitaly                  | `8075` / `9999`         |
+
+NOTE:
+Gitaly does not directly connect to Praefect. However, requests from Gitaly to the Praefect
+load balancer may still be blocked unless firewalls on the Praefect nodes allow traffic from
+the Gitaly nodes.
+
 ## Setup Instructions
 
 If you [installed](https://about.gitlab.com/install/) GitLab using the Omnibus GitLab package
@@ -129,7 +148,7 @@ The following options are available:
 
 - For non-Geo installations, either:
   - Use one of the documented [PostgreSQL setups](../postgresql/index.md).
-  - Use your own third-party database setup. This will require [manual setup](#manual-database-setup).
+  - Use your own third-party database setup. This requires [manual setup](#manual-database-setup).
 - For Geo instances, either:
   - Set up a separate [PostgreSQL instance](https://www.postgresql.org/docs/11/high-availability.html).
   - Use a cloud-managed PostgreSQL service. AWS
@@ -457,7 +476,7 @@ On the **Praefect** node:
    In [GitLab 13.8 and earlier](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/4988),
    Gitaly nodes were configured directly under the virtual storage, and not under the `nodes` key.
 
-1. [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2013) in GitLab 13.1 and later, enable [distribution of reads](#distributed-reads).
+1. [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2013) in GitLab 13.1 and later, enable [distribution of reads](index.md#distributed-reads).
 
 1. Save the changes to `/etc/gitlab/gitlab.rb` and [reconfigure
    Praefect](../restart_gitlab.md#omnibus-gitlab-reconfigure):
@@ -1053,75 +1072,9 @@ To get started quickly:
 Congratulations! You've configured an observable fault-tolerant Praefect
 cluster.
 
-## Network connectivity requirements
+## Configure strong consistency
 
-Gitaly Cluster components need to communicate with each other over many routes.
-Your firewall rules must allow the following for Gitaly Cluster to function properly:
-
-| From                   | To                      | Default port / TLS port |
-|:-----------------------|:------------------------|:------------------------|
-| GitLab                 | Praefect load balancer  | `2305` / `3305`         |
-| Praefect load balancer | Praefect                | `2305` / `3305`         |
-| Praefect               | Gitaly                  | `8075` / `9999`         |
-| Gitaly                 | GitLab (internal API)   | `80` / `443`            |
-| Gitaly                 | Praefect load balancer  | `2305` / `3305`         |
-| Gitaly                 | Praefect                | `2305` / `3305`         |
-| Gitaly                 | Gitaly                  | `8075` / `9999`         |
-
-NOTE:
-Gitaly does not directly connect to Praefect. However, requests from Gitaly to the Praefect
-load balancer may still be blocked unless firewalls on the Praefect nodes allow traffic from
-the Gitaly nodes.
-
-## Distributed reads
-
-> - Introduced in GitLab 13.1 in [beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha-beta-ga) with feature flag `gitaly_distributed_reads` set to disabled.
-> - [Made generally available and enabled by default](https://gitlab.com/gitlab-org/gitaly/-/issues/2951) in GitLab 13.3.
-> - [Disabled by default](https://gitlab.com/gitlab-org/gitaly/-/issues/3178) in GitLab 13.5.
-> - [Enabled by default](https://gitlab.com/gitlab-org/gitaly/-/issues/3334) in GitLab 13.8.
-> - [Feature flag removed](https://gitlab.com/gitlab-org/gitaly/-/issues/3383) in GitLab 13.11.
-
-Praefect supports distribution of read operations across Gitaly nodes that are
-configured for the virtual node.
-
-All RPCs marked with `ACCESSOR` option like
-[GetBlob](https://gitlab.com/gitlab-org/gitaly/-/blob/v12.10.6/proto/blob.proto#L16)
-are redirected to an up to date and healthy Gitaly node.
-
-_Up to date_ in this context means that:
-
-- There is no replication operations scheduled for this node.
-- The last replication operation is in _completed_ state.
-
-If there is no such nodes, or any other error occurs during node selection, the primary
-node is chosen to serve the request.
-
-To track distribution of read operations, you can use the `gitaly_praefect_read_distribution`
-Prometheus counter metric. It has two labels:
-
-- `virtual_storage`.
-- `storage`.
-
-They reflect configuration defined for this instance of Praefect.
-
-## Strong consistency
-
-> - Introduced in GitLab 13.1 in [alpha](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha-beta-ga), disabled by default.
-> - Entered [beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha-beta-ga) in GitLab 13.2, disabled by default.
-> - In GitLab 13.3, disabled unless primary-wins voting strategy is disabled.
-> - From GitLab 13.4, enabled by default.
-> - From GitLab 13.5, you must use Git v2.28.0 or higher on Gitaly nodes to enable strong consistency.
-> - From GitLab 13.6, primary-wins voting strategy and `gitaly_reference_transactions_primary_wins` feature flag were removed from the source code.
-
-Praefect guarantees eventual consistency by replicating all writes to secondary nodes
-after the write to the primary Gitaly node has happened.
-
-Praefect can instead provide strong consistency by creating a transaction and writing
-changes to all Gitaly nodes at once.
-If enabled, transactions are only available for a subset of RPCs. For more
-information, see the [strong consistency epic](https://gitlab.com/groups/gitlab-org/-/epics/1189).
-
-To enable strong consistency:
+To enable [strong consistency](index.md#strong-consistency):
 
 - In GitLab 13.5, you must use Git v2.28.0 or higher on Gitaly nodes to enable strong consistency.
 - In GitLab 13.4 and later, the strong consistency voting strategy has been improved and enabled by default.
@@ -1155,14 +1108,7 @@ To monitor strong consistency, you can use the following Prometheus metrics:
 - `gitaly_hook_transaction_voting_delay_seconds`: Client-side delay introduced
   by waiting for the transaction to be committed.
 
-## Replication factor
-
-Replication factor is the number of copies Praefect maintains of a given repository. A higher
-replication factor offers better redundancy and distribution of read workload, but also results
-in a higher storage cost. By default, Praefect replicates repositories to every storage in a
-virtual storage.
-
-### Configure replication factor
+## Configure replication factor
 
 WARNING:
 Configurable replication factors require [repository-specific primary nodes](#repository-specific-primary-nodes) to be used.

doc/development/testing_guide/end_to_end/rspec_metadata_tests.md

@@ -16,7 +16,7 @@ This is a partial list of the [RSpec metadata](https://relishapp.com/rspec/rspec
 | `:elasticsearch`  | The test requires an Elasticsearch service. It is used by the [instance-level scenario](https://gitlab.com/gitlab-org/gitlab-qa#definitions) [`Test::Integration::Elasticsearch`](https://gitlab.com/gitlab-org/gitlab/-/blob/72b62b51bdf513e2936301cb6c7c91ec27c35b4d/qa/qa/ee/scenario/test/integration/elasticsearch.rb) to include only tests that require Elasticsearch. |
 | `:except`         | The test is to be run in their typical execution contexts _except_ as specified. See [test execution context selection](execution_context_selection.md) for more information. |
 | `:geo`            | The test requires two GitLab Geo instances - a primary and a secondary - to be spun up. |
-| `:gitaly_cluster` | The test runs against a GitLab instance where repositories are stored on redundant Gitaly nodes behind a Praefect node. All nodes are [separate containers](../../../administration/gitaly/praefect.md#requirements-for-configuring-a-gitaly-cluster). Tests that use this tag have a longer setup time since there are three additional containers that need to be started. |
+| `:gitaly_cluster` | The test runs against a GitLab instance where repositories are stored on redundant Gitaly nodes behind a Praefect node. All nodes are [separate containers](../../../administration/gitaly/praefect.md#requirements). Tests that use this tag have a longer setup time since there are three additional containers that need to be started. |
 | `:github`         | The test requires a GitHub personal access token. |
 | `:group_saml`     | The test requires a GitLab instance that has SAML SSO enabled at the group level. Interacts with an external SAML identity provider. Paired with the `:orchestrated` tag. |
 | `:instance_saml`  | The test requires a GitLab instance that has SAML SSO enabled at the instance level. Interacts with an external SAML identity provider. Paired with the `:orchestrated` tag. |