Commit 85937dd2 authored by Vitaly Slobodin's avatar Vitaly Slobodin

Merge branch 'eread/refactor-repository-mirroring-admin-area-config' into 'master'

Refactor repository mirroring admin area configuration UI and docs

See merge request gitlab-org/gitlab!65935
parents b9bdb109 1336308f
......@@ -3,14 +3,13 @@
%fieldset
.form-group
= f.label :mirror_available, _('Enable mirror configuration'), class: 'label-bold'
= f.label :mirror_available, _('Repository mirroring configuration'), class: 'label-bold'
.form-check
= f.check_box :mirror_available, class: 'form-check-input'
= f.label :mirror_available, class: 'form-check-label' do
= _('Allow repository mirroring to be configured by project maintainers')
= _('Allow project maintainers to configure repository mirroring')
%span.form-text.text-muted
= _('If disabled, only admins will be able to configure repository mirroring.')
= link_to sprite_icon('question-o'), help_page_path('user/project/repository/repository_mirroring.md')
= _('If disabled, only administrators can configure repository mirroring.')
= render_if_exists 'admin/application_settings/mirror_settings', form: f
......
......@@ -21,6 +21,7 @@
= expanded_by_default? ? 'Collapse' : 'Expand'
%p
= _('Configure repository mirroring.')
= link_to s_('Learn more.'), help_page_path('user/project/repository/repository_mirroring.md'), target: '_blank', rel: 'noopener noreferrer'
.settings-content
= render partial: 'repository_mirrors_form'
......
......@@ -7,7 +7,7 @@ type: reference, api
# Project remote mirrors API **(FREE)**
[Push mirrors](../user/project/repository/repository_mirroring.md#pushing-to-a-remote-repository)
[Push mirrors](../user/project/repository/repository_mirroring.md#push-to-a-remote-repository)
defined on a project's repository settings are called "remote mirrors", and the
state of these mirrors can be queried and modified via the remote mirror API
outlined below.
......
......@@ -20,7 +20,7 @@ To use GitLab CI/CD with a Bitbucket Cloud repository:
![Create project](img/external_repository.png)
GitLab imports the repository and enables [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository).
GitLab imports the repository and enables [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pull-from-a-remote-repository).
1. In GitLab create a
[Personal Access Token](../../user/profile/personal_access_tokens.md)
......
......@@ -46,7 +46,7 @@ repositories:
GitLab:
1. Imports the project.
1. Enables [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository)
1. Enables [Pull Mirroring](../../user/project/repository/repository_mirroring.md#pull-from-a-remote-repository)
1. Enables [GitHub project integration](../../user/project/integrations/github.md)
1. Creates a web hook on GitHub to notify GitLab of new commits.
......
......@@ -38,7 +38,7 @@ A typical pipeline might consist of four stages, executed in the following order
- A `production` stage, with a job called `deploy-to-prod`.
NOTE:
If you have a [mirrored repository that GitLab pulls from](../../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository),
If you have a [mirrored repository that GitLab pulls from](../../user/project/repository/repository_mirroring.md#pull-from-a-remote-repository),
you may need to enable pipeline triggering in your project's
**Settings > Repository > Pull from a remote repository > Trigger pipelines for mirror updates**.
......
......@@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
<!-- vale gitlab.Spelling = NO -->
In December 2018, Tiago Botelho hosted a Deep Dive (GitLab team members only: `https://gitlab.com/gitlab-org/create-stage/issues/1`)
on the GitLab [Pull Repository Mirroring functionality](../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository)
on the GitLab [Pull Repository Mirroring functionality](../user/project/repository/repository_mirroring.md#pull-from-a-remote-repository)
to share his domain specific knowledge with anyone who may work in this part of the
codebase in the future. You can find the <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [recording on YouTube](https://www.youtube.com/watch?v=sSZq0fpdY-Y),
and the slides in [PDF](https://gitlab.com/gitlab-org/create-stage/uploads/8693404888a941fd851f8a8ecdec9675/Gitlab_Create_-_Pull_Mirroring_Deep_Dive.pdf).
......
......@@ -89,14 +89,14 @@ the tiers are no longer mentioned in GitLab documentation:
- Repositories:
- [Repository size limit](../user/admin_area/settings/account_and_limit_settings.md#repository-size-limit)
- Repository mirroring:
- [Pull mirroring](../user/project/repository/repository_mirroring.md#pulling-from-a-remote-repository) outside repositories in a GitLab repository
- [Pull mirroring](../user/project/repository/repository_mirroring.md#pull-from-a-remote-repository) outside repositories in a GitLab repository
- [Overwrite diverged branches](../user/project/repository/repository_mirroring.md#overwrite-diverged-branches)
- [Trigger pipelines for mirror updates](../user/project/repository/repository_mirroring.md#trigger-pipelines-for-mirror-updates)
- [Hard failures](../user/project/repository/repository_mirroring.md#hard-failure) when mirroring fails
- [Trigger pull mirroring from the API](../user/project/repository/repository_mirroring.md#trigger-an-update-using-the-api)
- [Mirror only protected branches](../user/project/repository/repository_mirroring.md#mirror-only-protected-branches)
- [Bidirectional mirroring](../user/project/repository/repository_mirroring.md#bidirectional-mirroring)
- [Mirroring with Perforce Helix via Git Fusion](../user/project/repository/repository_mirroring.md#mirroring-with-perforce-helix-via-git-fusion)
- [Mirror with Perforce Helix via Git Fusion](../user/project/repository/repository_mirroring.md#mirror-with-perforce-helix-via-git-fusion)
- Runners:
- Run pipelines in the parent project [for merge requests from a forked project](../ci/pipelines/merge_request_pipelines.md#run-pipelines-in-the-parent-project-for-merge-requests-from-a-forked-project)
- [Shared runners pipeline minutes quota](../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota)
......
......@@ -11,13 +11,16 @@ Repository mirroring allows for the mirroring of repositories to and from extern
can use it to mirror branches, tags, and commits between repositories. It helps you use
a repository outside of GitLab.
A repository mirror at GitLab updates automatically. You can also manually trigger an update
at most once every five minutes on GitLab.com with [the limit set by the administrator on self-managed instances](../../../administration/instance_limits.md#pull-mirroring-interval).
A repository mirror at GitLab updates automatically. You can also manually trigger an update:
- At most once every five minutes on GitLab.com.
- According to a [limit set by the administrator](../../../administration/instance_limits.md#pull-mirroring-interval)
on self-managed instances.
There are two kinds of repository mirroring supported by GitLab:
- [Push](#pushing-to-a-remote-repository): for mirroring a GitLab repository to another location. **(FREE)**
- [Pull](#pulling-from-a-remote-repository): for mirroring a repository from another location to GitLab. **(PREMIUM)**
- [Push](#push-to-a-remote-repository): for mirroring a GitLab repository to another location.
- [Pull](#pull-from-a-remote-repository): for mirroring a repository from another location to GitLab.
When the mirror repository is updated, all new branches, tags, and commits are visible in the
project's activity feed.
......@@ -48,9 +51,9 @@ The following are some possible use cases for repository mirroring:
GitLab.com repository that's public, allows you to open source specific projects and contribute back
to the open source community.
## Pushing to a remote repository **(FREE)**
## Push to a remote repository
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40137) in GitLab 13.5: LFS support over HTTPS.
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40137) in GitLab 13.5: LFS support over HTTPS.
For an existing project, you can set up push mirroring as follows:
......@@ -63,8 +66,6 @@ For an existing project, you can set up push mirroring as follows:
1. Select the **Keep divergent refs** check box, if desired.
1. Select **Mirror repository** to save the configuration.
![Repository mirroring push settings screen](img/repository_mirroring_push_settings.png)
When push mirroring is enabled, only push commits directly to the mirrored repository to prevent the
mirror diverging.
......@@ -72,7 +73,7 @@ Unlike [pull mirroring](#how-it-works), the mirrored repository is not periodica
The mirrored repository receives all changes only when:
- Commits are pushed to GitLab.
- A [forced update](#forcing-an-update) is initiated.
- A [forced update](#force-an-update) is initiated.
Changes pushed to files in the repository are automatically pushed to the remote mirror at least:
......@@ -82,14 +83,14 @@ Changes pushed to files in the repository are automatically pushed to the remote
In the case of a diverged branch, an error displays in the **Mirroring repositories**
section.
### Configuring push mirrors through the API
### Configure push mirrors through the API
You can also create and modify project push mirrors through the
[remote mirrors API](../../../api/remote_mirrors.md).
### Keep divergent refs
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208828) in GitLab 13.0.
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208828) in GitLab 13.0.
By default, if any ref on the remote mirror has diverged from the local
repository, the *entire push* fails, and no updates occur.
......@@ -108,7 +109,7 @@ update.
NOTE:
After the mirror is created, this option can only be modified via the [API](../../../api/remote_mirrors.md).
### Setting up a push mirror from GitLab to GitHub
### Set up a push mirror from GitLab to GitHub
To set up a mirror from GitLab to GitHub, you need to follow these steps:
......@@ -121,7 +122,7 @@ The mirrored repository is listed. For example, `https://*****:*****@github.com/
The repository pushes shortly thereafter. To force a push, select the **Update now** (**{retry}**) button.
### Setting up a push mirror from GitLab to AWS CodeCommit
### Set up a push mirror from GitLab to AWS CodeCommit
AWS CodeCommit push mirroring is the best way to connect GitLab repositories to
AWS CodePipeline, as GitLab isn't yet supported as one of their Source Code Management (SCM) providers.
......@@ -203,7 +204,7 @@ To test mirroring by forcing a push, select the half-circle arrows button (hover
If **Last successful update** shows a date, you have configured mirroring correctly.
If it isn't working correctly, a red `error` tag appears and shows the error message as hover text.
### Setting up a push mirror to another GitLab instance with 2FA activated
### Set up a push mirror to another GitLab instance with 2FA activated
1. On the destination GitLab instance, create a [personal access token](../../profile/personal_access_tokens.md) with `write_repository` scope.
1. On the source GitLab instance:
......@@ -211,7 +212,7 @@ If it isn't working correctly, a red `error` tag appears and shows the error mes
1. Fill in the **Password** field with the GitLab personal access token created on the destination GitLab instance.
1. Select **Mirror repository**.
## Pulling from a remote repository **(PREMIUM)**
## Pull from a remote repository **(PREMIUM)**
> - [Added Git LFS support](https://gitlab.com/gitlab-org/gitlab/-/issues/10871) in GitLab 11.11.
> - Moved to GitLab Premium in 13.9.
......@@ -238,18 +239,12 @@ mirror pulling:
- **Only mirror protected branches**
1. Select **Mirror repository** to save the configuration.
![Repository mirroring pull settings screen - upper part](img/repository_mirroring_pull_settings_upper.png)
---
![Repository mirroring pull settings screen - lower part](img/repository_mirroring_pull_settings_lower.png)
Because GitLab is now set to pull changes from the upstream repository, you should not push commits
directly to the repository on GitLab. Instead, any commits should be pushed to the remote repository.
Changes pushed to the remote repository are pulled into the GitLab repository, either:
- Automatically in a certain period of time.
- When a [forced update](#forcing-an-update) is initiated.
- When a [forced update](#force-an-update) is initiated.
WARNING:
If you do manually update a branch in the GitLab repository, the branch becomes diverged from
......@@ -273,7 +268,7 @@ Repository mirrors are updated as Sidekiq becomes available to process them. If
### Overwrite diverged branches **(PREMIUM)**
> - Moved to GitLab Premium in 13.9.
> Moved to GitLab Premium in 13.9.
You can choose to always update your local branches with remote versions, even if they have
diverged from the remote.
......@@ -285,7 +280,7 @@ To use this option, check the **Overwrite diverged branches** box when creating
### Trigger pipelines for mirror updates **(PREMIUM)**
> - Moved to GitLab Premium in 13.9.
> Moved to GitLab Premium in 13.9.
If this option is enabled, pipelines trigger when branches or tags are
updated from the remote repository. Depending on the activity of the remote
......@@ -295,7 +290,7 @@ assigned when you set up pull mirroring.
### Hard failure **(PREMIUM)**
> - Moved to GitLab Premium in 13.9.
> Moved to GitLab Premium in 13.9.
After 14 consecutive unsuccessful retries, the mirroring process is marked as a hard failure
and mirroring attempts stop. This failure is visible in either the:
......@@ -303,11 +298,11 @@ and mirroring attempts stop. This failure is visible in either the:
- Project's main dashboard.
- Pull mirror settings page.
You can resume the project mirroring again by [forcing an update](#forcing-an-update).
You can resume the project mirroring again by [forcing an update](#force-an-update).
### Trigger an update using the API **(PREMIUM)**
> - Moved to GitLab Premium in 13.9.
> Moved to GitLab Premium in 13.9.
Pull mirroring uses polling to detect new branches and commits added upstream, often minutes
afterwards. If you notify GitLab by [API](../../../api/projects.md#start-the-pull-mirroring-process-for-a-project),
......@@ -317,7 +312,7 @@ For more information, see [Start the pull mirroring process for a Project](../..
## Mirror only protected branches **(PREMIUM)**
> - Moved to GitLab Premium in 13.9.
> Moved to GitLab Premium in 13.9.
Based on the mirror direction that you choose, you can opt to mirror only the
[protected branches](../protected_branches.md) in the mirroring project,
......@@ -329,7 +324,7 @@ creating a repository mirror. **(PREMIUM)**
## SSH authentication
> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22982) in GitLab 11.6 for Push mirroring.
> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22982) in GitLab 11.6 for Push mirroring.
SSH authentication is mutual:
......@@ -418,7 +413,7 @@ NOTE:
The generated keys are stored in the GitLab database, not in the file system. Therefore,
SSH public key authentication for mirrors cannot be used in a pre-receive hook.
## Forcing an update **(FREE)**
## Force an update **(FREE)**
While mirrors are scheduled to update automatically, you can always force an update by using the
update button which is available on the **Mirroring repositories** section of the **Repository Settings** page.
......@@ -427,7 +422,7 @@ update button which is available on the **Mirroring repositories** section of th
## Bidirectional mirroring **(PREMIUM)**
> - Moved to GitLab Premium in 13.9.
> Moved to GitLab Premium in 13.9.
WARNING:
Bidirectional mirroring may cause conflicts.
......@@ -451,13 +446,17 @@ protected branches.
### Configure a webhook to trigger an immediate pull to GitLab
Assuming you have already configured the [push](#setting-up-a-push-mirror-to-another-gitlab-instance-with-2fa-activated) and [pull](#pulling-from-a-remote-repository) mirrors in the upstream GitLab instance, to trigger an immediate pull as suggested above, you must configure a [Push Event Web Hook](../integrations/webhooks.md#push-events) in the downstream instance.
Assuming you have already configured the [push](#set-up-a-push-mirror-to-another-gitlab-instance-with-2fa-activated)
and [pull](#pull-from-a-remote-repository) mirrors in the upstream GitLab instance, to trigger an
immediate pull as suggested above, you must configure a [Push Event Web Hook](../integrations/webhooks.md#push-events)
in the downstream instance.
To do this:
1. Create a [personal access token](../../profile/personal_access_tokens.md) with `API` scope.
1. In your project, go to **Settings > Webhooks**.
1. Add the webhook URL which (in this case) uses the [Pull Mirror API](../../../api/projects.md#start-the-pull-mirroring-process-for-a-project) request to trigger an immediate pull after updates to the repository.
1. Add the webhook URL which (in this case) uses the [Pull Mirror API](../../../api/projects.md#start-the-pull-mirroring-process-for-a-project)
request to trigger an immediate pull after updates to the repository.
```plaintext
https://gitlab.example.com/api/v4/projects/:id/mirror/pull?private_token=<your_access_token>
......@@ -468,7 +467,7 @@ To do this:
To test the integration, select the **Test** button and confirm GitLab doesn't return an error message.
### Preventing conflicts using a `pre-receive` hook
### Prevent conflicts using a pre-receive hook
WARNING:
The solution proposed negatively affects the performance of
......@@ -551,7 +550,7 @@ Note that this sample has a few limitations:
- The script circumvents the Git hook quarantine environment because the update of `$TARGET_REPO`
is seen as a ref update, and Git displays warnings about it.
### Mirroring with Perforce Helix via Git Fusion **(PREMIUM)**
### Mirror with Perforce Helix via Git Fusion **(PREMIUM)**
> Moved to GitLab Premium in 13.9.
......@@ -584,15 +583,20 @@ Should an error occur during a push, GitLab displays an **Error** highlight for
### 13:Received RST_STREAM with error code 2 with GitHub
If you receive an "13:Received RST_STREAM with error code 2" while mirroring to a GitHub repository, your GitHub settings might be set to block pushes that expose your email address used in commits. Either set your email address on GitHub to be public, or disable the [Block command line pushes that expose my email](https://github.com/settings/emails) setting.
If you receive a "13:Received RST_STREAM with error code 2" message while mirroring to a GitHub repository,
your GitHub settings might be set to block pushes that expose your email address used in commits. Either
set your email address on GitHub to be public, or disable the [Block command line pushes that expose my email](https://github.com/settings/emails) setting.
### 4:Deadline Exceeded
When upgrading to GitLab 11.11.8 or newer, a change in how usernames are represented means that you may need to update your mirroring username and password to ensure that `%40` characters are replaced with `@`.
When upgrading to GitLab 11.11.8 or newer, a change in how usernames are represented means that you
may need to update your mirroring username and password to ensure that `%40` characters are replaced with `@`.
### Connection blocked because server only allows public key authentication
As the error indicates, the connection is getting blocked between GitLab and the remote repository. Even if a [TCP Check](../../../administration/raketasks/maintenance.md#check-tcp-connectivity-to-a-remote-site) is successful, you must check any networking components in the route from GitLab to the remote Server to ensure there's no blockage.
As the error indicates, the connection is getting blocked between GitLab and the remote repository. Even if a
[TCP Check](../../../administration/raketasks/maintenance.md#check-tcp-connectivity-to-a-remote-site) is successful,
you must check any networking components in the route from GitLab to the remote Server to ensure there's no blockage.
For example, we've seen this error when a Firewall was performing a `Deep SSH Inspection` on outgoing packets.
......
......@@ -3363,6 +3363,9 @@ msgstr ""
msgid "Allow owners to manually add users outside of LDAP"
msgstr ""
msgid "Allow project maintainers to configure repository mirroring"
msgstr ""
msgid "Allow projects and subgroups to override the group setting"
msgstr ""
......@@ -3378,9 +3381,6 @@ msgstr ""
msgid "Allow rendering of diagrams in AsciiDoc and Markdown documents using %{link}."
msgstr ""
msgid "Allow repository mirroring to be configured by project maintainers"
msgstr ""
msgid "Allow requests to the local network from hooks and services."
msgstr ""
......@@ -12199,9 +12199,6 @@ msgstr ""
msgid "Enable maintenance mode"
msgstr ""
msgid "Enable mirror configuration"
msgstr ""
msgid "Enable or disable the Pseudonymizer data collection."
msgstr ""
......@@ -16478,7 +16475,7 @@ msgstr ""
msgid "If disabled, a diverged local branch will not be automatically updated with commits from its remote counterpart, to prevent local data loss. If the default branch (%{default_branch}) has diverged and cannot be updated, mirroring will fail. Other diverged branches are silently ignored."
msgstr ""
msgid "If disabled, only admins will be able to configure repository mirroring."
msgid "If disabled, only administrators can configure repository mirroring."
msgstr ""
msgid "If enabled, GitLab will handle Object Storage replication using Geo. %{linkStart}More information%{linkEnd}"
......@@ -27785,6 +27782,9 @@ msgstr ""
msgid "Repository mirroring"
msgstr ""
msgid "Repository mirroring configuration"
msgstr ""
msgid "Repository must contain at least 1 file."
msgstr ""
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment