Skip to content

G
gitlab-ce
Commit b8c6eb48 authored by GitLab Bot's avatar GitLab Bot
Browse files
Options

Automatic merge of gitlab-org/gitlab master

parents 124a1020 e54e53f2
Hide whitespace changes
Inline Side-by-side
Showing with 98 additions and 153 deletions
app/models/deployment.rb
View file @ b8c6eb48
......@@ -107,10 +107,7 @@ class Deployment < ApplicationRecord
deployment.run_after_commit do
Deployments::UpdateEnvironmentWorker.perform_async(id)
Deployments::LinkMergeRequestWorker.perform_async(id)
if ::Feature.enabled?(:deployments_archive, deployment.project, default_enabled: :yaml)
Deployments::ArchiveInProjectWorker.perform_async(deployment.project_id)
end
Deployments::ArchiveInProjectWorker.perform_async(deployment.project_id)
end
end
......
app/services/deployments/archive_in_project_service.rb
View file @ b8c6eb48
......@@ -7,10 +7,6 @@ module Deployments
BATCH_SIZE = 100
def execute
unless ::Feature.enabled?(:deployments_archive, project, default_enabled: :yaml)
return error('Feature flag is not enabled')
end
deployments = Deployment.archivables_in(project, limit: BATCH_SIZE)
return success(result: :empty) if deployments.empty?
......
app/views/sherlock/transactions/show.html.haml
View file @ b8c6eb48
......@@ -8,13 +8,11 @@
%li
%a{ href: "#tab-queries", data: { toggle: "tab" } }
= t('sherlock.queries')
%span.badge.badge-pill
#{@transaction.queries.length}
= gl_badge_tag @transaction.queries.length.to_s
%li
%a{ href: "#tab-file-samples", data: { toggle: "tab" } }
= t('sherlock.file_samples')
%span.badge.badge-pill
#{@transaction.file_samples.length}
= gl_badge_tag @transaction.file_samples.length.to_s
.row-content-block
.float-right
......
config/feature_flags/development/deployments_archive.yml deleted 100644 → 0
View file @ 124a1020
---
name: deployments_archive
introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73628
rollout_issue_url: https://gitlab.com/gitlab-org/gitlab/-/issues/345027
milestone: '14.5'
type: development
group: group::release
default_enabled: true
doc/administration/docs_self_host.md
View file @ b8c6eb48
......@@ -6,18 +6,37 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# How to self-host the docs site **(FREE SELF)**
The following guide describes how to use a local instance of the docs site with
a self-managed GitLab instance.
If you have a self-managed instance of GitLab, you may not be able to access the
product documentation as hosted on `docs.gitlab.com` from your GitLab instance.
## Run the docs site
Be aware of the following items if you self-host the product documentation:
The easiest way to run the docs site locally it to pick up one of the existing
Docker images that contain the HTML files.
- You must host the product documentation site under a subdirectory that matches
your installed GitLab version (for example, `14.5/`). The
[Docker images](https://gitlab.com/gitlab-org/gitlab-docs/container_registry/631635)
hosted by the GitLab Docs team provide this by default. We use a
[script](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/2995d1378175803b22fb8806ba77adf63e79f32c/scripts/normalize-links.sh#L28-82)
to normalize the links and prefix them with the respective version.
- The version dropdown will display additional versions that don't exist, selecting
those versions will display a 404 Not Found page.
- Results when using the search box will display results from `docs.gitlab.com`
and not the local documentation.
- When you use the Docker images to serve the product documentation site, by
default the landing page redirects to the respective version (for example, `/14.5/`),
which causes the landing page <https://docs.gitlab.com> to not be displayed.
## Documentation self-hosting options
Pick the version that matches your GitLab version and run it, in the following
examples 14.5.
You can self-host the GitLab product documentation locally using one of these
methods:
### Host the docs site using Docker
- Docker
- GitLab Pages
- From your own webserver
The examples on this page are based on GitLab 14.5.
### Self-host the product documentation with Docker
The Docker images use a built-in webserver listening on port `4000`, so you need
to expose that.
......@@ -42,9 +61,9 @@ services:
- '4000:4000'
```
### Host the docs site using GitLab Pages
### Self-host the product documentation with GitLab Pages
You can also host the docs site with GitLab Pages.
You use GitLab Pages to host the GitLab product documentation locally.
Prerequisite:
......@@ -53,11 +72,11 @@ Prerequisite:
main domain or subdomain. For example, URLs like `https://example.com/docs/`
are not supported.
To host the docs site with GitLab Pages:
To host the product documentation site with GitLab Pages:
1. [Create a new blank project](../user/project/working_with_projects.md#create-a-blank-project).
1. Create a new or edit your existing `.gitlab-ci.yml` file and add the following
`pages` job. Make sure the version is the same as your GitLab installation:
1. Create a new or edit your existing `.gitlab-ci.yml` file, and add the following
`pages` job, while ensuring the version is the same as your GitLab installation:
```yaml
image: registry.gitlab.com/gitlab-org/gitlab-docs:14.5
......@@ -70,20 +89,22 @@ To host the docs site with GitLab Pages:
- public
```
1. (Optional) Set the Pages domain name. Depending on the type of the Pages website,
you have two options:
1. Optional. Set the GitLab Pages domain name. Depending on the type of the
GitLab Pages website, you have two options:
| Type of website | [Default domain](../user/project/pages/getting_started_part_one.md#gitlab-pages-default-domain-names) | [Custom domain](../user/project/pages/custom_domains_ssl_tls_certification/index.md) |
| --------------- | -------------- | ------------- |
| Type of website | [Default domain](../user/project/pages/getting_started_part_one.md#gitlab-pages-default-domain-names) | [Custom domain](../user/project/pages/custom_domains_ssl_tls_certification/index.md) |
|-------------------------|----------------|---------------|
| [Project website](../user/project/pages/getting_started_part_one.md#project-website-examples) | Not supported | Supported |
| [User or group website](../user/project/pages/getting_started_part_one.md#user-and-group-website-examples) | Supported | Supported |
### Host the docs site on your own webserver
### Self-host the product documentation on your own webserver
Because the product documentation site is static, you can grab the directory from
the container (in `/usr/share/nginx/html`) and use your own web server to host
it wherever you want.
Since the docs site is static, you can grab the directory from the container
(under `/usr/share/nginx/html`) and use your own web server to host
it wherever you want. Replace `<destination>` with the directory where the
docs will be copied to:
Use the following commands, and replace `<destination>` with the directory where the
documentation files will be copied to:
```shell
docker create -it --name gitlab-docs registry.gitlab.com/gitlab-org/gitlab-docs:14.5
......@@ -93,32 +114,18 @@ docker rm -f gitlab-docs
## Redirect the `/help` links to the new docs page
When the docs site is up and running:
1. [Enable the help page redirects](../user/admin_area/settings/help_page.md#redirect-help-pages).
Use the Fully Qualified Domain Name as the docs URL. For example, if you
used the [Docker method](#host-the-docs-site-using-docker) , enter `http://0.0.0.0:4000`.
You don't need to append the version, it is detected automatically.
1. Test that everything works by selecting the **Learn more** link on the page
you're on. Your GitLab version is automatically detected and appended to the docs URL
you set in the admin area. In this example, if your GitLab version is 14.5,
`https://<instance_url>/` becomes `http://0.0.0.0:4000/14.5/`.
The link inside GitLab link shows as
`<instance_url>/help/user/admin_area/settings/help_page#destination-requirements`,
but when you select it, you are redirected to
`http://0.0.0.0:4000/14.5/ee/user/admin_area/settings/help_page/#destination-requirements`.
## Caveats
- You need to host the docs site under a subdirectory matching your GitLab version,
in the example of this guide `14.5/`. The
[Docker images](https://gitlab.com/gitlab-org/gitlab-docs/container_registry/631635)
hosted by the Docs team provide this by default. We use a
[script](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/2995d1378175803b22fb8806ba77adf63e79f32c/scripts/normalize-links.sh#L28-82)
to normalize the links and prefix them with the respective version.
- The version dropdown will show more versions which do not exist and will lead
to 404 if selected.
- The search results point to `docs.gitlab.com` and not the local docs.
- When you use the Docker images to serve the docs site, the landing page redirects
by default to the respective version, for example `/14.5/`, so you don't
see the landing page as seen at <https://docs.gitlab.com>.
After your local product documentation site is running, [redirect the help
links](../user/admin_area/settings/help_page.md#redirect-help-pages) in the GitLab
application to your local site.
Be sure to use the fully qualified domain name as the docs URL. For example, if you
used the [Docker method](#self-host-the-product-documentation-with-docker), enter `http://0.0.0.0:4000`.
You don't need to append the version, as GitLab will detect it and append it to
any documentation URL requests, as needed. For example, if your GitLab version is
14.5, the GitLab Docs URL becomes `http://0.0.0.0:4000/14.5/`. The link
inside GitLab displays as `<instance_url>/help/user/admin_area/settings/help_page#destination-requirements`,
but when you select it, you are redirected to
`http://0.0.0.0:4000/14.5/ee/user/admin_area/settings/help_page/#destination-requirements`.
To test the setting, select a **Learn more** link within the GitLab application.
doc/ci/environments/index.md
View file @ b8c6eb48
......@@ -774,10 +774,8 @@ fetch = +refs/environments/*:refs/remotes/origin/environments/*
### Archive Old Deployments
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73628) in GitLab 14.5.
> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/337507) in GitLab 14.6.
FLAG:
On self-managed GitLab, by default this feature is available. To hide the feature per project or for your entire instance, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `deployments_archive`. On GitLab.com, this feature is available.
> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/345027) in GitLab 14.6.
> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73628) in GitLab 14.0. [Feature flag `deployments_archive`](https://gitlab.com/gitlab-org/gitlab/-/issues/345027) removed.
When a new deployment happens in your project,
GitLab creates [a special Git-ref to the deployment](#check-out-deployments-locally).
......
doc/development/background_migrations.md
View file @ b8c6eb48
......@@ -83,23 +83,11 @@ replacing the class name and arguments with whatever values are necessary for
your migration:
```ruby
migrate_async('BackgroundMigrationClassName', [arg1, arg2, ...])
migrate_in('BackgroundMigrationClassName', [arg1, arg2, ...])
```
Usually it's better to enqueue jobs in bulk, for this you can use
`bulk_migrate_async`:
```ruby
bulk_migrate_async(
[['BackgroundMigrationClassName', [1]],
['BackgroundMigrationClassName', [2]]]
)
```
Note that this will queue a Sidekiq job immediately: if you have a large number
of records, this may not be what you want. You can use the function
`queue_background_migration_jobs_by_range_at_intervals` to split the job into
batches:
You can use the function `queue_background_migration_jobs_by_range_at_intervals`
to automatically split the job into batches:
```ruby
queue_background_migration_jobs_by_range_at_intervals(
......@@ -117,16 +105,6 @@ consuming migrations it's best to schedule a background job using an
updates. Removals in turn can be handled by simply defining foreign keys with
cascading deletes.
If you would like to schedule jobs in bulk with a delay, you can use
`BackgroundMigrationWorker.bulk_perform_in`:
```ruby
jobs = [['BackgroundMigrationClassName', [1]],
['BackgroundMigrationClassName', [2]]]
bulk_migrate_in(5.minutes, jobs)
```
### Rescheduling background migrations
If one of the background migrations contains a bug that is fixed in a patch
......@@ -197,53 +175,47 @@ the new format.
## Example
To explain all this, let's use the following example: the table `services` has a
To explain all this, let's use the following example: the table `integrations` has a
field called `properties` which is stored in JSON. For all rows you want to
extract the `url` key from this JSON object and store it in the `services.url`
column. There are millions of services and parsing JSON is slow, thus you can't
extract the `url` key from this JSON object and store it in the `integrations.url`
column. There are millions of integrations and parsing JSON is slow, thus you can't
do this in a regular migration.
To do this using a background migration we'll start with defining our migration
class:
```ruby
class Gitlab::BackgroundMigration::ExtractServicesUrl
class Service < ActiveRecord::Base
self.table_name = 'services'
class Gitlab::BackgroundMigration::ExtractIntegrationsUrl
class Integration < ActiveRecord::Base
self.table_name = 'integrations'
end
def perform(service_id)
# A row may be removed between scheduling and starting of a job, thus we
# need to make sure the data is still present before doing any work.
service = Service.select(:properties).find_by(id: service_id)
def perform(start_id, end_id)
Integration.where(id: start_id..end_id).each do |integration|
json = JSON.load(integration.properties)
return unless service
begin
json = JSON.load(service.properties)
integration.update(url: json['url']) if json['url']
rescue JSON::ParserError
# If the JSON is invalid we don't want to keep the job around forever,
# instead we'll just leave the "url" field to whatever the default value
# is.
return
next
end
service.update(url: json['url']) if json['url']
end
end
```
Next we'll need to adjust our code so we schedule the above migration for newly
created and updated services. We can do this using something along the lines of
created and updated integrations. We can do this using something along the lines of
the following:
```ruby
class Service < ActiveRecord::Base
after_commit :schedule_service_migration, on: :update
after_commit :schedule_service_migration, on: :create
class Integration < ActiveRecord::Base
after_commit :schedule_integration_migration, on: :update
after_commit :schedule_integration_migration, on: :create
def schedule_service_migration
BackgroundMigrationWorker.perform_async('ExtractServicesUrl', [id])
def schedule_integration_migration
BackgroundMigrationWorker.perform_async('ExtractIntegrationsUrl', [id, id])
end
end
```
......@@ -253,21 +225,20 @@ before the transaction completes as doing so can lead to race conditions where
the changes are not yet visible to the worker.
Next we'll need a post-deployment migration that schedules the migration for
existing data. Since we're dealing with a lot of rows we'll schedule jobs in
batches instead of doing this one by one:
existing data.
```ruby
class ScheduleExtractServicesUrl < Gitlab::Database::Migration[1.0]
class ScheduleExtractIntegrationsUrl < Gitlab::Database::Migration[1.0]
disable_ddl_transaction!
def up
define_batchable_model('services').select(:id).in_batches do |relation|
jobs = relation.pluck(:id).map do |id|
['ExtractServicesUrl', [id]]
end
MIGRATION = 'ExtractIntegrationsUrl'
DELAY_INTERVAL = 2.minutes
BackgroundMigrationWorker.bulk_perform_async(jobs)
end
def up
queue_background_migration_jobs_by_range_at_intervals(
define_batchable_model('integrations'),
MIGRATION,
DELAY_INTERVAL)
end
def down
......@@ -284,18 +255,18 @@ jobs and manually run on any un-migrated rows. Such a migration would look like
this:
```ruby
class ConsumeRemainingExtractServicesUrlJobs < Gitlab::Database::Migration[1.0]
class ConsumeRemainingExtractIntegrationsUrlJobs < Gitlab::Database::Migration[1.0]
disable_ddl_transaction!
def up
# This must be included
Gitlab::BackgroundMigration.steal('ExtractServicesUrl')
Gitlab::BackgroundMigration.steal('ExtractIntegrationsUrl')
# This should be included, but can be skipped - see below
define_batchable_model('services').where(url: nil).each_batch(of: 50) do |batch|
define_batchable_model('integrations').where(url: nil).each_batch(of: 50) do |batch|
range = batch.pluck('MIN(id)', 'MAX(id)').first
Gitlab::BackgroundMigration::ExtractServicesUrl.new.perform(*range)
Gitlab::BackgroundMigration::ExtractIntegrationsUrl.new.perform(*range)
end
end
......@@ -313,9 +284,9 @@ If the application does not depend on the data being 100% migrated (for
instance, the data is advisory, and not mission-critical), then this final step
can be skipped.
This migration will then process any jobs for the ExtractServicesUrl migration
This migration will then process any jobs for the ExtractIntegrationsUrl migration
and continue once all jobs have been processed. Once done you can safely remove
the `services.properties` column.
the `integrations.properties` column.
## Testing
......
doc/user/clusters/agent/repository.md
View file @ b8c6eb48
......@@ -12,9 +12,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w
> - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) from GitLab Premium to GitLab Free in 14.5.
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332227) in GitLab 14.0, the `resource_inclusions` and `resource_exclusions` attributes were removed and `reconcile_timeout`, `dry_run_strategy`, `prune`, `prune_timeout`, `prune_propagation_policy`, and `inventory_policy` attributes were added.
WARNING:
This feature might not be available to you. Check the **version history** note above for details.
The [GitLab Agent](index.md) supports hosting your configuration for
multiple agents in a single repository. These agents can be running
in the same cluster or in multiple clusters, and potentially with more than one agent per cluster.
......
spec/services/deployments/archive_in_project_service_spec.rb
View file @ b8c6eb48
......@@ -50,17 +50,6 @@ RSpec.describe Deployments::ArchiveInProjectService do
end
end
context 'when deployments_archive feature flag is disabled' do
before do
stub_feature_flags(deployments_archive: false)
end
it 'does not do anything' do
expect(subject[:status]).to eq(:error)
expect(subject[:message]).to eq('Feature flag is not enabled')
end
end
def deployment_refs_exist?
deployment_refs.map { |path| project.repository.ref_exists?(path) }
end
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7