Commit 68f1860e authored by GitLab Bot's avatar GitLab Bot

Add latest changes from gitlab-org/gitlab@master

parent 1af0d38d
Please view this file on the master branch, on stable branches it's out of date. Please view this file on the master branch, on stable branches it's out of date.
## 12.9.0 (2020-03-22)
### Removed (1 change)
- Remove Security Dashboard button from project page. !24781
### Fixed (20 changes)
- Ensure User IDs are Removed When Moving to All Users Strategy. !25190
- Fix dismissed vulnerability comment updates. !25328
- Generate illustrations URLs in the backend. !25375
- Check user role for each of namespaces while deciding if it is using a paid seat in it. !25437
- Change MR approval popover to use hover trigger. !25494
- Change copy of Welcome screen. !25526
- Fix weight not showing in linked issues. !25718
- Scope name uniqueness of approval rules to type. !25875
- Pass correct Time object to Value Stream Management DataCollector. !25885
- API for Namespace members has correct boolean if user using paid seat. !25988
- Set the default SSL cert path environment variables when invoking indexer. !26036
- Add pagination to Instance Level Security Dashboard project selector. !26138
- Exclude creation event from audit changes. !26140
- Resolve Usage Ping timeouts on security_products_usage for large hosts. !26296
- Fix SCIM API not creating new users. !26514
- BulkIndexer change _routing -> routing to support ES 7. !26639
- Ensure design events are correctly visible. !26708
- Add support for async search within Epics dropdown. !26980
- Fix configuration issue for License Compliance. !27016
- Geo - Fix ambiguous reference column while loading migrated local files to clean up. !27252
### Changed (22 changes, 1 of them is from the community)
- Add auto_renew attribute to gitlab_subscriptions and expose via api. !24896 (jejacks0n)
- Remove admin user count from Users statistics. !25337
- Improve error messages in the Dependency List. !25369
- Do not automatically add approvers as participants of each merge request. !25546
- Improve loading UX in the License Management list. !25620
- Improve audit log & events header layout. !25821
- Adjust the MR approval rule to use a smaller modal. !25955
- Resolve Code review start time doesnt account for approvals. !25997
- Prevent 'Invite group' for groups outside a group-managed account group. !26081
- Improve language when MR Security Report is outdated. !26150
- Prevent projects from being shared outside a group with managed accounts. !26163
- Prevent projects from being shared outside a group with managed accounts for forked projects. !26186
- Extract time range information to be reused. !26431
- Restrict inviting outside group to a project forked from a group with enforced SSO. !26456
- Add severity badge to security reports. !26715
- Consider MR reassign as code review start. !26891
- Add application limits for Ci::PipelineSchedule. !27004
- Update license compliance policy classifications text in the UI. !27058
- Remove approval metric recalculation when approval was removed. !27071
- Move Value Stream Analytics to the group level. !27304
- Remove release tag filter from Issues Analytics. !208903
- Change dependency & license scan pipeline text message.
### Performance (4 changes)
- Optimize service desk issue counters query performance in usage data. !26445
- Preload database records in bulk indexing. !26754
- Geo: Make the LFS registry the SSOT to optimize query performance. !27154
- Remove feature flag elastic_bulk_incremental_updates. !27293
### Added (58 changes, 9 of them are from the community)
- Audit number of required approvals changes. !7531
- Add weight and progress information in Roadmap Epic bars. !18957
- Adds the ability to dismiss multiple vulnerabilities. !21480
- Add code quality tab to pipeline view. !21527
- Add milestones to roadmap. !22748
- Add Threat Monitoring page. !22911
- Add epic in filtered search. !22958
- Add deep links for cycle analytics. !23493
- Allow existing users to transfer their account to SAML Group managed accounts. !24329
- Use snowplow for user error reporting in trial. !24430
- Time period for usage activity by stage in usage data. !24431
- Adds sorting to group packages api. !24432
- Export Board Milestones in Group Export. !24606
- Add possibility to search for epics by partial iids using GraphQL. !24673
- Add restrict outer forks functionality for group SAML. !24698
- Release Jira connect feature. !24818
- Adds project_path sorting to group package API. !24833
- Add webhook limits for groups. !25129
- Default design images to 100% of viewport width. !25327
- Add npm_package_requests_forwarding to application settings. !25346
- Add bot count to Users statistics in Admin Area. !25355
- Adding CODEOWNER approval metrics for merge requests. !25386
- Separate 4 ee entities into own class file. !25400 (Rajendra Kadam)
- Improve zoom reset after resizing window in Design Management. !25406
- Introduce Project level issues analytics. !25417
- Add tooltip to guests count in Users Statistics. !25433
- Separate epic, epic_issue, linked epic ee entities into own class file. !25592 (Rajendra Kadam)
- Add API endpoints for Group Activity Analytics. !25606
- Add GraphQL mutation to set the epic of an issue. !25628
- Add merge request metrics first approved at timestamp. !25681
- Separate ee entities into own class file. !25709 (Rajendra Kadam)
- Embed cluster health metrics in GitLab-flavored Markdown. !25739
- Always show the Design Tab on Issues but indicate user if feature is not available. !25796
- Status page settings. !25820
- Audit user blocked via API. !25872
- Create a vulnerability-list component. !25927
- Audit project MR approval permission changes. !25959
- Add health status data to the epic GraphQL endpoint. !25989
- Display correct approval rules based on target branch in Edit MR form. !26053
- Activate triggering webhooks from a subgroup feature. !26070
- Separate vulnerabilities, feature flags, dependency entities into own class files. !26088 (Rajendra Kadam)
- Support drag-and-drop design uploads in Design Management. !26139
- Add distinct user count of secure stage items to usage ping. !26194
- Support drag-and-drop on existing designs in Design Management. !26446
- Add worker to sync paid seats info daily in the background. !26467
- Separate project alias, build info, package entities into own class files. !26493 (Rajendra Kadam)
- Support async loading & search of projects within Epics Tree. !26661
- Add code review reassign metric structure. !26664
- Separate npm, subs and geo entities into own class file. !26674 (Rajendra Kadam)
- Separate geo_node and license entities into own class file. !26685 (Rajendra Kadam)
- Separate merge request approval entities into own class files. !26860 (Rajendra Kadam)
- Include snippet description as part of snippet title search (advanced search, Elasticsearch reindexing required). !26870
- Add Roadmap GraphQL and unfiltered epic aggregates. !26919
- Add NPM package requests forwarding to npmjs.org. !26932
- Separate approval entities into own class files. !27336 (Rajendra Kadam)
- View description changes history of issues, epics and merge requests. !27445
- Enable license compliance policies tab for on prem installs.
- Add a bulk processor for ES incremental updates.
### Other (5 changes, 1 of them is from the community)
- Refactor WeightSelect to use gitlab-ui dropdown. !20770
- Update to new severity badge design. !25489
- Rename close_{at,by} to dismissed_{at,by} in Vulnerabilities. !25609 (Michał Zając)
- Style MR approval popover link to be consistent text size. !25657
- Allow users to be marked as service users. !202680
## 12.8.7 (2020-03-16) ## 12.8.7 (2020-03-16)
### Fixed (1 change) ### Fixed (1 change)
......
...@@ -56,7 +56,7 @@ export default { ...@@ -56,7 +56,7 @@ export default {
</gl-link> </gl-link>
</div> </div>
</div> </div>
<div class="blob-viewer"> <div class="blob-viewer" data-qa-selector="blob_viewer_content">
<gl-loading-icon v-if="loading > 0" size="md" color="dark" class="my-4 mx-auto" /> <gl-loading-icon v-if="loading > 0" size="md" color="dark" class="my-4 mx-auto" />
<div v-else-if="readme" ref="readme" v-html="readme.html"></div> <div v-else-if="readme" ref="readme" v-html="readme.html"></div>
</div> </div>
......
...@@ -434,8 +434,50 @@ class MergeRequestDiff < ApplicationRecord ...@@ -434,8 +434,50 @@ class MergeRequestDiff < ApplicationRecord
merge_request_diff_files.reset merge_request_diff_files.reset
end end
# Transactionally migrate the current merge_request_diff_files entries from
# external storage, back to the database. This is the rollback operation for
# +migrate_files_to_external_storage!+
#
# If this diff isn't in external storage, the method is a no-op.
def migrate_files_to_database!
return unless stored_externally?
return if merge_request_diff_files.count == 0
rows = convert_external_diffs_to_database
transaction do
MergeRequestDiffFile.where(merge_request_diff_id: id).delete_all
Gitlab::Database.bulk_insert('merge_request_diff_files', rows)
update!(stored_externally: false)
end
# Only delete the external diff file after the contents have been saved to
# the database
remove_external_diff!
merge_request_diff_files.reset
end
private private
def convert_external_diffs_to_database
opening_external_diff do |external_file|
merge_request_diff_files.map do |diff_file|
row = diff_file.attributes.except('diff')
raise "Diff file lacks external diff offset or size: #{row.inspect}" unless
row['external_diff_offset'] && row['external_diff_size']
# The diff in the external file is already base64-encoded if necessary,
# matching the 'binary' attribute of the row. Reading it directly allows
# a cycle of decode-encode to be skipped
external_file.seek(row.delete('external_diff_offset'))
row['diff'] = external_file.read(row.delete('external_diff_size'))
row
end
end
end
def diffs_in_batch_collection(batch_page, batch_size, diff_options:) def diffs_in_batch_collection(batch_page, batch_size, diff_options:)
Gitlab::Diff::FileCollection::MergeRequestDiffBatch.new(self, Gitlab::Diff::FileCollection::MergeRequestDiffBatch.new(self,
batch_page, batch_page,
......
...@@ -7,7 +7,7 @@ ...@@ -7,7 +7,7 @@
= button_tag class: 'toggle-mobile-nav', type: 'button' do = button_tag class: 'toggle-mobile-nav', type: 'button' do
%span.sr-only= _("Open sidebar") %span.sr-only= _("Open sidebar")
= icon ('bars') = icon ('bars')
.breadcrumbs-links.js-title-container .breadcrumbs-links.js-title-container{ data: { qa_selector: 'breadcrumb_links_content' } }
%ul.list-unstyled.breadcrumbs-list.js-breadcrumbs-list %ul.list-unstyled.breadcrumbs-list.js-breadcrumbs-list
- unless hide_top_links - unless hide_top_links
= header_title = header_title
......
---
title: Allow external diff files to be removed
merge_request: 27602
author:
type: changed
...@@ -67,6 +67,8 @@ in your SAML IdP: ...@@ -67,6 +67,8 @@ in your SAML IdP:
auto_link_saml_user: true auto_link_saml_user: true
``` ```
1. Ensure that the SAML [`NameID`](../user/group/saml_sso/index.md#nameid) and email address are fixed for each user, as described in the section on [Security](#security). Otherwise, your users will be able to sign in as other authorized users.
1. Add the provider configuration: 1. Add the provider configuration:
For Omnibus package: For Omnibus package:
...@@ -430,6 +432,8 @@ args: { ...@@ -430,6 +432,8 @@ args: {
} }
``` ```
Make sure you read the [Security](#security) section before changing this value.
## Response signature validation (required) ## Response signature validation (required)
We require Identity Providers to sign SAML responses to ensure that the assertions are We require Identity Providers to sign SAML responses to ensure that the assertions are
...@@ -544,6 +548,17 @@ args: { ...@@ -544,6 +548,17 @@ args: {
GitLab will sign the request with the provided private key. GitLab will include the configured public x500 certificate in the metadata for your Identity Provider to validate the signature of the received request with. For more information on this option, see the [Ruby SAML gem documentation](https://github.com/onelogin/ruby-saml/tree/v1.7.0). The Ruby SAML gem is used by the [OmniAuth SAML gem](https://github.com/omniauth/omniauth-saml) to implement the client side of the SAML authentication. GitLab will sign the request with the provided private key. GitLab will include the configured public x500 certificate in the metadata for your Identity Provider to validate the signature of the received request with. For more information on this option, see the [Ruby SAML gem documentation](https://github.com/onelogin/ruby-saml/tree/v1.7.0). The Ruby SAML gem is used by the [OmniAuth SAML gem](https://github.com/omniauth/omniauth-saml) to implement the client side of the SAML authentication.
## Security
Avoid user control of the following attributes:
- [`*NameID*`](../user/group/saml_sso/index.md#nameid)
- *Email* when used with `omniauth_auto_link_saml_user`
These attributes define the SAML user. If users can change these attributes, they can impersonate others.
Refer to the documentation for your [SAML Identity Provider](../user/group/saml_sso/index.md#providers) for information on how to fix these attributes.
## Troubleshooting ## Troubleshooting
### GitLab+SAML Testing Environments ### GitLab+SAML Testing Environments
......
...@@ -98,7 +98,10 @@ module Gitlab ...@@ -98,7 +98,10 @@ module Gitlab
def check_single_change_access(change) def check_single_change_access(change)
Checks::SnippetCheck.new(change, logger: logger).validate! Checks::SnippetCheck.new(change, logger: logger).validate!
Checks::PushFileCountCheck.new(change, repository: repository, limit: Snippet::MAX_FILE_COUNT, logger: logger).validate!
if Feature.enabled?(:snippet_count_check)
Checks::PushFileCountCheck.new(change, repository: repository, limit: Snippet::MAX_FILE_COUNT, logger: logger).validate!
end
rescue Checks::TimedLogger::TimeoutError rescue Checks::TimedLogger::TimeoutError
raise TimeoutError, logger.full_message raise TimeoutError, logger.full_message
end end
......
# frozen_string_literal: true # frozen_string_literal: true
require 'shellwords'
module Gitlab module Gitlab
module SidekiqCluster module SidekiqCluster
# The signals that should terminate both the master and workers. # The signals that should terminate both the master and workers.
...@@ -73,9 +75,9 @@ module Gitlab ...@@ -73,9 +75,9 @@ module Gitlab
counts = count_by_queue(queues) counts = count_by_queue(queues)
cmd = %w[bundle exec sidekiq] cmd = %w[bundle exec sidekiq]
cmd << "-c #{self.concurrency(queues, min_concurrency, max_concurrency)}" cmd << "-c#{self.concurrency(queues, min_concurrency, max_concurrency)}"
cmd << "-e#{env}" cmd << "-e#{env}"
cmd << "-gqueues: #{proc_details(counts)}" cmd << "-gqueues:#{proc_details(counts)}"
cmd << "-r#{directory}" cmd << "-r#{directory}"
counts.each do |queue, count| counts.each do |queue, count|
...@@ -83,7 +85,7 @@ module Gitlab ...@@ -83,7 +85,7 @@ module Gitlab
end end
if dryrun if dryrun
puts "Sidekiq command: #{cmd}" # rubocop:disable Rails/Output puts Shellwords.join(cmd) # rubocop:disable Rails/Output
return return
end end
...@@ -112,7 +114,7 @@ module Gitlab ...@@ -112,7 +114,7 @@ module Gitlab
else else
"#{queue} (#{count})" "#{queue} (#{count})"
end end
end.join(', ') end.join(',')
end end
def self.concurrency(queues, min_concurrency, max_concurrency) def self.concurrency(queues, min_concurrency, max_concurrency)
......
...@@ -64,7 +64,9 @@ module Gitlab ...@@ -64,7 +64,9 @@ module Gitlab
'No queues found, you must select at least one queue' 'No queues found, you must select at least one queue'
end end
@logger.info("Starting cluster with #{queue_groups.length} processes") unless @dryrun
@logger.info("Starting cluster with #{queue_groups.length} processes")
end
@processes = SidekiqCluster.start( @processes = SidekiqCluster.start(
queue_groups, queue_groups,
......
...@@ -385,6 +385,7 @@ module QA ...@@ -385,6 +385,7 @@ module QA
# Classes describing components that are used by several pages. # Classes describing components that are used by several pages.
# #
module Component module Component
autoload :Breadcrumbs, 'qa/page/component/breadcrumbs'
autoload :CiBadgeLink, 'qa/page/component/ci_badge_link' autoload :CiBadgeLink, 'qa/page/component/ci_badge_link'
autoload :ClonePanel, 'qa/page/component/clone_panel' autoload :ClonePanel, 'qa/page/component/clone_panel'
autoload :LazyLoader, 'qa/page/component/lazy_loader' autoload :LazyLoader, 'qa/page/component/lazy_loader'
......
# frozen_string_literal: true
module QA
module Page
module Component
module Breadcrumbs
def self.included(base)
base.view 'app/views/layouts/nav/_breadcrumbs.html.haml' do
element :breadcrumb_links_content
end
end
def has_breadcrumb?(text)
has_element?(:breadcrumb_links_content, text: text)
end
end
end
end
end
...@@ -5,8 +5,13 @@ module QA ...@@ -5,8 +5,13 @@ module QA
module Project module Project
class Show < Page::Base class Show < Page::Base
include Page::Component::ClonePanel include Page::Component::ClonePanel
include Page::Component::Breadcrumbs
include Page::Project::SubMenus::Settings include Page::Project::SubMenus::Settings
view 'app/assets/javascripts/repository/components/preview/index.vue' do
element :blob_viewer_content
end
view 'app/assets/javascripts/repository/components/table/row.vue' do view 'app/assets/javascripts/repository/components/table/row.vue' do
element :file_name_link element :file_name_link
end end
...@@ -90,7 +95,7 @@ module QA ...@@ -90,7 +95,7 @@ module QA
def click_file(filename) def click_file(filename)
within_element(:file_tree_table) do within_element(:file_tree_table) do
click_on filename click_element(:file_name_link, text: filename)
end end
end end
...@@ -115,6 +120,10 @@ module QA ...@@ -115,6 +120,10 @@ module QA
has_element?(:project_name_content, text: name) has_element?(:project_name_content, text: name)
end end
def has_readme_content?(text)
has_element?(:blob_viewer_content, text: text)
end
def last_commit_content def last_commit_content
find_element(:commit_content).text find_element(:commit_content).text
end end
......
...@@ -47,8 +47,8 @@ module QA ...@@ -47,8 +47,8 @@ module QA
Page::Project::Settings::Main.perform(&:click_project) Page::Project::Settings::Main.perform(&:click_project)
Page::Project::Show.perform do |project| Page::Project::Show.perform do |project|
expect(project).to have_text(target_group.path) expect(project).to have_breadcrumb(target_group.path)
expect(project).to have_text(edited_readme_content) expect(project).to have_readme_content(edited_readme_content)
end end
end end
end end
......
...@@ -60,12 +60,15 @@ module QA ...@@ -60,12 +60,15 @@ module QA
end end
end end
project.visit!
project.wait_for_push_new_branch project.wait_for_push_new_branch
project.visit!
expect(page).to have_content(file_name)
expect(page).to have_content(file_content)
expect(git_protocol_reported).to eq(git_protocol) expect(git_protocol_reported).to eq(git_protocol)
Page::Project::Show.perform do |show|
expect(show).to have_file(file_name)
expect(show).to have_readme_content(file_content)
end
end end
end end
end end
......
...@@ -24,8 +24,10 @@ module QA ...@@ -24,8 +24,10 @@ module QA
Page::Project::Show.perform(&:wait_for_viewers_to_load) Page::Project::Show.perform(&:wait_for_viewers_to_load)
expect(page).to have_content('README.md') Page::Project::Show.perform do |project|
expect(page).to have_content('This is a test project') expect(project).to have_file('README.md')
expect(project).to have_readme_content('This is a test project')
end
end end
end end
end end
......
...@@ -12,8 +12,10 @@ module QA ...@@ -12,8 +12,10 @@ module QA
push.commit_message = 'Add README.md' push.commit_message = 'Add README.md'
end.project.visit! end.project.visit!
expect(page).to have_content('README.md') Page::Project::Show.perform do |project|
expect(page).to have_content('This is a test project') expect(project).to have_file('README.md')
expect(project).to have_readme_content('This is a test project')
end
end end
end end
end end
......
# frozen_string_literal: true # frozen_string_literal: true
require 'spec_helper' require 'spec_helper'
require 'shellwords'
describe 'bin/sidekiq-cluster' do describe 'bin/sidekiq-cluster' do
using RSpec::Parameterized::TableSyntax using RSpec::Parameterized::TableSyntax
...@@ -18,9 +19,9 @@ describe 'bin/sidekiq-cluster' do ...@@ -18,9 +19,9 @@ describe 'bin/sidekiq-cluster' do
output, status = Gitlab::Popen.popen(cmd, Rails.root.to_s) output, status = Gitlab::Popen.popen(cmd, Rails.root.to_s)
expect(status).to be(0) expect(status).to be(0)
expect(output).to include('"bundle", "exec", "sidekiq"') expect(output).to include('bundle exec sidekiq')
expect(output).to include(included) expect(Shellwords.split(output)).to include(included)
expect(output).not_to include(excluded) expect(Shellwords.split(output)).not_to include(excluded)
end end
end end
end end
...@@ -36,9 +37,9 @@ describe 'bin/sidekiq-cluster' do ...@@ -36,9 +37,9 @@ describe 'bin/sidekiq-cluster' do
output, status = Gitlab::Popen.popen(cmd, Rails.root.to_s) output, status = Gitlab::Popen.popen(cmd, Rails.root.to_s)
expect(status).to be(0) expect(status).to be(0)
expect(output).to include('"bundle", "exec", "sidekiq"') expect(output).to include('bundle exec sidekiq')
expect(output).to include('-qdefault,1') expect(Shellwords.split(output)).to include('-qdefault,1')
expect(output).to include('-qcronjob:ci_archive_traces_cron,1') expect(Shellwords.split(output)).to include('-qcronjob:ci_archive_traces_cron,1')
end end
end end
end end
......
...@@ -27,6 +27,7 @@ exports[`Repository file preview component renders file HTML 1`] = ` ...@@ -27,6 +27,7 @@ exports[`Repository file preview component renders file HTML 1`] = `
<div <div
class="blob-viewer" class="blob-viewer"
data-qa-selector="blob_viewer_content"
> >
<div> <div>
<div <div
......
...@@ -209,6 +209,16 @@ describe Gitlab::GitAccessSnippet do ...@@ -209,6 +209,16 @@ describe Gitlab::GitAccessSnippet do
expect { push_access_check }.to raise_forbidden('foo') expect { push_access_check }.to raise_forbidden('foo')
end end
context 'when feature flag :snippet_count_check is disabled' do
it 'does not check push file count' do
stub_feature_flags(snippet_count_check: false)
expect(Gitlab::Checks::PushFileCountCheck).not_to receive(:new)
expect { push_access_check }.not_to raise_error
end
end
end end
private private
......
...@@ -156,6 +156,41 @@ describe MergeRequestDiff do ...@@ -156,6 +156,41 @@ describe MergeRequestDiff do
end end
end end
describe '#migrate_files_to_database!' do
let(:diff) { create(:merge_request).merge_request_diff }
it 'converts from external to in-database storage' do
stub_external_diffs_setting(enabled: true)
expect(diff).to be_stored_externally
expect(diff).to receive(:update!).and_call_original
file = diff.external_diff
file_data = file.read
diff.migrate_files_to_database!
expect(diff).not_to be_stored_externally
expect(file).not_to exist
expect(diff.merge_request_diff_files.map(&:diff).join('')).to eq(file_data)
end
it 'does nothing with an in-database diff' do
expect(diff).not_to be_stored_externally
expect(diff).not_to receive(:update!)
diff.migrate_files_to_database!
end
it 'does nothing with an empty diff' do
stub_external_diffs_setting(enabled: true)
MergeRequestDiffFile.where(merge_request_diff_id: diff.id).delete_all
expect(diff).not_to receive(:update!)
diff.migrate_files_to_database!
end
end
describe '#latest?' do describe '#latest?' do
let!(:mr) { create(:merge_request, :with_diffs) } let!(:mr) { create(:merge_request, :with_diffs) }
let!(:first_diff) { mr.merge_request_diff } let!(:first_diff) { mr.merge_request_diff }
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment