Skip to content

G
gitlab-ce
Commit a40517e8 authored by Nikola Milojevic's avatar Nikola Milojevic
Browse files
Options

Merge branch '335331-ensure-experiment-is-in-gon-object' into 'master' 

Ensure trial status popover events include gitlab_experiment context

See merge request gitlab-org/gitlab!69327
parents e31a494e 708d7fd9
Show whitespace changes
Inline Side-by-side
Showing with 227 additions and 97 deletions
app/assets/javascripts/experimentation/utils.js
View file @ a40517e8
......@@ -3,7 +3,12 @@ import { get } from 'lodash';
import { DEFAULT_VARIANT, CANDIDATE_VARIANT, TRACKING_CONTEXT_SCHEMA } from './constants';
function getExperimentsData() {
return get(window, ['gon', 'experiment'], {});
// Pull from deprecated window.gon.experiment
const experimentsFromGon = get(window, ['gon', 'experiment'], {});
// Pull from preferred window.gl.experiments
const experimentsFromGl = get(window, ['gl', 'experiments'], {});
return { ...experimentsFromGon, ...experimentsFromGl };
}
function convertExperimentDataToExperimentContext(experimentData) {
......
app/views/layouts/_published_experiments.html.haml 0 → 100644
View file @ a40517e8
= javascript_tag(nonce: content_security_policy_nonce) do
:plain
gl = window.gl || {};
gl.experiments = #{raw ApplicationExperiment.published_experiments.reject { |name, data| data[:excluded] }.to_json};
app/views/layouts/application.html.haml
View file @ a40517e8
......@@ -16,4 +16,5 @@
= render 'layouts/img_loader'
= render 'layouts/published_experiments'
= yield :scripts_body
doc/development/experiment_guide/gitlab_experiment.md
View file @ a40517e8
......@@ -92,7 +92,7 @@ end
```
When this code executes, the experiment is run, a variant is assigned, and (if within a
controller or view) a `window.gon.experiment.pill_color` object will be available in the
controller or view) a `window.gl.experiments.pill_color` object will be available in the
client layer, with details like:
- The assigned variant.
......@@ -522,14 +522,14 @@ shared example: [tracks assignment and records the subject](https://gitlab.com/g
This is in flux as of GitLab 13.10, and can't be documented just yet.
Any experiment that's been run in the request lifecycle surfaces in `window.gon.experiment`,
Any experiment that's been run in the request lifecycle surfaces in and `window.gl.experiments`,
and matches [this schema](https://gitlab.com/gitlab-org/iglu/-/blob/master/public/schemas/com.gitlab/gitlab_experiment/jsonschema/1-0-0)
so you can use it when resolving some concepts around experimentation in the client layer.
### Use experiments in Vue
With the `gitlab-experiment` component, you can define slots that match the name of the
variants pushed to `window.gon.experiment`. For example, if we alter the `pill_color`
variants pushed to `window.gl.experiments`. For example, if we alter the `pill_color`
experiment to just use the default variants of `control` and `candidate` like so:
```ruby
......@@ -587,7 +587,51 @@ For example, the Vue component for the previously-defined `pill_color` experimen
```
NOTE:
When there is no experiment data in the `window.gon.experiment` object for the given experiment name, the `control` slot will be used, if it exists.
When there is no experiment data in the `window.gl.experiments` object for the given experiment name, the `control` slot will be used, if it exists.
## Test with Jest
### Stub Helpers
You can stub experiments using the `stubExperiments` helper defined in `spec/frontend/__helpers__/experimentation_helper.js`.
```javascript
import { stubExperiments } from 'helpers/experimentation_helper';
import { getExperimentData } from '~/experimentation/utils';
describe('when my_experiment is enabled', () => {
beforeEach(() => {
stubExperiments({ my_experiment: 'candidate' });
});
it('sets the correct data', () => {
expect(getExperimentData('my_experiment')).toEqual({ experiment: 'my_experiment', variant: 'candidate' });
});
});
```
NOTE:
This method of stubbing in Jest specs will not automatically un-stub itself at the end of the test. We merge our stubbed experiment in with all the other global data in `window.gl`. If you need to remove the stubbed experiment(s) after your test or ensure a clean global object before your test, you'll need to manage the global object directly yourself:
```javascript
desribe('tests that care about global state', () => {
const originalObjects = [];
beforeEach(() => {
// For backwards compatibility for now, we're using both window.gon & window.gl
originalObjects.push(window.gon, window.gl);
});
afterEach(() => {
[window.gon, window.gl] = originalObjects;
});
it('stubs experiment in fresh global state', () => {
stubExperiment({ my_experiment: 'candidate' });
// ...
});
})
```
## Notes on feature flags
......
ee/spec/frontend/vue_mr_widget/components/states/mr_widget_enable_feature_prompt_spec.js
View file @ a40517e8
import { mount } from '@vue/test-utils';
import MrWidgetEnableFeaturePrompt from 'ee/vue_merge_request_widget/components/states/mr_widget_enable_feature_prompt.vue';
import { assignGitlabExperiment } from 'helpers/experimentation_helper';
import { stubExperiments } from 'helpers/experimentation_helper';
import CiIcon from '~/vue_shared/components/ci_icon.vue';
const FEATURE = 'my_feature_name';
......@@ -30,19 +30,16 @@ describe('MrWidgetEnableFeaturePrompt', () => {
});
describe('when the experiment is not enabled', () => {
beforeAll(() => {
assignGitlabExperiment(FEATURE, 'control');
});
it('renders nothing', () => {
stubExperiments({ [FEATURE]: 'control' });
expect(wrapper.text()).toBe('');
});
});
describe('when the experiment is enabled', () => {
beforeAll(() => {
stubExperiments({ [FEATURE]: 'candidate' });
localStorage.removeItem(LOCAL_STORAGE_KEY);
assignGitlabExperiment(FEATURE, 'candidate');
});
it('shows a neutral icon', () => {
......
ee/spec/frontend/vue_mr_widget/ee_mr_widget_options_spec.js
View file @ a40517e8
......@@ -24,7 +24,7 @@ import {
coverageFuzzingDiffSuccessMock,
apiFuzzingDiffSuccessMock,
} from 'ee_jest/vue_shared/security_reports/mock_data';
import { assignGitlabExperiment } from 'helpers/experimentation_helper';
import { stubExperiments } from 'helpers/experimentation_helper';
import createMockApollo from 'helpers/mock_apollo_helper';
import { TEST_HOST } from 'helpers/test_constants';
import { trimText } from 'helpers/text_helper';
......@@ -221,7 +221,9 @@ describe('ee merge request widget options', () => {
});
describe('security_reports_mr_widget_prompt experiment', () => {
assignGitlabExperiment('security_reports_mr_widget_prompt', 'candidate');
beforeEach(() => {
stubExperiments({ security_reports_mr_widget_prompt: 'candidate' });
});
it('prompts to enable the feature', () => {
createComponent({ propsData: { mrData: mockData } });
......
spec/frontend/__helpers__/experimentation_helper.js
View file @ a40517e8
import { merge } from 'lodash';
// This helper is for specs that use `gitlab/experimentation` module
export function withGonExperiment(experimentKey, value = true) {
let origGon;
......@@ -12,16 +13,26 @@ export function withGonExperiment(experimentKey, value = true) {
window.gon = origGon;
});
}
// This helper is for specs that use `gitlab-experiment` utilities, which have a different schema that gets pushed via Gon compared to `Experimentation Module`
export function assignGitlabExperiment(experimentKey, variant) {
let origGon;
beforeEach(() => {
origGon = window.gon;
window.gon = { experiment: { [experimentKey]: { variant } } };
});
// The following helper is for specs that use `gitlab-experiment` utilities,
// which have a different schema that gets pushed to the frontend compared to
// the `Experimentation` Module.
//
// Usage: stubExperiments({ experiment_feature_flag_name: 'variant_name', ... })
export function stubExperiments(experiments = {}) {
// Deprecated
window.gon = window.gon || {};
window.gon.experiment = window.gon.experiment || {};
// Preferred
window.gl = window.gl || {};
window.gl.experiments = window.gl.experiemnts || {};
afterEach(() => {
window.gon = origGon;
Object.entries(experiments).forEach(([name, variant]) => {
const experimentData = { experiment: name, variant };
// Deprecated
window.gon.experiment[name] = experimentData;
// Preferred
window.gl.experiments[name] = experimentData;
});
}
spec/frontend/boards/components/new_board_button_spec.js
View file @ a40517e8
......@@ -2,7 +2,7 @@ import { mount } from '@vue/test-utils';
import { GlButton } from '@gitlab/ui';
import NewBoardButton from '~/boards/components/new_board_button.vue';
import { extendedWrapper } from 'helpers/vue_test_utils_helper';
import { assignGitlabExperiment } from 'helpers/experimentation_helper';
import { stubExperiments } from 'helpers/experimentation_helper';
import eventHub from '~/boards/eventhub';
const FEATURE = 'prominent_create_board_btn';
......@@ -28,7 +28,9 @@ describe('NewBoardButton', () => {
});
describe('control variant', () => {
assignGitlabExperiment(FEATURE, 'control');
beforeAll(() => {
stubExperiments({ [FEATURE]: 'control' });
});
it('renders nothing', () => {
wrapper = createComponent();
......@@ -38,7 +40,9 @@ describe('NewBoardButton', () => {
});
describe('candidate variant', () => {
assignGitlabExperiment(FEATURE, 'candidate');
beforeAll(() => {
stubExperiments({ [FEATURE]: 'candidate' });
});
it('renders New board button when `candidate` variant', () => {
wrapper = createComponent();
......
spec/frontend/experimentation/utils_spec.js
View file @ a40517e8
import { assignGitlabExperiment } from 'helpers/experimentation_helper';
import { stubExperiments } from 'helpers/experimentation_helper';
import {
DEFAULT_VARIANT,
CANDIDATE_VARIANT,
......@@ -7,15 +7,45 @@ import {
import * as experimentUtils from '~/experimentation/utils';
describe('experiment Utilities', () => {
const TEST_KEY = 'abc';
const ABC_KEY = 'abc';
const DEF_KEY = 'def';
let origGon;
let origGl;
beforeEach(() => {
origGon = window.gon;
origGl = window.gl;
window.gon.experiment = {};
window.gl.experiments = {};
});
afterEach(() => {
window.gon = origGon;
window.gl = origGl;
});
describe('getExperimentData', () => {
const ABC_DATA = '_abc_data_';
const ABC_DATA2 = '_updated_abc_data_';
const DEF_DATA = '_def_data_';
describe.each`
gon | input | output
${[TEST_KEY, '_data_']} | ${[TEST_KEY]} | ${{ variant: '_data_' }}
${[]} | ${[TEST_KEY]} | ${undefined}
`('with input=$input and gon=$gon', ({ gon, input, output }) => {
assignGitlabExperiment(...gon);
gonData | glData | input | output
${[ABC_KEY, ABC_DATA]} | ${[]} | ${[ABC_KEY]} | ${{ experiment: ABC_KEY, variant: ABC_DATA }}
${[]} | ${[ABC_KEY, ABC_DATA]} | ${[ABC_KEY]} | ${{ experiment: ABC_KEY, variant: ABC_DATA }}
${[ABC_KEY, ABC_DATA]} | ${[DEF_KEY, DEF_DATA]} | ${[ABC_KEY]} | ${{ experiment: ABC_KEY, variant: ABC_DATA }}
${[ABC_KEY, ABC_DATA]} | ${[DEF_KEY, DEF_DATA]} | ${[DEF_KEY]} | ${{ experiment: DEF_KEY, variant: DEF_DATA }}
${[ABC_KEY, ABC_DATA]} | ${[ABC_KEY, ABC_DATA2]} | ${[ABC_KEY]} | ${{ experiment: ABC_KEY, variant: ABC_DATA2 }}
${[]} | ${[]} | ${[ABC_KEY]} | ${undefined}
`('with input=$input, gon=$gonData, & gl=$glData', ({ gonData, glData, input, output }) => {
beforeEach(() => {
const [gonKey, gonVariant] = gonData;
const [glKey, glVariant] = glData;
if (gonKey) window.gon.experiment[gonKey] = { experiment: gonKey, variant: gonVariant };
if (glKey) window.gl.experiments[glKey] = { experiment: glKey, variant: glVariant };
});
it(`returns ${output}`, () => {
expect(experimentUtils.getExperimentData(...input)).toEqual(output);
......@@ -25,66 +55,47 @@ describe('experiment Utilities', () => {
describe('getAllExperimentContexts', () => {
const schema = TRACKING_CONTEXT_SCHEMA;
let origGon;
beforeEach(() => {
origGon = window.gon;
});
afterEach(() => {
window.gon = origGon;
});
it('collects all of the experiment contexts into a single array', () => {
const experiments = [
{ experiment: 'abc', variant: 'candidate' },
{ experiment: 'def', variant: 'control' },
{ experiment: 'ghi', variant: 'blue' },
];
window.gon = {
experiment: experiments.reduce((collector, { experiment, variant }) => {
return { ...collector, [experiment]: { experiment, variant } };
}, {}),
};
const experiments = { [ABC_KEY]: 'candidate', [DEF_KEY]: 'control', ghi: 'blue' };
stubExperiments(experiments);
expect(experimentUtils.getAllExperimentContexts()).toEqual(
experiments.map((data) => ({ schema, data })),
Object.entries(experiments).map(([experiment, variant]) => ({
schema,
data: { experiment, variant },
})),
);
});
it('returns an empty array if there are no experiments', () => {
window.gon.experiment = {};
expect(experimentUtils.getAllExperimentContexts()).toEqual([]);
});
it('includes all additional experiment data', () => {
const experiment = 'experimentWithCustomData';
const data = { experiment, variant: 'control', color: 'blue', style: 'rounded' };
window.gon.experiment[experiment] = data;
expect(experimentUtils.getAllExperimentContexts()).toContainEqual({ schema, data });
});
});
describe('isExperimentVariant', () => {
describe.each`
gon | input | output
${[TEST_KEY, DEFAULT_VARIANT]} | ${[TEST_KEY, DEFAULT_VARIANT]} | ${true}
${[TEST_KEY, '_variant_name']} | ${[TEST_KEY, '_variant_name']} | ${true}
${[TEST_KEY, '_variant_name']} | ${[TEST_KEY, '_bogus_name']} | ${false}
${[TEST_KEY, '_variant_name']} | ${['boguskey', '_variant_name']} | ${false}
${[]} | ${[TEST_KEY, '_variant_name']} | ${false}
`('with input=$input and gon=$gon', ({ gon, input, output }) => {
assignGitlabExperiment(...gon);
experiment | variant | input | output
${ABC_KEY} | ${DEFAULT_VARIANT} | ${[ABC_KEY, DEFAULT_VARIANT]} | ${true}
${ABC_KEY} | ${'_variant_name'} | ${[ABC_KEY, '_variant_name']} | ${true}
${ABC_KEY} | ${'_variant_name'} | ${[ABC_KEY, '_bogus_name']} | ${false}
${ABC_KEY} | ${'_variant_name'} | ${['boguskey', '_variant_name']} | ${false}
${undefined} | ${undefined} | ${[ABC_KEY, '_variant_name']} | ${false}
`(
'with input=$input, experiment=$experiment, variant=$variant',
({ experiment, variant, input, output }) => {
it(`returns ${output}`, () => {
if (experiment) stubExperiments({ [experiment]: variant });
expect(experimentUtils.isExperimentVariant(...input)).toEqual(output);
});
});
},
);
});
describe('experiment', () => {
const experiment = 'marley';
const useSpy = jest.fn();
const controlSpy = jest.fn();
const trySpy = jest.fn();
......@@ -98,49 +109,62 @@ describe('experiment Utilities', () => {
};
describe('when there is no experiment data', () => {
it('calls control variant', () => {
experimentUtils.experiment('marley', variants);
it('calls the use variant', () => {
experimentUtils.experiment(experiment, variants);
expect(useSpy).toHaveBeenCalled();
});
describe("when 'control' is provided instead of 'use'", () => {
it('calls the control variant', () => {
experimentUtils.experiment(experiment, { control: controlSpy });
expect(controlSpy).toHaveBeenCalled();
});
});
});
describe('when experiment variant is "control"', () => {
assignGitlabExperiment('marley', DEFAULT_VARIANT);
beforeEach(() => {
stubExperiments({ [experiment]: DEFAULT_VARIANT });
});
it('calls the control variant', () => {
experimentUtils.experiment('marley', variants);
it('calls the use variant', () => {
experimentUtils.experiment(experiment, variants);
expect(useSpy).toHaveBeenCalled();
});
describe("when 'control' is provided instead of 'use'", () => {
it('calls the control variant', () => {
experimentUtils.experiment('marley', { control: controlSpy });
experimentUtils.experiment(experiment, { control: controlSpy });
expect(controlSpy).toHaveBeenCalled();
});
});
});
describe('when experiment variant is "candidate"', () => {
assignGitlabExperiment('marley', CANDIDATE_VARIANT);
beforeEach(() => {
stubExperiments({ [experiment]: CANDIDATE_VARIANT });
});
it('calls the candidate variant', () => {
experimentUtils.experiment('marley', variants);
it('calls the try variant', () => {
experimentUtils.experiment(experiment, variants);
expect(trySpy).toHaveBeenCalled();
});
describe("when 'candidate' is provided instead of 'try'", () => {
it('calls the control variant', () => {
experimentUtils.experiment('marley', { candidate: candidateSpy });
it('calls the candidate variant', () => {
experimentUtils.experiment(experiment, { candidate: candidateSpy });
expect(candidateSpy).toHaveBeenCalled();
});
});
});
describe('when experiment variant is "get_up_stand_up"', () => {
assignGitlabExperiment('marley', 'get_up_stand_up');
beforeEach(() => {
stubExperiments({ [experiment]: 'get_up_stand_up' });
});
it('calls the get-up-stand-up variant', () => {
experimentUtils.experiment('marley', variants);
experimentUtils.experiment(experiment, variants);
expect(getUpStandUpSpy).toHaveBeenCalled();
});
});
......@@ -148,14 +172,17 @@ describe('experiment Utilities', () => {
describe('getExperimentVariant', () => {
it.each`
gon | input | output
${{ experiment: { [TEST_KEY]: { variant: DEFAULT_VARIANT } } }} | ${[TEST_KEY]} | ${DEFAULT_VARIANT}
${{ experiment: { [TEST_KEY]: { variant: CANDIDATE_VARIANT } } }} | ${[TEST_KEY]} | ${CANDIDATE_VARIANT}
${{}} | ${[TEST_KEY]} | ${DEFAULT_VARIANT}
`('with input=$input and gon=$gon, returns $output', ({ gon, input, output }) => {
window.gon = gon;
expect(experimentUtils.getExperimentVariant(...input)).toEqual(output);
});
experiment | variant | input | output
${ABC_KEY} | ${DEFAULT_VARIANT} | ${ABC_KEY} | ${DEFAULT_VARIANT}
${ABC_KEY} | ${CANDIDATE_VARIANT} | ${ABC_KEY} | ${CANDIDATE_VARIANT}
${undefined} | ${undefined} | ${ABC_KEY} | ${DEFAULT_VARIANT}
`(
'with input=$input, experiment=$experiment, & variant=$variant; returns $output',
({ experiment, variant, input, output }) => {
stubExperiments({ [experiment]: variant });
expect(experimentUtils.getExperimentVariant(input)).toEqual(output);
},
);
});
});
spec/views/layouts/_published_experiments.html.haml_spec.rb 0 → 100644
View file @ a40517e8
# frozen_string_literal: true
require 'spec_helper'
RSpec.describe 'layouts/_published_experiments', :experiment do
before do
stub_const('TestControlExperiment', ApplicationExperiment)
stub_const('TestCandidateExperiment', ApplicationExperiment)
stub_const('TestExcludedExperiment', ApplicationExperiment)
TestControlExperiment.new('test_control').tap do |e|
e.variant(:control)
e.publish
end
TestCandidateExperiment.new('test_candidate').tap do |e|
e.variant(:candidate)
e.publish
end
TestExcludedExperiment.new('test_excluded').tap do |e|
e.exclude!
e.publish
end
render
end
it 'renders out data for all non-excluded, published experiments' do
output = rendered
expect(output).to include('gl.experiments = {')
expect(output).to match(/"test_control":\{[^}]*"variant":"control"/)
expect(output).to match(/"test_candidate":\{[^}]*"variant":"candidate"/)
expect(output).not_to include('"test_excluded"')
end
end
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7