There are two types of test suites you'll encounter while developing frontend code
at GitLab. We use Karma and Jasmine for JavaScript unit and integration testing,
at GitLab. We use Karma with Jasmine and Jest for JavaScript unit and integration testing,
and RSpec feature tests with Capybara for e2e (end-to-end) integration testing.
Unit and feature tests need to be written for all new features.
...
...
@@ -13,6 +13,10 @@ in the future.
See the [Testing Standards and Style Guidelines](index.md) page for more
information on general testing practices at GitLab.
## Vue.js testing
If you are looking for a guide on Vue component testing, you can jump right away to this [section][vue-test].
## Jest
We have started to migrate frontend tests to the [Jest](https://jestjs.io) testing framework (see also the corresponding
...
...
@@ -20,6 +24,25 @@ We have started to migrate frontend tests to the [Jest](https://jestjs.io) testi
Jest tests can be found in `/spec/frontend` and `/ee/spec/frontend` in EE.
> **Note:**
>
> Most examples have a Jest and Karma example. See the Karma examples only as explanation to what's going on in the code, should you stumble over some usescases during your discovery. The Jest examples are the one you should follow.
## Karma test suite
While GitLab is switching over to [Jest][jest] you'll still find Karma tests in our application. [Karma][karma] is a test runner which uses [Jasmine] as its test
framework. Jest also uses Jasmine as foundation, that's why it's looking quite similar.
Karma tests live in `spec/javascripts/` and `/ee/spec/javascripts` in EE.
`app/assets/javascripts/behaviors/autosize.js`
might have a corresponding `spec/javascripts/behaviors/autosize_spec.js` file.
Keep in mind that in a CI environment, these tests are run in a headless
browser and you will not have access to certain APIs, such as
Remember that the performance of each test depends on the environment.
### Manual module mocks
## What and how to test
Jest supports [manual module mocks](https://jestjs.io/docs/en/manual-mocks) by placing a mock in a `__mocks__/` directory next to the source module. **Don't do this.** We want to keep all of our test-related code in one place (the `spec/` folder), and the logic that Jest uses to apply mocks from `__mocks__/` is rather inconsistent.
Before jumping into more gritty details about Jest-specific workflows like mocks and spies, we should briefly cover what to test with Jest.
Instead, our test runner detects manual mocks from `spec/frontend/mocks/`. Any mock placed here is automatically picked up and injected whenever you import its source module.
### Don't test the library
- Files in `spec/frontend/mocks/ce` will mock the corresponding CE module from `app/assets/javascripts`, mirroring the source module's path.
- Example: `spec/frontend/mocks/ce/lib/utils/axios_utils` will mock the module `~/lib/utils/axios_utils`.
- Files in `spec/frontend/mocks/node` will mock NPM packages of the same name or path.
- We don't support mocking EE modules yet.
Libraries are an integral part of any JavaScript developer's life. The general advice would be to not test library internals, but expect that the library knows what it's supposed to do and has test coverage on its own.
A general example could be something like this
If a mock is found for which a source module doesn't exist, the test suite will fail. 'Virtual' mocks, or mocks that don't have a 1-to-1 association with a source module, are not supported yet.
Create a JS module in the appropriate place in `spec/frontend/mocks/`. That's it. It will automatically mock its source package in all tests.
functiongetFahrenheit(celsius){
returnconvertToFahrenheit(celsius)
}
```
Make sure that your mock's export has the same format as the mocked module. So, if you're mocking a CommonJS module, you'll need to use `module.exports` instead of the ES6 `export`.
It does not make sense to test our `getFahrenheit` function because underneath it does nothing else but invoking the library function, and we can expect that one is working as intended. (Simplified, I know)
It might be useful for a mock to expose a property that indicates if the mock was loaded. This way, tests can assert the presence of a mock without calling any logic and causing side-effects. The `~/lib/utils/axios_utils` module mock has such a property, `isMock`, that is `true` in the mock and undefined in the original class. Jest's mock functions also have a `mock` property that you can test.
Let's take a short look into Vue land. Vue is a critical part of the GitLab JavaScript codebase. When writing specs for Vue components, a common gotcha is to actually end up testing Vue provided functionality, because it appears to be the easiest thing to test. Here's an example taken from our codebase.
#### Bypassing mocks
```javascript
// Component
{
computed:{
hasMetricTypes(){
returnthis.metricTypes.length;
},
}
```
If you ever need to import the original module in your tests, use [`jest.requireActual()`](https://jestjs.io/docs/en/jest-object#jestrequireactualmodulename)(or`jest.requireActual().default` for the default export). The `jest.mock()` and `jest.unmock()` won't have an effect on modules that have a manual mock, because mocks are imported and cached before any tests are run.
and here's the corresponding spec
#### Keep mocks light
```javascript
describe('computed',()=>{
describe('hasMetricTypes',()=>{
it('returns true if metricTypes exist',()=>{
factory({metricTypes});
expect(wrapper.vm.hasMetricTypes).toBe(2);
});
Global mocks introduce magic and can affect how modules are imported in your tests. Try to keep them as light as possible and dependency-free. A global mock should be useful for any unit test. For example, the `axios_utils` and `jquery` module mocks throw an error when an HTTP request is attempted, since this is useful behaviour in >99% of tests.
it('returns true if no metricTypes exist',()=>{
factory();
expect(wrapper.vm.hasMetricTypes).toBe(0);
});
});
});
```
When in doubt, construct mocks in your test file using [`jest.mock()`](https://jestjs.io/docs/en/jest-object#jestmockmodulename-factory-options), [`jest.spyOn()`](https://jestjs.io/docs/en/jest-object#jestspyonobject-methodname), etc.
Testing the `hasMetricTypes` computed prop would seem like a given, but to test if the computed property is returning the length of `metricTypes`, is testing the Vue library itself. There is no value in this, besides it adding to the test suite. Better is to test it in the way the user interacts with it. Probably through the template.
### Data-driven tests
Keep an eye out for these kinds of tests, as they just make updating logic more fragile and tedious than it needs to be. This is also true for other libraries.
Similar to [RSpec's parameterized tests](best_practices.md#table-based--parameterized-tests),
Jest supports data-driven tests for:
Some more examples can be found further down [below](#unit-testing-guidelines)
- Individual tests using [`test.each`](https://jestjs.io/docs/en/api#testeachtable-name-fn-timeout)(aliased to `it.each`).
- Groups of tests using [`describe.each`](https://jestjs.io/docs/en/api#describeeachtable-name-fn-timeout).
### Don't test your mock
These can be useful for reducing repetition within tests. Each option can take an array of
data values or a tagged template literal.
Another common gotcha is that the specs end up verifying the mock is working. If you are using mocks, the mock should support the test, but not be the target of the test.
For example, it's better to use the generated markup to trigger a button click and validate the markup changed accordingly than to call a method manually and verify data structures or computed properties. There's always the chance of accidentally breaking the user flow, while the tests pass and provide a false sense of security.
## Common practices
### Best practices
Following you'll find some general common practices you will find as part of our testsuite. Should you stumble over something not following this guide, ideally fix it right away. 🎉
#### Naming unit tests
### Naming unit tests
When writing describe test blocks to test specific functions/methods,
please use the method name as the describe block name.
When testing Promises you should always make sure that the test is asynchronous and rejections are handled.
Your Promise chain should therefore end with a call of the `done` callback and `done.fail` in case an error occurred.
When testing Promises you should always make sure that the test is asynchronous and rejections are handled. It's now possible to use the `async/await` syntax in the test suite:
```javascript
it('tests a promise',async()=>{
constusers=awaitfetchUsers()
expect(users.length).toBe(42)
});
it('tests a promise rejection',async()=>{
expect.assertions(1);
try{
awaituser.getUserName(1);
}catch(e){
expect(e).toEqual({
error:'User with 1 not found.',
});
}
});
```
You can also work with Promise chains. In this case, you can make use of the `done` callback and `done.fail` in case an error occurred. Following are some examples:
and `jasmine.createSpyObject` to facilitate replacing methods with dummy
placeholders, and recalling when they are called and the arguments that are
passed to them. These tools should be used liberally, to test for expected
behavior, to mock responses, and to block unwanted side effects (such as a
method that would generate a network request or alter `window.location`). The
documentation for these methods can be found in the [Jasmine introduction page](https://jasmine.github.io/2.0/introduction.html#section-Spies).
Sometimes we have to test time-sensitive code. For example, recurring events that run every X amount of seconds or similar. Here you'll find some strategies to deal with that:
Sometimes you may need to spy on a method that is directly imported by another
module. GitLab has a custom `spyOnDependency` method which utilizes
[babel-plugin-rewire](https://github.com/speedskater/babel-plugin-rewire) to
achieve this. It can be used like so:
#### `setTimeout()` / `setInterval()` in application
```javascript
// my_module.js
import{visitUrl}from'~/lib/utils/url_utility';
If the application itself is waiting for some time, mock await the waiting. In Jest this is already
[done by default](https://gitlab.com/gitlab-org/gitlab/blob/a2128edfee799e49a8732bfa235e2c5e14949c68/jest.config.js#L47)
(see also [Jest Timer Mocks](https://jestjs.io/docs/en/timer-mocks)). In Karma you can use the
If the application triggers an event that you need to wait for in your test, register an event handler which contains
the assertions:
...
...
@@ -501,7 +480,7 @@ it('waits for an event', () => {
});
```
#### Ensuring that tests are isolated
### Ensuring that tests are isolated
Tests are normally architected in a pattern which requires a recurring setup and breakdown of the component under test. This is done by making use of the `beforeEach` and `afterEach` hooks.
...
...
@@ -534,19 +513,51 @@ In order to ensure that a clean wrapper object and DOM are being used in each te
See also the [Vue Test Utils documentation on `destroy`](https://vue-test-utils.vuejs.org/api/wrapper/#destroy).
#### Migrating flaky Karma tests to Jest
## Factories
TBU
Some of our Karma tests are flaky because they access the properties of a shared scope.
This also means that they are not easily parallelized.
## Mocking Strategies with Jest
Migrating flaky Karma tests to Jest will help significantly as each test is executed
in an isolated scope, improving performance and predictability.
### Stubbing and Mocking
### Vue.js unit tests
Jasmine provides stubbing and mocking capabilities. There are some subtle differences in how to use it within Karma and Jest.
See this [section][vue-test].
Stubs or spies are often used synonymously. In Jest it's quite easy thanks to the `.spyOn` method. [Official docs][jestspy]
The more challenging part are mocks, which can be used for functions or even dependencies.
### Running frontend tests
### Manual module mocks
Jest supports [manual module mocks](https://jestjs.io/docs/en/manual-mocks) by placing a mock in a `__mocks__/` directory next to the source module. **Don't do this.** We want to keep all of our test-related code in one place (the `spec/` folder), and the logic that Jest uses to apply mocks from `__mocks__/` is rather inconsistent.
Instead, our test runner detects manual mocks from `spec/frontend/mocks/`. Any mock placed here is automatically picked up and injected whenever you import its source module.
- Files in `spec/frontend/mocks/ce` will mock the corresponding CE module from `app/assets/javascripts`, mirroring the source module's path.
- Example: `spec/frontend/mocks/ce/lib/utils/axios_utils` will mock the module `~/lib/utils/axios_utils`.
- Files in `spec/frontend/mocks/node` will mock NPM packages of the same name or path.
- We don't support mocking EE modules yet.
If a mock is found for which a source module doesn't exist, the test suite will fail. 'Virtual' mocks, or mocks that don't have a 1-to-1 association with a source module, are not supported yet.
### Writing a mock
Create a JS module in the appropriate place in `spec/frontend/mocks/`. That's it. It will automatically mock its source package in all tests.
Make sure that your mock's export has the same format as the mocked module. So, if you're mocking a CommonJS module, you'll need to use `module.exports` instead of the ES6 `export`.
It might be useful for a mock to expose a property that indicates if the mock was loaded. This way, tests can assert the presence of a mock without calling any logic and causing side-effects. The `~/lib/utils/axios_utils` module mock has such a property, `isMock`, that is `true` in the mock and undefined in the original class. Jest's mock functions also have a `mock` property that you can test.
### Bypassing mocks
If you ever need to import the original module in your tests, use [`jest.requireActual()`](https://jestjs.io/docs/en/jest-object#jestrequireactualmodulename)(or`jest.requireActual().default` for the default export). The `jest.mock()` and `jest.unmock()` won't have an effect on modules that have a manual mock, because mocks are imported and cached before any tests are run.
### Keep mocks light
Global mocks introduce magic and can affect how modules are imported in your tests. Try to keep them as light as possible and dependency-free. A global mock should be useful for any unit test. For example, the `axios_utils` and `jquery` module mocks throw an error when an HTTP request is attempted, since this is useful behaviour in >99% of tests.
When in doubt, construct mocks in your test file using [`jest.mock()`](https://jestjs.io/docs/en/jest-object#jestmockmodulename-factory-options), [`jest.spyOn()`](https://jestjs.io/docs/en/jest-object#jestspyonobject-methodname), etc.
## Running Frontend Tests
For running the frontend tests, you need the following commands:
...
...
@@ -556,17 +567,40 @@ For running the frontend tests, you need the following commands:
As long as the fixtures don't change, `yarn test` is sufficient (and saves you some time).
### Live testing and focused testing
### Live testing and focused testing -- Jest
While you work on a testsuite, you may want to run these specs in watch mode, so they rerun automatically on every save.
```bash
# Watch and rerun all specs matching the name icon
yarn jest --watch icon
# Watch and rerun one specifc file
yarn jest --watch path/to/spec/file.spec.js
```
You can also run some focused tests without the `--watch` flag
```bash
# Run specific jest file
yarn jest ./path/to/local_spec.js
# Run specific jest folder
yarn jest ./path/to/folder/
# Run all jest files which path contain term
yarn jest term
```
### Live testing and focused testing -- Karma
Karma allows something similar, but it's way more costly.
While developing locally, it may be helpful to keep Karma running so that you
can get instant feedback on as you write tests and modify code. To do this
you can start Karma with `yarn run karma-start`. It will compile the JavaScript
Running Karma with `yarn run karma-start` will compile the JavaScript
assets and run a server at `http://localhost:9876/` where it will automatically
run the tests on any browser which connects to it. You can enter that url on
multiple browsers at once to have it run the tests on each in parallel.
While Karma is running, any changes you make will instantly trigger a recompile
and retest of the entire test suite, so you can see instantly if you've broken
and retest of the **entire test suite**, so you can see instantly if you've broken
a test with your changes. You can use [Jasmine focused][jasmine-focus] or
excluded tests (with `fdescribe` or `xdescribe`) to get Karma to run only the
tests you want while you're working on a specific feature, but make sure to
...
...
@@ -594,24 +628,6 @@ glob otherwise your shell may split it into multiple arguments:
It is also possible to target individual Jest / RSpec tests:
```bash
# Run specific jest file
yarn jest ./path/to/local_spec.js
# Run specific jest folder
yarn jest ./path/to/folder/
# Run all jest files which path contain term
yarn jest term
```
```bash
# Run specific rspec file
rspec ./path/to/local_spec.rb
# Run specific block within rspec file
rspec ./path/to/local_spec.rb:15
```
## Frontend test fixtures
Code that is added to HAML templates (in `app/views/`) or makes Ajax requests to the backend has tests that require HTML or JSON from the backend.
...
...
@@ -654,6 +670,52 @@ Fixtures are regenerated using the `bin/rake frontend:fixtures` command but you
for example `bin/rspec spec/frontend/fixtures/merge_requests.rb`.
When creating a new fixture, it often makes sense to take a look at the corresponding tests for the endpoint in `(ee/)spec/controllers/` or `(ee/)spec/requests/`.
## Data-driven tests
Similar to [RSpec's parameterized tests](best_practices.md#table-based--parameterized-tests),
Jest supports data-driven tests for:
- Individual tests using [`test.each`](https://jestjs.io/docs/en/api#testeachtable-name-fn-timeout)(aliased to `it.each`).
- Groups of tests using [`describe.each`](https://jestjs.io/docs/en/api#describeeachtable-name-fn-timeout).
These can be useful for reducing repetition within tests. Each option can take an array of