GitLab has a massive test suite that, without [parallelization](ci.md#test-suite-parallelization-on-the-ci), can take hours
to run. It's important that we make an effort to write tests that are accurate
and effective _as well as_ fast.
Here are some things to keep in mind regarding test performance:
-`instance_double` and `spy` are faster than `FactoryBot.build(...)`
-`FactoryBot.build(...)` and `.build_stubbed` are faster than `.create`.
- Don't `create` an object when `build`, `build_stubbed`, `attributes_for`,
`spy`, or `instance_double` will do. Database persistence is slow!
- Don't mark a feature as requiring JavaScript (through `:js` in RSpec) unless it's _actually_ required for the test
to be valid. Headless browser testing is slow!
## RSpec
To run RSpec tests:
...
...
@@ -57,6 +50,51 @@ bundle exec guard
When using spring and guard together, use `SPRING=1 bundle exec guard` instead to make use of spring.
### Test speed
GitLab has a massive test suite that, without [parallelization](ci.md#test-suite-parallelization-on-the-ci), can take hours
to run. It's important that we make an effort to write tests that are accurate
and effective _as well as_ fast.
Test performance is important to maintaining quality and velocity, and has a
direct impact on CI build times and thus fixed costs. We want thorough, correct,
and fast tests. Here you can find some information about tools and techniques
available to you to achieve that.
#### Don't request capabilities you don't need
We make it easy to add capabilities to our examples by annotating the example or
a parent context. Examples of these are:
-`:js` in feature specs, which runs a full JavaScript capable headless browser.
-`:clean_gitlab_redis_cache` which provides a clean Redis cache to the examples.
-`:request_store` which provides a request store to the examples.
Obviously we should reduce test dependencies, and avoiding
capabilities also reduces the amount of set-up needed.
`:js` is particularly important to avoid. This must only be used if the feature
test requires JavaScript reactivity in the browser, since using a headless
browser is much slower than parsing the HTML response from the app.
#### Optimize factory usage
A common cause of slow tests is excessive creation of objects, and thus
computation and DB time. Factories are essential to development, but they can
make inserting data into the DB so easy that we may be able to optimize.
The two basic techniques to bear in mind here are:
-**Reduce**: avoid creating objects, and avoid persisting them.
-**Reuse**: shared objects, especially nested ones we do not examine, can generally be shared.
To avoid creation, it is worth bearing in mind that:
-`instance_double` and `spy` are faster than `FactoryBot.build(...)`.
-`FactoryBot.build(...)` and `.build_stubbed` are faster than `.create`.
- Don't `create` an object when `build`, `build_stubbed`, `attributes_for`,
`spy`, or `instance_double` will do. Database persistence is slow!
Use [Factory Doctor](https://test-prof.evilmartians.io/#/profilers/factory_doctor) to find cases where database persistence is not needed in a given test.
```shell
...
...
@@ -64,7 +102,7 @@ Use [Factory Doctor](https://test-prof.evilmartians.io/#/profilers/factory_docto
FDOC=1 bin/rspec spec/[path]/[to]/[spec].rb
```
A common change is to use `build` instead of `create`:
A common change is to use `build`or `build_stubbed`instead of `create`:
A common cause of a large number of created factories is [factory cascades](https://github.com/test-prof/test-prof/blob/master/docs/profilers/factory_prof.md#factory-flamegraph), which result when factories create and recreate associations.
They can be identified by a noticeable difference between `total time` and `top-level time` numbers:
```shell
```plaintext
total top-level total time time per call top-level time name
208 0 9.5812s 0.0461s 0.0000s namespace
208 76 37.4214s 0.1799s 13.8749s project
```
In order to reuse a single factory for all implicit parent associations,
The table above shows us that we never create any `namespace` objects explicitly
(`top-level == 0`) - they are all created implicitly for us. But we still end up
with 208 of them (one for each project) and this takes 9.5 seconds.
In order to reuse a single object for all calls to a named factory in implicit parent associations,
Issue relative positioning behaves like a class that supports relative positioning .move_nulls_to_end manages to move nulls to the end, stacking if we cannot create enough space
Issue relative positioning behaves like a class that supports relative positioning .move_nulls_to_start manages to move nulls to the end, stacking if we cannot create enough space
Issue behaves like an editable mentionable behaves like a mentionable when there are cached markdown fields sends in cached markdown fields when appropriate
Issue behaves like an editable mentionable when there are cached markdown fields when the markdown cache is stale persists the refreshed cache so that it does not have to be refreshed every time
