Commit 942be4c5 authored by Amy Qualls's avatar Amy Qualls

Merge branch 'russell/remove-future-tense-testing-guide' into 'master'

Remove future tense from the Testing Guide

See merge request gitlab-org/gitlab!49890
parents d7406570 1f2ed6b9
...@@ -27,7 +27,7 @@ Our current CI parallelization setup is as follows: ...@@ -27,7 +27,7 @@ Our current CI parallelization setup is as follows:
`knapsack/rspec*_pg_*.json` files and merge them all together into a single `knapsack/rspec*_pg_*.json` files and merge them all together into a single
`knapsack/report-master.json` file that is saved as artifact. `knapsack/report-master.json` file that is saved as artifact.
After that, the next pipeline will use the up-to-date `knapsack/report-master.json` file. After that, the next pipeline uses the up-to-date `knapsack/report-master.json` file.
## Monitoring ## Monitoring
......
...@@ -164,7 +164,7 @@ used by the `review-deploy` and `review-stop` jobs. ...@@ -164,7 +164,7 @@ used by the `review-deploy` and `review-stop` jobs.
You need to [open an access request (internal link)](https://gitlab.com/gitlab-com/access-requests/-/issues/new) You need to [open an access request (internal link)](https://gitlab.com/gitlab-com/access-requests/-/issues/new)
for the `gcp-review-apps-dev` GCP group and role. for the `gcp-review-apps-dev` GCP group and role.
This will grant you the following permissions for: This grants you the following permissions for:
- [Retrieving pod logs](#dig-into-a-pods-logs). Granted by [Viewer (`roles/viewer`)](https://cloud.google.com/iam/docs/understanding-roles#kubernetes-engine-roles). - [Retrieving pod logs](#dig-into-a-pods-logs). Granted by [Viewer (`roles/viewer`)](https://cloud.google.com/iam/docs/understanding-roles#kubernetes-engine-roles).
- [Running a Rails console](#run-a-rails-console). Granted by [Kubernetes Engine Developer (`roles/container.pods.exec`)](https://cloud.google.com/iam/docs/understanding-roles#kubernetes-engine-roles). - [Running a Rails console](#run-a-rails-console). Granted by [Kubernetes Engine Developer (`roles/container.pods.exec`)](https://cloud.google.com/iam/docs/understanding-roles#kubernetes-engine-roles).
...@@ -317,7 +317,7 @@ kubectl get cm --sort-by='{.metadata.creationTimestamp}' | grep 'review-' | grep ...@@ -317,7 +317,7 @@ kubectl get cm --sort-by='{.metadata.creationTimestamp}' | grep 'review-' | grep
### Using K9s ### Using K9s
[K9s](https://github.com/derailed/k9s) is a powerful command line dashboard which allows you to filter by labels. This can help identify trends with apps exceeding the [review-app resource requests](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/review_apps/base-config.yaml). Kubernetes will schedule pods to nodes based on resource requests and allow for CPU usage up to the limits. [K9s](https://github.com/derailed/k9s) is a powerful command line dashboard which allows you to filter by labels. This can help identify trends with apps exceeding the [review-app resource requests](https://gitlab.com/gitlab-org/gitlab/-/blob/master/scripts/review_apps/base-config.yaml). Kubernetes schedules pods to nodes based on resource requests and allow for CPU usage up to the limits.
- In K9s you can sort or add filters by typing the `/` character - In K9s you can sort or add filters by typing the `/` character
- `-lrelease=<review-app-slug>` - filters down to all pods for a release. This aids in determining what is having issues in a single deployment - `-lrelease=<review-app-slug>` - filters down to all pods for a release. This aids in determining what is having issues in a single deployment
...@@ -395,8 +395,8 @@ helm ls -d | grep "Jun 4" | cut -f1 | xargs helm delete --purge ...@@ -395,8 +395,8 @@ helm ls -d | grep "Jun 4" | cut -f1 | xargs helm delete --purge
#### Mitigation steps taken to avoid this problem in the future #### Mitigation steps taken to avoid this problem in the future
We've created a new node pool with smaller machines so that it's less likely We've created a new node pool with smaller machines to reduce the risk
that a machine will hit the "too many mount points" problem in the future. that a machine reaches the "too many mount points" problem in the future.
## Frequently Asked Questions ## Frequently Asked Questions
......
...@@ -7,11 +7,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w ...@@ -7,11 +7,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Smoke Tests # Smoke Tests
It is imperative in any testing suite that we have Smoke Tests. In short, smoke It is imperative in any testing suite that we have Smoke Tests. In short, smoke
tests will run quick sanity end-to-end functional tests from GitLab QA and are tests run quick end-to-end functional tests from GitLab QA and are
designed to run against the specified environment to ensure that basic designed to run against the specified environment to ensure that basic
functionality is working. functionality is working.
Currently, our suite consists of this basic functionality coverage: Our suite consists of this basic functionality coverage:
- User standard authentication - User standard authentication
- SSH Key creation and addition to a user - SSH Key creation and addition to a user
......
...@@ -129,14 +129,14 @@ graph RL ...@@ -129,14 +129,14 @@ graph RL
- **All server requests**: - **All server requests**:
When running frontend unit tests, the backend may not be reachable, so all outgoing requests need to be mocked. When running frontend unit tests, the backend may not be reachable, so all outgoing requests need to be mocked.
- **Asynchronous background operations**: - **Asynchronous background operations**:
Background operations cannot be stopped or waited on, so they will continue running in the following tests and cause side effects. Background operations cannot be stopped or waited on, so they continue running in the following tests and cause side effects.
#### What *not* to mock in unit tests #### What *not* to mock in unit tests
- **Non-exported functions or classes**: - **Non-exported functions or classes**:
Everything that is not exported can be considered private to the module, and will be implicitly tested through the exported classes and functions. Everything that is not exported can be considered private to the module, and is implicitly tested through the exported classes and functions.
- **Methods of the class under test**: - **Methods of the class under test**:
By mocking methods of the class under test, the mocks will be tested and not the real methods. By mocking methods of the class under test, the mocks are tested and not the real methods.
- **Utility functions (pure functions, or those that only modify parameters)**: - **Utility functions (pure functions, or those that only modify parameters)**:
If a function has no side effects because it has no state, it is safe to not mock it in tests. If a function has no side effects because it has no state, it is safe to not mock it in tests.
- **Full HTML pages**: - **Full HTML pages**:
...@@ -206,7 +206,7 @@ graph RL ...@@ -206,7 +206,7 @@ graph RL
- **All server requests**: - **All server requests**:
Similar to unit tests, when running component tests, the backend may not be reachable, so all outgoing requests need to be mocked. Similar to unit tests, when running component tests, the backend may not be reachable, so all outgoing requests need to be mocked.
- **Asynchronous background operations**: - **Asynchronous background operations**:
Similar to unit tests, background operations cannot be stopped or waited on. This means they will continue running in the following tests and cause side effects. Similar to unit tests, background operations cannot be stopped or waited on. This means they continue running in the following tests and cause side effects.
- **Child components**: - **Child components**:
Every component is tested individually, so child components are mocked. Every component is tested individually, so child components are mocked.
See also [`shallowMount()`](https://vue-test-utils.vuejs.org/api/#shallowmount) See also [`shallowMount()`](https://vue-test-utils.vuejs.org/api/#shallowmount)
...@@ -214,7 +214,7 @@ graph RL ...@@ -214,7 +214,7 @@ graph RL
#### What *not* to mock in component tests #### What *not* to mock in component tests
- **Methods or computed properties of the component under test**: - **Methods or computed properties of the component under test**:
By mocking part of the component under test, the mocks will be tested and not the real component. By mocking part of the component under test, the mocks are tested and not the real component.
- **Functions and classes independent from Vue**: - **Functions and classes independent from Vue**:
All plain JavaScript code is already covered by unit tests and needs not to be mocked in component tests. All plain JavaScript code is already covered by unit tests and needs not to be mocked in component tests.
...@@ -295,7 +295,7 @@ graph RL ...@@ -295,7 +295,7 @@ graph RL
Similar to unit and component tests, when running component tests, the backend may not be reachable, so all outgoing requests must be mocked. Similar to unit and component tests, when running component tests, the backend may not be reachable, so all outgoing requests must be mocked.
- **Asynchronous background operations that are not perceivable on the page**: - **Asynchronous background operations that are not perceivable on the page**:
Background operations that affect the page must be tested on this level. Background operations that affect the page must be tested on this level.
All other background operations cannot be stopped or waited on, so they will continue running in the following tests and cause side effects. All other background operations cannot be stopped or waited on, so they continue running in the following tests and cause side effects.
#### What *not* to mock in integration tests #### What *not* to mock in integration tests
...@@ -360,7 +360,7 @@ possible). ...@@ -360,7 +360,7 @@ possible).
| Tests path | Testing engine | Notes | | Tests path | Testing engine | Notes |
| ---------- | -------------- | ----- | | ---------- | -------------- | ----- |
| `spec/features/` | [Capybara](https://github.com/teamcapybara/capybara) + [RSpec](https://github.com/rspec/rspec-rails#feature-specs) | If your test has the `:js` metadata, the browser driver will be [Poltergeist](https://github.com/teamcapybara/capybara#poltergeist), otherwise it's using [RackTest](https://github.com/teamcapybara/capybara#racktest). | | `spec/features/` | [Capybara](https://github.com/teamcapybara/capybara) + [RSpec](https://github.com/rspec/rspec-rails#feature-specs) | If your test has the `:js` metadata, the browser driver is [Poltergeist](https://github.com/teamcapybara/capybara#poltergeist), otherwise it's using [RackTest](https://github.com/teamcapybara/capybara#racktest). |
### Frontend feature tests ### Frontend feature tests
...@@ -453,13 +453,13 @@ should take care of not introducing too many (slow and duplicated) tests. ...@@ -453,13 +453,13 @@ should take care of not introducing too many (slow and duplicated) tests.
The reasons why we should follow these best practices are as follows: The reasons why we should follow these best practices are as follows:
- System tests are slow to run since they spin up the entire application stack - System tests are slow to run because they spin up the entire application stack
in a headless browser, and even slower when they integrate a JS driver in a headless browser, and even slower when they integrate a JS driver
- When system tests run with a JavaScript driver, the tests are run in a - When system tests run with a JavaScript driver, the tests are run in a
different thread than the application. This means it does not share a different thread than the application. This means it does not share a
database connection and your test will have to commit the transactions in database connection and your test must commit the transactions in
order for the running application to see the data (and vice-versa). In that order for the running application to see the data (and vice-versa). In that
case we need to truncate the database after each spec instead of simply case we need to truncate the database after each spec instead of
rolling back a transaction (the faster strategy that's in use for other kind rolling back a transaction (the faster strategy that's in use for other kind
of tests). This is slower than transactions, however, so we want to use of tests). This is slower than transactions, however, so we want to use
truncation only when necessary. truncation only when necessary.
......
...@@ -24,15 +24,15 @@ Adding a `:migration` tag to a test signature enables some custom RSpec ...@@ -24,15 +24,15 @@ Adding a `:migration` tag to a test signature enables some custom RSpec
[`spec/support/migration.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/f81fa6ab1dd788b70ef44b85aaba1f31ffafae7d/spec/support/migration.rb) [`spec/support/migration.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/f81fa6ab1dd788b70ef44b85aaba1f31ffafae7d/spec/support/migration.rb)
to run. to run.
A `before` hook will revert all migrations to the point that a migration A `before` hook reverts all migrations to the point that a migration
under test is not yet migrated. under test is not yet migrated.
In other words, our custom RSpec hooks will find a previous migration, and In other words, our custom RSpec hooks finds a previous migration, and
migrate the database **down** to the previous migration version. migrate the database **down** to the previous migration version.
With this approach you can test a migration against a database schema. With this approach you can test a migration against a database schema.
An `after` hook will migrate the database **up** and reinstitute the latest An `after` hook migrates the database **up** and reinstitutes the latest
schema version, so that the process does not affect subsequent specs and schema version, so that the process does not affect subsequent specs and
ensures proper isolation. ensures proper isolation.
...@@ -40,7 +40,7 @@ ensures proper isolation. ...@@ -40,7 +40,7 @@ ensures proper isolation.
To test an `ActiveRecord::Migration` class (i.e., a To test an `ActiveRecord::Migration` class (i.e., a
regular migration `db/migrate` or a post-migration `db/post_migrate`), you regular migration `db/migrate` or a post-migration `db/post_migrate`), you
will need to load the migration file by using the `require_migration!` helper must load the migration file by using the `require_migration!` helper
method because it is not autoloaded by Rails. method because it is not autoloaded by Rails.
Example: Example:
...@@ -57,12 +57,12 @@ RSpec.describe ... ...@@ -57,12 +57,12 @@ RSpec.describe ...
#### `require_migration!` #### `require_migration!`
Since the migration files are not autoloaded by Rails, you will need to manually Since the migration files are not autoloaded by Rails, you must manually
load the migration file. To do so, you can use the `require_migration!` helper method load the migration file. To do so, you can use the `require_migration!` helper method
which can automatically load the correct migration file based on the spec filename. which can automatically load the correct migration file based on the spec filename.
For example, if your spec file is named as `populate_foo_column_spec.rb` then the For example, if your spec file is named as `populate_foo_column_spec.rb` then the
helper method will try to load `${schema_version}_populate_foo_column.rb` migration file. helper method tries to load `${schema_version}_populate_foo_column.rb` migration file.
In case there is no pattern between your spec file and the actual migration file, In case there is no pattern between your spec file and the actual migration file,
you can provide the migration filename without the schema version, like so: you can provide the migration filename without the schema version, like so:
...@@ -85,8 +85,8 @@ project = table(:projects).create!(id: 1, name: 'gitlab1', path: 'gitlab1') ...@@ -85,8 +85,8 @@ project = table(:projects).create!(id: 1, name: 'gitlab1', path: 'gitlab1')
#### `migrate!` #### `migrate!`
Use the `migrate!` helper to run the migration that is under test. It will not only Use the `migrate!` helper to run the migration that is under test. It
run the migration, but will also bump the schema version in the `schema_migrations` runs the migration and bumps the schema version in the `schema_migrations`
table. It is necessary because in the `after` hook we trigger the rest of table. It is necessary because in the `after` hook we trigger the rest of
the migrations, and we need to know where to start. Example: the migrations, and we need to know where to start. Example:
...@@ -103,7 +103,7 @@ end ...@@ -103,7 +103,7 @@ end
#### `reversible_migration` #### `reversible_migration`
Use the `reversible_migration` helper to test migrations with either a Use the `reversible_migration` helper to test migrations with either a
`change` or both `up` and `down` hooks. This will test that the state of `change` or both `up` and `down` hooks. This tests that the state of
the application and its data after the migration becomes reversed is the the application and its data after the migration becomes reversed is the
same as it was before the migration ran in the first place. The helper: same as it was before the migration ran in the first place. The helper:
...@@ -184,7 +184,7 @@ end ...@@ -184,7 +184,7 @@ end
## Testing a non-`ActiveRecord::Migration` class ## Testing a non-`ActiveRecord::Migration` class
To test a non-`ActiveRecord::Migration` test (a background migration), To test a non-`ActiveRecord::Migration` test (a background migration),
you will need to manually provide a required schema version. Please add a you must manually provide a required schema version. Please add a
`schema` tag to a context that you want to switch the database schema within. `schema` tag to a context that you want to switch the database schema within.
If not set, `schema` defaults to `:latest`. If not set, `schema` defaults to `:latest`.
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment