mongo/jstests/README.md

# Javascript Test Guide

At MongoDB we write integration tests in JavaScript. These are tests written to exercise some
behavior of a running MongoDB server, replica set, or sharded cluster. This guide aims to provide
some general guidelines and best practices on how to write good tests.

## Principles

### Minimize the test case as much as possible while still exercising and testing the desired behavior.

- For example, if you are testing that document deletion works correctly, it may be entirely
  sufficient to insert just a single document and then delete that document. Inserting multiple
  documents would be unnecessary. A guiding principle on this is to ask yourself how easy it would
  be for a new person coming to this test to quickly understand it. If there are multiple documents
  being inserted into a collection, in a test that only tests document deletion, a newcomer might
  ask the question: “is it important that the test uses multiple documents, or incidental?”. It is
  best if you can remove these kinds of questions from a person’s mind, by keeping only the absolute
  essential parts of a test.
- We should always strive for unittesting when possible, so if the functionality you want to test
  can be covered by a unit test, we should write a unit test instead.

### Add a block comment at the top of the JavaScript test file giving a clear and concise overview of what a test is trying to verify.

- For tests that are more complicated, a brief description of the test steps might be useful as
  well.

### Keep debuggability in mind.

- Assertion error messages should contain all information relevant to debugging the test. This means
  the server’s response from the failed command should almost always be included in the assertion
  error message. It can also be helpful to include parameters that vary during the test to avoid
  requiring the investigator to use the logs/backtrace to determine what the test was attempting to
  do.
- Think about how easy it would be to debug your test if something failed and a newcomer only had
  the logs of the test to look at. This can help guide your decision on what log messages to include
  and to what level of detail. The jsTestLog function is useful for this, as it is good at visually
  demarcating different phases of a test. As a tip, run your test a few times and just study the log
  messages, imagining you are an engineer debugging the test with only these logs to look at. Think
  about how understandable the logs would be to a newcomer. It is easy to add log messages to a test
  but then forget to see how they would actually appear.
- Never insert identical documents unless necessary. It is very useful in debugging to be able to
  figure out where a given piece of data came from.
- If a test does the same thing multiple times, consider factoring it out into a library. Shorter
  running tests are easier to debug and code duplication is always bad.

### Do not hardcode collection or database names, especially if they are used multiple times throughout a test.

It is best to use variable names that attempt to describe what a value is used for. For example,
naming a variable that stores a collection named `collectionToDrop` is much better than just naming
the variable `collName`.

### Make every effort to make your test as deterministic as possible.

- Non-deterministic tests add noise to our build system and, in general, make it harder for yourself
  and other engineers to determine if the system really is working correctly or not. Flaky
  integration tests should be considered bugs, and we should not allow them to be committed to the
  server codebase. One way to make jstests more deterministic is to use failpoints to force the
  events happening in expected order. However, if we have to use failpoints to make this test
  deterministic, we should consider write a unit test instead.
- Note that our fuzzer and concurrency test suites are often an exception to this rule. In those
  cases we sometimes give up some level of determinism in order to trigger a wider class of rare
  edge cases. For targeted JavaScript integration tests, however, highly deterministic tests should
  be the goal.

### Think hard about all the assumptions that the test relies on.

- For example, if a certain phase of the test ran much slower or much faster, would it cause your
  test to fail for the wrong reason?
- If your test includes hard-coded timeouts, make sure they are set appropriately. If a test is
  waiting for a certain condition to be true, and the test should not proceed until that condition
  is met, it is often correct to just wait “indefinitely”, instead of adding some arbitrary timeout
  value, like 30 seconds. In practice this usually means setting some reasonable upper limit, for
  example, 10 minutes.
- Also, for replication tests, make sure data exists on the right nodes at the right time. For
  example, if you a do a write and don’t explicitly wait for it to replicate, it might not reach a
  secondary node before you try to do the next step of the test.
- Does your test require data to be stored persistently? Remember that we have test variants that
  run on in-memory/ephemeral storage engines
- There are timeouts in the test suites and we aim to make all tests in the same suite finish before
  timeout. That says we should always make the test run quickly to keep the test short in terms of
  duration.

### Make tests fail as early as possible.

- If something goes wrong early in the test, it’s much harder to diagnose when that error becomes
  visible much later.
- Wrap every command in assert.commandWorked, or assert.commandFailedWithCode. There is also
  assert.commandFailed that won't check the return error code, but we should always try to use
  assert.commandFailedWithCode to make sure the test won't pass on an unexpected error.

### Be aware of all the configurations and variants that your test might run under.

- Make sure that your test still works correctly if is run in a different configuration or on a
  different platform than the one you might have tested on.
- Varying storage engines and suites can often affect a test’s behavior. For example, maybe your
  test fails unexpectedly if it runs with authentication turned on with an in-memory storage engine.
  You don’t have to run a new test on every possible platform before committing it, but you should
  be confident that your test doesn’t break in an unexpected configuration.

### Avoid assertions that verify properties indirectly.

All assertions in a test should attempt to verify the most specific property possible. For example,
if you are trying to test that a certain collection exists, it is better to assert that the
collection’s exact name exists in the list of collections, as opposed to verifying that the
collection count is equal to 1. The desired collection’s existence is sufficient for the collection
count to be 1, but not necessary (a different collection could exist in its place). Be wary of
adding these kind of indirect assertions in a test.

### Test Isolation

Your JS test will likely be running with many other files before and after it. It's important to
start from a known state, and to restore that state (to a reasonable extent) at the end of your test
content.

- **Before**: If there are critical assumptions about the environment that your test needs, assert
  for it explicitly before proceeding to the real test content (instead of debugging side effects of
  that not being the case)
  - If you have a precondition on the _environment_, use [`@tags`](./tags.md) instead of just an
    early-return. This will avoid the test being scheduled in the first place if the environment is
    not supported.
- **After**: If you are modifying the fixture, do everything possible to safely restore those
  changes at the end of your test content, even after a test failure. Resmokes'
  `--continueOnFailure` flag is used in CI, so the fixture is shared across many test files, and is
  only torn down at the end.
  - Note, a fixture _can_ immediately "abort" after a test failure, only if
    [archiving](../../../../buildscripts/resmokeconfig/suites/README.md#executorarchive) is
    configured, but that shouldn't be assumed because that is a per-suite configuration (and your
    test can run in many passthrough suite combinations).
  - One easy approach to restoring your state is to use the
    [Mocha-style](#use-mocha-style-constructs) `after` hooks in your test content.

## Modern JS: Modules in Practice

We have fully migrated to the modularized JavaScript world so any new test should use modules and
adapt the new style.

### Only import/export what you need.

It's always important to keep the test context clean so we should only import/export what we need.

- The unused import is against [no-unused-vars](https://eslint.org/docs/latest/rules/no-unused-vars)
  rule in ESLint though we haven't enforced it.
- We don't have a linter to check export since it's hard to tell the necessity, but we should only
  export the modules that are imported by other tests or will be needed in the future.

### Declare variables in proper scope.

In the past, we have seen tests referring some "undeclared" or "redeclared" variables, which are
actually introduced through `load()`. Now with modules, the scope is more clear. We can use global
variables properly to setup the test and don't need to worry about polluting other tests.

### Name variables properly when exporting.

To avoid naming conflicts, we should not make the name of exported variables too general which could
easily conflict with another variable from the test which import your module. For example, in the
following case, the module exported a variable named `alphabet` and it will lead to a re-declaration
error.

```
import {alphabet} from "/matts/module.js";
const alphabet = "xyz"; // ERROR
```

### Prefer let/const over var

`let/const` should be preferred over `var` since these can help detect double declaration at the
first place. Like, in the naming conflict example, if the second line is using var, it could easily
mess up without throwing an error.

### Export in ES6 style

Due to legacy, we have a lot of code that is using the old style to do export, like the following.

```
const MyModule = (function() {
  function myFeature() {}
  function myOtherFeature() {}

  return {myFeature, myOtherFeature};
})();
```

Instead, we should use the ES6 way to do export, as follows.

```
export function myFeature() {}
export function myOtherFeature() {}

// When import from test
import * as MyModule from "/path/to/my_module.js";
```

This can help the language server to discover the methods and provide code navigation for it.

### Use Mocha-style Constructs

The [mochalite.js](../jstests/libs/mochalite.js) library ports over a subset of
[MochaJS](https://mochajs.org/) functionality for the shell, including:

- `it` test contruction
- `describe` suite structures
- `it.only` and `describe.only` to run only those suites and tests
- `it.skip` and `describe.skip` to skip those suites and tests
- `before` and `after` hooks, to run _once_ around _all_ `it` tests
- `beforeEach` and `afterEach` hooks, to run around _each_ `it` test
- The above (excluding `describe` variants) also support `async` functions
- Resmoke test filtering using the `--mochagrep` flag, which mirrors the
  [`grep`](https://mochajs.org/#-grep-regexp-g-regexp) flag from MochaJS

Example using several APIs:

```js
import {after, afterEach, before, beforeEach, describe, it} from "jstests/libs/mochalite.js";

describe("simple inserts and finds", () => {
  before(() => {
    this.fixtureDB = startupNewDB();
  });
  beforeEach(() => {
    this.fixtureDB.seed();
  });
  afterEach(async () => {
    await this.fixtureDB.clear();
  });
  after(() => {
    this.fixtureDB.shutdown();
  });
  it("should do something", () => {
    this.fixtureDB.insert({name: "test"});
    assert.eq(this.fixtureDB.find({name: "test"}).count(), 1);
  });
  it("should error on invalid data", () => {
    const e = assert.throws(() => this.fixtureDB.insert({notafield: undefined}));
    assert.eq(e.message, "Field 'notafield' not found");
  });
});
```

Use `it.only` to run just the one test (eg. during debugging):

```js
it.only("should do something", () => {
  this.fixtureDB.insert({name: "test"});
  assert.eq(this.fixtureDB.find({name: "test"}).count(), 1);
});
```

or use the filter from resmoke to avoid any file edits:

```sh
buildscripts/resmoke.py run --suites=no_passthrough --mochagrep "do something" jstests/noPassthrough/mytest.js
```

## Test Tags

JS Test files can leverage "tags" that suites can key off of to include and/or exclude as necessary.
Not scheduling a test to run is much faster than the test doing an early-return when preconditions
are not met.

The simplest use case is having something like the following at the top of your js test file:

```js
/**
 * Tests for the XYZ feature
 * @tags: [requires_fcv_81]
 */
```

See [tags.md](./tags.md) for more details.