SERVER-121513: add an initial version of a resmoke agent skill (#49498)

GitOrigin-RevId: eb21af19273ec93a29f8dc1962db715fe45789f2
This commit is contained in:
Mathias Stearn 2026-03-13 16:46:20 +01:00 committed by MongoDB Bot
parent 0922c63c5c
commit c26287edb2
4 changed files with 277 additions and 0 deletions

View File

@ -0,0 +1,5 @@
version: 2.0.0
filters:
- "resmoke":
approvers:
- 10gen/devprod-correctness

View File

@ -0,0 +1,175 @@
---
name: resmoke
description: Run MongoDB integration and correctness tests using the resmoke test orchestrator. Use when working with jstests, test suites, or running MongoDB integration tests.
---
# Resmoke Test Orchestrator
Resmoke is MongoDB's integration testing framework for running JavaScript tests, C++ integration tests, and multiversion tests. Resmoke manages its own clean test environment; no pre-cleanup of data directories is needed.
## Basic Usage
Activate the virtual environment before running resmoke:
```bash
source python3-venv/bin/activate
```
### Running Tests
Run a test suite:
```bash
python3 buildscripts/resmoke.py run --suites=core
```
Run multiple suites:
```bash
python3 buildscripts/resmoke.py run --suites=core,noPassthrough
```
Run a single test file:
```bash
python3 buildscripts/resmoke.py run --suites=core jstests/core/administrative/auth1.js
```
Run multiple test files:
```bash
python3 buildscripts/resmoke.py run --suites=core jstests/core/administrative/auth1.js jstests/core/administrative/auth2.js
```
Run tests matching a pattern:
```bash
python3 buildscripts/resmoke.py run --suites=core jstests/core/administrative/*.js
```
Dry run to see what tests would execute:
```bash
python3 buildscripts/resmoke.py run --suites=core -n
```
### Common Test Suites
| Suite | Purpose |
| --------------- | -------------------------------------------- |
| `core` | General mongod tests (runs as standalone) |
| `noPassthrough` | Tests that manage their own mongod instances |
| `auth` | Authentication/authorization tests |
| `sharding` | Sharded cluster tests |
| `replica_sets` | Replication tests |
| `concurrency` | Concurrency and locking tests |
## Exploring Tests
List available suites:
```bash
python3 buildscripts/resmoke.py list-suites
```
Discover tests in a suite:
```bash
python3 buildscripts/resmoke.py test-discovery --suite=core
```
View suite configuration:
```bash
python3 buildscripts/resmoke.py suiteconfig --suite=core
```
List available tags:
```bash
python3 buildscripts/resmoke.py list-tags
```
## Output Format and Parallel Execution
### Output Structure
Resmoke logs are prefixed with component tags:
```
[resmoke] 2026-03-02T12:59:22.431Z Starting test execution...
[job0] 2026-03-02T12:59:23.123Z Running jstests/core/administrative/auth1.js...
[job1] 2026-03-02T12:59:23.145Z Running jstests/core/administrative/auth2.js...
```
Useful patterns to grep for:
- `\[resmoke\]` - Main orchestrator messages
- `\[job\d+\]` - Individual job execution logs
- `FAILED` - Test failures
- `PASSED` - Successful test completion
- `Running` - Currently executing test
- `Starting.*test|test.*complete` - Test execution lifecycle
### Parallel Execution
By default, resmoke runs tests sequentially. Use `-j` for parallel execution:
```bash
python3 buildscripts/resmoke.py run --suites=core -j4
python3 buildscripts/resmoke.py run --suites=core -j$(nproc)
```
Each parallel job:
- Gets a unique `jobN` prefix in logs for identification
- Has isolated port ranges for fixture processes
- Maintains separate data directories
When debugging failures in parallel runs, use `--continueOnFailure` to run all tests before reporting:
```bash
python3 buildscripts/resmoke.py run --suites=core -j4 --continueOnFailure
```
> [!NOTE]
> Full suites can take well over 10 minutes. For long runs, use `--alwaysUseLogFiles`; the per-job log files can be read or searched incrementally while the suite is running.
## Common Options
| Option | Description |
| ----------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| `-n` / `--dryRun=tests` | Dry run mode (shows tests without running) |
| `-j JOBS` | Parallel job count |
| `--continueOnFailure` | Continue after test failures |
| `--testTimeout SECONDS` | Timeout per test |
| `--logLevel DEBUG` | Verbose logging |
| `--alwaysUseLogFiles` | Write per-job log files (`resmoke_job0.log`, etc.) to the working directory; useful for monitoring long-running suites |
### Additional options
You can always run `resmoke -h` to discover all available options. It can also be passed to subcommands like `resmoke run -h`. If you discover something generically helpful that isn't listed anywhere in this skill yet, you should add it to one of the other files so other agents can benefit.
## Debugging Failed Tests
### Hang Analyzer
Analyze hung processes:
```bash
python3 buildscripts/resmoke.py hang-analyzer -p mongod,mongos
```
### Core Dump Analysis
Analyze core dumps:
```bash
python3 buildscripts/resmoke.py core-analyzer -c core.dump
```
## Advanced Topics
For detailed information on:
- **Advanced options & multiversion testing**: See [advanced-usage.md](advanced-usage.md)

View File

@ -0,0 +1,95 @@
# Resmoke Advanced Usage
## Multiversion Testing
Run tests against multiple MongoDB versions:
```bash
# Generate multiversion config
python3 buildscripts/resmoke.py multiversion-config -f multiversion.yml
# Run with multiversion binaries
python3 buildscripts/resmoke.py run --suites=aggregation_multiversion_fuzzer_last_lts \
--multiversionDir=/path/to/binaries
```
## Tag-based Test Selection
Include/exclude tests by tags:
```bash
# Exclude tests with specific tags
python3 buildscripts/resmoke.py run --suites=core --excludeWithAnyTags=requires_replication,sharding
# Include only tests with specific tags
python3 buildscripts/resmoke.py run --suites=core --includeWithAnyTags=requires_auth
```
Common tags: `requires_auth`, `requires_replication`, `requires_sharding`, `assumes_standalone_mongod`, `assumes_against_mongod_not_mongos`
## Fixture Configuration
Override fixture settings via command line:
```bash
# Replica set with 3 nodes
python3 buildscripts/resmoke.py run --suites=replica_sets --numReplSetNodes=3
# Sharded cluster with 2 shards
python3 buildscripts/resmoke.py run --suites=sharding --numShards=2
# Custom storage engine
python3 buildscripts/resmoke.py run --suites=core --storageEngine=wiredTiger
```
## Server Parameters
Set mongod/mongos parameters:
```bash
python3 buildscripts/resmoke.py run --suites=core \
--mongodSetParameters='{enableTestCommands: 1, logComponentVerbosity: {command: 2}}'
```
## Repeating Tests
Repeat tests for flake detection:
```bash
# Repeat each test N times
python3 buildscripts/resmoke.py run --suites=core --repeatTests=5
# Repeat for minimum duration
python3 buildscripts/resmoke.py run --suites=core --repeatTestsSecs=300
# Shuffle test order
python3 buildscripts/resmoke.py run --suites=core --shuffle
```
## Running Without Hooks
Disable all test hooks for debugging:
```bash
python3 buildscripts/resmoke.py run --suites=core --noHooks
```
## Powercycle Tests
Test server robustness across power events:
```bash
python3 buildscripts/resmoke.py powercycle --sshConnectionString=user@host
```
## Fuzz Testing
Generate and run with fuzzed configurations:
```bash
# Generate fuzz config
python3 buildscripts/resmoke.py generate-fuzz-config
# Run with fuzzing enabled
python3 buildscripts/resmoke.py run --suites=core --fuzzMongodConfigs=on --configFuzzSeed=12345
```

View File

@ -7,3 +7,5 @@ filters:
Not production code, more like linter config. All teams should be able to edit
their own module definitions, but there are advantages to keeping all modules
defined in a single file.
- filter: "/.agents/skills/OWNERS.yml"
justification: Teams should be able to add their own skills and claim them without asking permission.