SERVER-121513: add an initial version of a resmoke agent skill (#49498)
GitOrigin-RevId: eb21af19273ec93a29f8dc1962db715fe45789f2
This commit is contained in:
parent
0922c63c5c
commit
c26287edb2
5
.agents/skills/OWNERS.yml
Normal file
5
.agents/skills/OWNERS.yml
Normal file
@ -0,0 +1,5 @@
|
||||
version: 2.0.0
|
||||
filters:
|
||||
- "resmoke":
|
||||
approvers:
|
||||
- 10gen/devprod-correctness
|
||||
175
.agents/skills/resmoke/SKILL.md
Normal file
175
.agents/skills/resmoke/SKILL.md
Normal file
@ -0,0 +1,175 @@
|
||||
---
|
||||
name: resmoke
|
||||
description: Run MongoDB integration and correctness tests using the resmoke test orchestrator. Use when working with jstests, test suites, or running MongoDB integration tests.
|
||||
---
|
||||
|
||||
# Resmoke Test Orchestrator
|
||||
|
||||
Resmoke is MongoDB's integration testing framework for running JavaScript tests, C++ integration tests, and multiversion tests. Resmoke manages its own clean test environment; no pre-cleanup of data directories is needed.
|
||||
|
||||
## Basic Usage
|
||||
|
||||
Activate the virtual environment before running resmoke:
|
||||
|
||||
```bash
|
||||
source python3-venv/bin/activate
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
Run a test suite:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py run --suites=core
|
||||
```
|
||||
|
||||
Run multiple suites:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py run --suites=core,noPassthrough
|
||||
```
|
||||
|
||||
Run a single test file:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py run --suites=core jstests/core/administrative/auth1.js
|
||||
```
|
||||
|
||||
Run multiple test files:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py run --suites=core jstests/core/administrative/auth1.js jstests/core/administrative/auth2.js
|
||||
```
|
||||
|
||||
Run tests matching a pattern:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py run --suites=core jstests/core/administrative/*.js
|
||||
```
|
||||
|
||||
Dry run to see what tests would execute:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py run --suites=core -n
|
||||
```
|
||||
|
||||
### Common Test Suites
|
||||
|
||||
| Suite | Purpose |
|
||||
| --------------- | -------------------------------------------- |
|
||||
| `core` | General mongod tests (runs as standalone) |
|
||||
| `noPassthrough` | Tests that manage their own mongod instances |
|
||||
| `auth` | Authentication/authorization tests |
|
||||
| `sharding` | Sharded cluster tests |
|
||||
| `replica_sets` | Replication tests |
|
||||
| `concurrency` | Concurrency and locking tests |
|
||||
|
||||
## Exploring Tests
|
||||
|
||||
List available suites:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py list-suites
|
||||
```
|
||||
|
||||
Discover tests in a suite:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py test-discovery --suite=core
|
||||
```
|
||||
|
||||
View suite configuration:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py suiteconfig --suite=core
|
||||
```
|
||||
|
||||
List available tags:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py list-tags
|
||||
```
|
||||
|
||||
## Output Format and Parallel Execution
|
||||
|
||||
### Output Structure
|
||||
|
||||
Resmoke logs are prefixed with component tags:
|
||||
|
||||
```
|
||||
[resmoke] 2026-03-02T12:59:22.431Z Starting test execution...
|
||||
[job0] 2026-03-02T12:59:23.123Z Running jstests/core/administrative/auth1.js...
|
||||
[job1] 2026-03-02T12:59:23.145Z Running jstests/core/administrative/auth2.js...
|
||||
```
|
||||
|
||||
Useful patterns to grep for:
|
||||
|
||||
- `\[resmoke\]` - Main orchestrator messages
|
||||
- `\[job\d+\]` - Individual job execution logs
|
||||
- `FAILED` - Test failures
|
||||
- `PASSED` - Successful test completion
|
||||
- `Running` - Currently executing test
|
||||
- `Starting.*test|test.*complete` - Test execution lifecycle
|
||||
|
||||
### Parallel Execution
|
||||
|
||||
By default, resmoke runs tests sequentially. Use `-j` for parallel execution:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py run --suites=core -j4
|
||||
python3 buildscripts/resmoke.py run --suites=core -j$(nproc)
|
||||
```
|
||||
|
||||
Each parallel job:
|
||||
|
||||
- Gets a unique `jobN` prefix in logs for identification
|
||||
- Has isolated port ranges for fixture processes
|
||||
- Maintains separate data directories
|
||||
|
||||
When debugging failures in parallel runs, use `--continueOnFailure` to run all tests before reporting:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py run --suites=core -j4 --continueOnFailure
|
||||
```
|
||||
|
||||
> [!NOTE]
|
||||
> Full suites can take well over 10 minutes. For long runs, use `--alwaysUseLogFiles`; the per-job log files can be read or searched incrementally while the suite is running.
|
||||
|
||||
## Common Options
|
||||
|
||||
| Option | Description |
|
||||
| ----------------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
||||
| `-n` / `--dryRun=tests` | Dry run mode (shows tests without running) |
|
||||
| `-j JOBS` | Parallel job count |
|
||||
| `--continueOnFailure` | Continue after test failures |
|
||||
| `--testTimeout SECONDS` | Timeout per test |
|
||||
| `--logLevel DEBUG` | Verbose logging |
|
||||
| `--alwaysUseLogFiles` | Write per-job log files (`resmoke_job0.log`, etc.) to the working directory; useful for monitoring long-running suites |
|
||||
|
||||
### Additional options
|
||||
|
||||
You can always run `resmoke -h` to discover all available options. It can also be passed to subcommands like `resmoke run -h`. If you discover something generically helpful that isn't listed anywhere in this skill yet, you should add it to one of the other files so other agents can benefit.
|
||||
|
||||
## Debugging Failed Tests
|
||||
|
||||
### Hang Analyzer
|
||||
|
||||
Analyze hung processes:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py hang-analyzer -p mongod,mongos
|
||||
```
|
||||
|
||||
### Core Dump Analysis
|
||||
|
||||
Analyze core dumps:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py core-analyzer -c core.dump
|
||||
```
|
||||
|
||||
## Advanced Topics
|
||||
|
||||
For detailed information on:
|
||||
|
||||
- **Advanced options & multiversion testing**: See [advanced-usage.md](advanced-usage.md)
|
||||
95
.agents/skills/resmoke/advanced-usage.md
Normal file
95
.agents/skills/resmoke/advanced-usage.md
Normal file
@ -0,0 +1,95 @@
|
||||
# Resmoke Advanced Usage
|
||||
|
||||
## Multiversion Testing
|
||||
|
||||
Run tests against multiple MongoDB versions:
|
||||
|
||||
```bash
|
||||
# Generate multiversion config
|
||||
python3 buildscripts/resmoke.py multiversion-config -f multiversion.yml
|
||||
|
||||
# Run with multiversion binaries
|
||||
python3 buildscripts/resmoke.py run --suites=aggregation_multiversion_fuzzer_last_lts \
|
||||
--multiversionDir=/path/to/binaries
|
||||
```
|
||||
|
||||
## Tag-based Test Selection
|
||||
|
||||
Include/exclude tests by tags:
|
||||
|
||||
```bash
|
||||
# Exclude tests with specific tags
|
||||
python3 buildscripts/resmoke.py run --suites=core --excludeWithAnyTags=requires_replication,sharding
|
||||
|
||||
# Include only tests with specific tags
|
||||
python3 buildscripts/resmoke.py run --suites=core --includeWithAnyTags=requires_auth
|
||||
```
|
||||
|
||||
Common tags: `requires_auth`, `requires_replication`, `requires_sharding`, `assumes_standalone_mongod`, `assumes_against_mongod_not_mongos`
|
||||
|
||||
## Fixture Configuration
|
||||
|
||||
Override fixture settings via command line:
|
||||
|
||||
```bash
|
||||
# Replica set with 3 nodes
|
||||
python3 buildscripts/resmoke.py run --suites=replica_sets --numReplSetNodes=3
|
||||
|
||||
# Sharded cluster with 2 shards
|
||||
python3 buildscripts/resmoke.py run --suites=sharding --numShards=2
|
||||
|
||||
# Custom storage engine
|
||||
python3 buildscripts/resmoke.py run --suites=core --storageEngine=wiredTiger
|
||||
```
|
||||
|
||||
## Server Parameters
|
||||
|
||||
Set mongod/mongos parameters:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py run --suites=core \
|
||||
--mongodSetParameters='{enableTestCommands: 1, logComponentVerbosity: {command: 2}}'
|
||||
```
|
||||
|
||||
## Repeating Tests
|
||||
|
||||
Repeat tests for flake detection:
|
||||
|
||||
```bash
|
||||
# Repeat each test N times
|
||||
python3 buildscripts/resmoke.py run --suites=core --repeatTests=5
|
||||
|
||||
# Repeat for minimum duration
|
||||
python3 buildscripts/resmoke.py run --suites=core --repeatTestsSecs=300
|
||||
|
||||
# Shuffle test order
|
||||
python3 buildscripts/resmoke.py run --suites=core --shuffle
|
||||
```
|
||||
|
||||
## Running Without Hooks
|
||||
|
||||
Disable all test hooks for debugging:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py run --suites=core --noHooks
|
||||
```
|
||||
|
||||
## Powercycle Tests
|
||||
|
||||
Test server robustness across power events:
|
||||
|
||||
```bash
|
||||
python3 buildscripts/resmoke.py powercycle --sshConnectionString=user@host
|
||||
```
|
||||
|
||||
## Fuzz Testing
|
||||
|
||||
Generate and run with fuzzed configurations:
|
||||
|
||||
```bash
|
||||
# Generate fuzz config
|
||||
python3 buildscripts/resmoke.py generate-fuzz-config
|
||||
|
||||
# Run with fuzzing enabled
|
||||
python3 buildscripts/resmoke.py run --suites=core --fuzzMongodConfigs=on --configFuzzSeed=12345
|
||||
```
|
||||
2
.github/ALLOWED_UNOWNED_FILES.yml
vendored
2
.github/ALLOWED_UNOWNED_FILES.yml
vendored
@ -7,3 +7,5 @@ filters:
|
||||
Not production code, more like linter config. All teams should be able to edit
|
||||
their own module definitions, but there are advantages to keeping all modules
|
||||
defined in a single file.
|
||||
- filter: "/.agents/skills/OWNERS.yml"
|
||||
justification: Teams should be able to add their own skills and claim them without asking permission.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user