mongodb/mongo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

SERVER-123092 Document FuzzTest API and usage (#50860)

Browse Source 
GitOrigin-RevId: 41d3a16cc9d74e53c05550140dc1f4c9eab9bec1
This commit is contained in:
Mike Merrill 2026-04-02 13:30:09 -04:00 committed by MongoDB Bot
parent 408a839ea9
commit 76ad9f4d53
4 changed files with 269 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
docs/OWNERS.yml
View File

@ -48,3 +48,6 @@ filters:
- "priority_port.md":
approvers:
- 10gen/server-networking-and-observability
- "fuzztest.md":
approvers:
- 10gen/platsec-server

259
docs/fuzztest.md Normal file
View File

@ -0,0 +1,259 @@
---
title: FuzzTest
---
FuzzTest is a coverage-guided fuzzing framework for C++ that integrates
directly with GoogleTest. FuzzTest lets you write _property-based tests_: you
describe the shape of your inputs using typed _domains_, and the framework
generates and mutates values that satisfy those constraints. FuzzTest
uses Centipede as its fuzzing engine and AUBSAN to surface undefined
behavior.
# When to use FuzzTest
- Your function under test accepts structured inputs (integers, strings,
custom types, BSON objects, etc.) rather than an opaque byte blob.
- You want to express correctness properties beyond "does not crash", such
as API invariants, differential equivalence, or roundtrip symmetry.
- You want a fuzz test that also runs cleanly as a unit test in normal CI,
without needing a special fuzzer build variant.
# How to use FuzzTest
## The property function and FUZZ_TEST macro
A FuzzTest consists of a _property function_ and a registration macro.
The property function is a plain C++ function whose parameters define the
inputs to fuzz. The framework calls it repeatedly with generated values,
looking for any call that triggers an assertion failure or sanitizer
error.
```cpp
#include "fuzztest/fuzztest.h"
#include "gtest/gtest.h"
void MyFunctionFuzzer(const std::string& input) {
MyFunction(input); // sanitizers catch undefined behavior implicitly
}
FUZZ_TEST(MyTestSuite, MyFunctionFuzzer);
```
When no `.WithDomains()` clause is provided, each parameter defaults to
`fuzztest::Arbitrary<T>()`, which covers most standard library types.
## Specifying input domains
Use `.WithDomains()` to constrain the generated inputs:
> ⚠️ **Warning:** Never initialize input domains with global objects initialized in other compilation units. For more information see [Fuzz_Test Macro](https://github.com/google/fuzztest/blob/main/doc/fuzz-test-macro.md)
```cpp
void ProcessRequestFuzzer(int opcode, const std::string& payload) {
ProcessRequest(opcode, payload);
}
FUZZ_TEST(MyTestSuite, ProcessRequestFuzzer)
.WithDomains(/*opcode=*/fuzztest::InRange(1, 255),
/*payload=*/fuzztest::Arbitrary<std::string>());
```
FuzzTest ships with a rich set of built-in domains. A complete list of default types implemented in fuzztest can be found in the [Fuzztest Domain Reference](https://github.com/google/fuzztest/blob/main/doc/domains-reference.md). Also see [BSON Fuzzing](#fuzzing-bson).
## Providing seeds
Seed values give the fuzzer a head start by providing known-interesting
inputs to mutate:
> ⚠️ **Warning:** Never initialize seeds with global objects initialized in other compilation units. For more information see [Fuzz_Test Macro](https://github.com/google/fuzztest/blob/main/doc/fuzz-test-macro.md)
```cpp
FUZZ_TEST(MyTestSuite, ProcessRequestFuzzer)
.WithDomains(fuzztest::InRange(1, 255),
fuzztest::Arbitrary<std::string>())
.WithSeeds({{1, "hello"}, {255, ""}});
```
You can also load seeds from a directory checked into the repository:
```cpp
FUZZ_TEST(MyTestSuite, ProcessRequestFuzzer)
.WithSeeds(fuzztest::ReadFilesFromDirectory(
absl::StrCat(std::getenv("TEST_SRCDIR"), "/path/to/corpus")));
```
## Common correctness patterns
Beyond "does not crash", FuzzTest makes it easy to assert higher-level
properties.
**Roundtrip**: verify that encode→decode (or serialize→parse) is the
identity:
```cpp
void SerializeRoundtrips(const MyMessage& msg) {
auto serialized = Serialize(msg);
auto parsed = Parse(serialized);
EXPECT_EQ(msg, parsed);
}
FUZZ_TEST(MyTestSuite, SerializeRoundtrips);
```
**Differential fuzzing**: compare two implementations of the same
operation:
```cpp
void ImplementationsAgree(const std::string& input) {
EXPECT_EQ(NewImpl(input), OldImpl(input));
}
FUZZ_TEST(MyTestSuite, ImplementationsAgree);
```
## Using fixtures
If your test requires expensive one-time setup (e.g. starting a service),
use a fixture with `FUZZ_TEST_F`. Any default-constructible class can be
a fixture; the constructor and destructor run once for the whole fuzz test,
not once per iteration. When using fixtures, care should be taken to ensure that only the initial fixture state is retained. Program state created during a test _**must**_ not affect or be affected by subsequent iterations.
```cpp
class MyServiceFuzzTest {
public:
MyServiceFuzzTest() { service_.Start(); }
~MyServiceFuzzTest() { service_.Stop(); }
void RequestFuzzer(const std::string& input) {
service_.Handle(input);
}
private:
MyService service_;
};
FUZZ_TEST_F(MyServiceFuzzTest, RequestFuzzer);
```
## Fuzzing BSON
MongoDB provides a custom FuzzTest domain for generating valid BSON
objects: `mongo::bson_mutator::BSONObjImpl`. It is registered as the
`Arbitrary<ConstSharedBuffer>` specialization, so any fuzz test that
accepts a `ConstSharedBuffer` will automatically receive well-formed BSON.
```cpp
#include "mongo/bson/bson_mutator/bson_mutator.h"
void MyCommandFuzzer(ConstSharedBuffer input) {
BSONObj obj(input);
MyCommand(obj);
}
FUZZ_TEST(MyCommandFuzzTest, MyCommandFuzzer);
```
To constrain which fields are present and their types, use the
`.With<Type>()` builders:
```cpp
FUZZ_TEST(MyCommandFuzzTest, MyCommandFuzzer)
.WithDomains(fuzztest::Arbitrary<mongo::ConstSharedBuffer>()
.WithInt("count")
.WithString("name")
.WithLong("limit", fuzztest::InRange(0LL, 1000LL)));
```
Fields added via `.With<Type>()` are not guaranteed to appear in every
generated object, which exercises missing-field error handling as well.
Use `.WithVariant()` when a field may legally hold more than one type:
```cpp
fuzztest::Arbitrary<mongo::ConstSharedBuffer>()
.WithVariant("value", {
{BSONType::numberInt, fuzztest::InRange(0, 100)},
{BSONType::numberLong, fuzztest::InRange(0LL, 100000LL)},
});
```
Use `.WithAny()` when a key should be present but its type is
unconstrained:
```cpp
fuzztest::Arbitrary<mongo::ConstSharedBuffer>().WithAny("filter");
```
## Bazel target
Use `mongo_cc_fuzztest` (from `//bazel:mongo_src_rules.bzl`) to declare a
fuzz test target. It links in FuzzTest and GoogleTest automatically:
```python
mongo_cc_fuzztest(
name = "my_command_fuzztest",
srcs = ["my_command_fuzztest.cpp"],
deps = [
"//src/mongo:base",
"//src/mongo/db/commands:my_command",
],
)
```
# Running FuzzTest
## Unit test mode
Every `FUZZ_TEST` is also a regular GoogleTest test. In unit test mode,
the property function is called a small number of times with minimal inputs. This lets fuzz tests run in ordinary CI
alongside unit tests:
```
bazel test --compiler_type=clang --config=fuzztest --fsan --opt=debug --allocator=system +my_command_fuzztest
```
## Fuzzing mode
Fuzzing mode enables sanitizer and coverage instrumentation and runs the
test indefinitely (or until a crash is found). It requires the `fsan`
build configuration. Check our Evergreen configuration for the current
bazel arguments, or run:
```
bazel run --compiler_type=clang --config=fuzztest --fsan --opt=debug --allocator=system +my_command_fuzztest -- \
--fuzz=MyCommandFuzzTest.MyCommandFuzzer
```
To fuzz all tests in a target for a fixed duration, use `--fuzz_for`:
```
bazel run --compiler_type=clang --config=fuzztest --fsan --opt=debug --allocator=system +my_command_fuzztest -- --fuzz_for=60s
```
## Evergreen
Fuzz tests defined in bazel using `mongo_cc_fuzztest` will periodically run on the master branch in evergreen. The compiled tests and their associated corpus are saved to S3 and can be downloaded for debugging issues. The corpus is reused between evergreen runs in order to increase fuzzing coverage.
## Useful flags
| Flag | Effect |
| ---------------------------------------- | --------------------------------- |
| `--fuzz=Suite.Test` | Fuzz a single test indefinitely |
| `--fuzz_for=T` | Fuzz all tests for duration `T` |
| `--rss_limit_mb=N` | Abort if memory exceeds N MB |
| `--time_limit_per_input=T` | Abort an input after duration `T` |
| `--reproduce_findings_as_separate_tests` | Reruns tests with crashing inputs |
| `--helpful` | Describes fuzztest's flags |
# Debugging Crashes in FuzzTest
## Reproducing crashes
```
bazel run --compiler_type=clang --config=fuzztest --fsan --opt=debug --allocator=system +my_command_fuzztest -- --reproduce_findings_as_separate_tests
```
# References
- [FuzzTest overview](https://github.com/google/fuzztest/blob/main/doc/overview.md)
- [The FUZZ_TEST macro](https://github.com/google/fuzztest/blob/main/doc/fuzz-test-macro.md)
- [Domains reference](https://github.com/google/fuzztest/blob/main/doc/domains-reference.md)
- [Use cases](https://github.com/google/fuzztest/blob/main/doc/use-cases.md)
- [Test fixtures](https://github.com/google/fuzztest/blob/main/doc/fixtures.md)
- [Flags reference](https://github.com/google/fuzztest/blob/main/doc/flags-reference.md)
- [FuzzTest on GitHub](https://github.com/google/fuzztest)
- [BSON Mutator](../src/mongo/bson/bson_mutator/bson_mutator.h)

4
docs/libfuzzer.md
View File

@ -2,6 +2,8 @@
title: LibFuzzer
---
> **!!NOTE!!**: LibFuzzer is deprecated and should not be used for new fuzz tests. See [FuzzTest](fuzztest.md) for new fuzzing implementations
LibFuzzer is a tool for performing coverage guided fuzzing of C/C++
code. LibFuzzer will try to trigger AUBSAN failures in a function you
provide, by repeatedly calling it with a carefully crafted byte array as
@ -12,6 +14,8 @@ possible behavior.
# When to use LibFuzzer
> **!!NOTE!!**: LibFuzzer is deprecated and should not be used for new fuzz tests. See [FuzzTest](fuzztest.md) for new fuzzing implementations
LibFuzzer is great for testing functions which accept a opaque blob of
untrusted user-provided data.

6
src/mongo/bson/bson_mutator/bson_mutator_util.cpp
View File

@ -422,7 +422,7 @@ fuzztest::Domain<BSONBinDataOwned> ArbitraryColumn() {
*
* Example:
* FUZZ_TEST(TestSuite, FuzzerTest)
* .WithDomains(ArbitraryMD5Type())
* .WithDomains(ArbitrarySensitive())
* .WithSeeds({
* {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16}
* });
@ -445,7 +445,7 @@ fuzztest::Domain<BSONBinDataOwned> ArbitrarySensitive() {
*
* Example:
* FUZZ_TEST(TestSuite, FuzzerTest)
* .WithDomains(ArbitraryMD5Type())
* .WithDomains(ArbitraryVector())
* .WithSeeds({
* {1,2,3,4,5},
* {6,7,8,9,0},
@ -469,7 +469,7 @@ fuzztest::Domain<BSONBinDataOwned> ArbitraryVector() {
*
* Example:
* FUZZ_TEST(TestSuite, FuzzerTest)
* .WithDomains(ArbitraryMD5Type())
* .WithDomains(ArbitraryUserData())
* .WithSeeds({
* {{1,2,3,4,5}, 128},
* {{5,4,3,2,1}, 200},