mongodb/mongo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

SERVER-121918: Implement MacOS Cross Compilation with RBE (#49787)

Browse Source 
GitOrigin-RevId: 6f2ab2faab8d2503c77373ca2f086d4a8668f711
This commit is contained in:
Zack Winter 2026-03-18 09:45:41 -07:00 committed by MongoDB Bot
parent 14ad2a9d3f
commit a62ba86084
20 changed files with 1358 additions and 349 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
.agents/skills/OWNERS.yml
View File

@ -1,5 +0,0 @@
version: 2.0.0
filters:
- "resmoke":
approvers:
- 10gen/devprod-correctness

175
.agents/skills/resmoke/SKILL.md
View File

@ -1,175 +0,0 @@
---
name: resmoke
description: Run MongoDB integration and correctness tests using the resmoke test orchestrator. Use when working with jstests, test suites, or running MongoDB integration tests.
---
# Resmoke Test Orchestrator
Resmoke is MongoDB's integration testing framework for running JavaScript tests, C++ integration tests, and multiversion tests. Resmoke manages its own clean test environment; no pre-cleanup of data directories is needed.
## Basic Usage
Activate the virtual environment before running resmoke:
```bash
source python3-venv/bin/activate
```
### Running Tests
Run a test suite:
```bash
python3 buildscripts/resmoke.py run --suites=core
```
Run multiple suites:
```bash
python3 buildscripts/resmoke.py run --suites=core,no_passthrough
```
Run a single test file:
```bash
python3 buildscripts/resmoke.py run --suites=core jstests/core/administrative/auth1.js
```
Run multiple test files:
```bash
python3 buildscripts/resmoke.py run --suites=core jstests/core/administrative/auth1.js jstests/core/administrative/auth2.js
```
Run tests matching a pattern:
```bash
python3 buildscripts/resmoke.py run --suites=core jstests/core/administrative/*.js
```
Dry run to see what tests would execute:
```bash
python3 buildscripts/resmoke.py run --suites=core -n
```
### Common Test Suites
| Suite | Purpose |
| ---------------- | -------------------------------------------- |
| `core` | General mongod tests (runs as standalone) |
| `no_passthrough` | Tests that manage their own mongod instances |
| `auth` | Authentication/authorization tests |
| `sharding` | Sharded cluster tests |
| `replica_sets` | Replication tests |
| `concurrency` | Concurrency and locking tests |
## Exploring Tests
List available suites:
```bash
python3 buildscripts/resmoke.py list-suites
```
Discover tests in a suite:
```bash
python3 buildscripts/resmoke.py test-discovery --suite=core
```
View suite configuration:
```bash
python3 buildscripts/resmoke.py suiteconfig --suite=core
```
List available tags:
```bash
python3 buildscripts/resmoke.py list-tags
```
## Output Format and Parallel Execution
### Output Structure
Resmoke logs are prefixed with component tags:
```
[resmoke] 2026-03-02T12:59:22.431Z Starting test execution...
[job0] 2026-03-02T12:59:23.123Z Running jstests/core/administrative/auth1.js...
[job1] 2026-03-02T12:59:23.145Z Running jstests/core/administrative/auth2.js...
```
Useful patterns to grep for:
- `\[resmoke\]` - Main orchestrator messages
- `\[job\d+\]` - Individual job execution logs
- `FAILED` - Test failures
- `PASSED` - Successful test completion
- `Running` - Currently executing test
- `Starting.*test|test.*complete` - Test execution lifecycle
### Parallel Execution
By default, resmoke runs tests sequentially. Use `-j` for parallel execution:
```bash
python3 buildscripts/resmoke.py run --suites=core -j4
python3 buildscripts/resmoke.py run --suites=core -j$(nproc)
```
Each parallel job:
- Gets a unique `jobN` prefix in logs for identification
- Has isolated port ranges for fixture processes
- Maintains separate data directories
When debugging failures in parallel runs, use `--continueOnFailure` to run all tests before reporting:
```bash
python3 buildscripts/resmoke.py run --suites=core -j4 --continueOnFailure
```
> [!NOTE]
> Full suites can take well over 10 minutes. For long runs, use `--alwaysUseLogFiles`; the per-job log files can be read or searched incrementally while the suite is running.
## Common Options
| Option | Description |
| ----------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| `-n` / `--dryRun=tests` | Dry run mode (shows tests without running) |
| `-j JOBS` | Parallel job count |
| `--continueOnFailure` | Continue after test failures |
| `--testTimeout SECONDS` | Timeout per test |
| `--logLevel DEBUG` | Verbose logging |
| `--alwaysUseLogFiles` | Write per-job log files (`resmoke_job0.log`, etc.) to the working directory; useful for monitoring long-running suites |
### Additional options
You can always run `resmoke -h` to discover all available options. It can also be passed to subcommands like `resmoke run -h`. If you discover something generically helpful that isn't listed anywhere in this skill yet, you should add it to one of the other files so other agents can benefit.
## Debugging Failed Tests
### Hang Analyzer
Analyze hung processes:
```bash
python3 buildscripts/resmoke.py hang-analyzer -p mongod,mongos
```
### Core Dump Analysis
Analyze core dumps:
```bash
python3 buildscripts/resmoke.py core-analyzer -c core.dump
```
## Advanced Topics
For detailed information on:
- **Advanced options & multiversion testing**: See [advanced-usage.md](advanced-usage.md)

95
.agents/skills/resmoke/advanced-usage.md
View File

@ -1,95 +0,0 @@
# Resmoke Advanced Usage
## Multiversion Testing
Run tests against multiple MongoDB versions:
```bash
# Generate multiversion config
python3 buildscripts/resmoke.py multiversion-config -f multiversion.yml
# Run with multiversion binaries
python3 buildscripts/resmoke.py run --suites=aggregation_multiversion_fuzzer_last_lts \
--multiversionDir=/path/to/binaries
```
## Tag-based Test Selection
Include/exclude tests by tags:
```bash
# Exclude tests with specific tags
python3 buildscripts/resmoke.py run --suites=core --excludeWithAnyTags=requires_replication,sharding
# Include only tests with specific tags
python3 buildscripts/resmoke.py run --suites=core --includeWithAnyTags=requires_auth
```
Common tags: `requires_auth`, `requires_replication`, `requires_sharding`, `assumes_standalone_mongod`, `assumes_against_mongod_not_mongos`
## Fixture Configuration
Override fixture settings via command line:
```bash
# Replica set with 3 nodes
python3 buildscripts/resmoke.py run --suites=replica_sets --numReplSetNodes=3
# Sharded cluster with 2 shards
python3 buildscripts/resmoke.py run --suites=sharding --numShards=2
# Custom storage engine
python3 buildscripts/resmoke.py run --suites=core --storageEngine=wiredTiger
```
## Server Parameters
Set mongod/mongos parameters:
```bash
python3 buildscripts/resmoke.py run --suites=core \
--mongodSetParameters='{enableTestCommands: 1, logComponentVerbosity: {command: 2}}'
```
## Repeating Tests
Repeat tests for flake detection:
```bash
# Repeat each test N times
python3 buildscripts/resmoke.py run --suites=core --repeatTests=5
# Repeat for minimum duration
python3 buildscripts/resmoke.py run --suites=core --repeatTestsSecs=300
# Shuffle test order
python3 buildscripts/resmoke.py run --suites=core --shuffle
```
## Running Without Hooks
Disable all test hooks for debugging:
```bash
python3 buildscripts/resmoke.py run --suites=core --noHooks
```
## Powercycle Tests
Test server robustness across power events:
```bash
python3 buildscripts/resmoke.py powercycle --sshConnectionString=user@host
```
## Fuzz Testing
Generate and run with fuzzed configurations:
```bash
# Generate fuzz config
python3 buildscripts/resmoke.py generate-fuzz-config
# Run with fuzzing enabled
python3 buildscripts/resmoke.py run --suites=core --fuzzMongodConfigs=on --configFuzzSeed=12345
```

18
.bazelrc
View File

@ -675,3 +675,21 @@ try-import %workspace%/.bazelrc.fuzztest
# Repository root absolute path to set --execution_log_compact_file
#try-import %workspace%/.bazelrc.exec_log_file
# ============================================================
# macOS Cross-Compilation Configuration
# ============================================================
# To cross-compile for macOS from Linux, you need:
# LLVM and the macOS SDK are automatically downloaded from S3.
# To override with local paths, set LLVM_PATH and/or MACOS_SDK_PATH env vars.
#
# The toolchain is hermetic (LLVM tools + SDK placed into the repo),
# so both local and remote execution work.
#
# Cross-compile for macOS arm64
build:macos-cross-arm64 --platforms=//bazel/platforms:macos_arm64_cross
build:macos-cross-arm64 --copt=-w
# Cross-compile for macOS x86_64
build:macos-cross-x86_64 --platforms=//bazel/platforms:macos_x86_64_cross
build:macos-cross-x86_64 --copt=-w

1
CLAUDE.md
View File

@ -1 +0,0 @@
@AGENTS.md

8
MODULE.bazel
View File

@ -267,6 +267,14 @@ register_toolchains(
"@mongo_apple_toolchain//...",
)
# Cross-compilation toolchain for building macOS binaries on Linux
setup_mongo_apple_cross_toolchains_extension = use_extension("//bazel/toolchains/cc/mongo_apple_cross:mongo_apple_cross_toolchain.bzl", "setup_mongo_apple_cross_toolchain_extension")
use_repo(setup_mongo_apple_cross_toolchains_extension, "mongo_apple_cross_toolchain", "mongo_apple_cross_toolchain_files")
register_toolchains(
"@mongo_apple_cross_toolchain//...",
)
setup_mongo_python_toolchains = use_extension("//bazel:bzlmod.bzl", "setup_mongo_python_toolchains")
use_repo(
setup_mongo_python_toolchains,

28
MODULE.bazel.lock generated
View File

@ -491,6 +491,34 @@
]
}
},
"//bazel/toolchains/cc/mongo_apple_cross:mongo_apple_cross_toolchain.bzl%setup_mongo_apple_cross_toolchain_extension": {
"general": {
"bzlTransitiveDigest": "oskcsYFrmneRRo1v+AhRS4kSzDQ2d421jHvWL0A1SRA=",
"usagesDigest": "EVkQddf0MbwFmuJ26u4/NZd4P4Z0AhChzItbpAItozs=",
"recordedFileInputs": {},
"recordedDirentsInputs": {},
"envVariables": {},
"generatedRepoSpecs": {
"mongo_apple_cross_toolchain": {
"bzlFile": "@@//bazel/toolchains/cc/mongo_apple_cross:mongo_apple_cross_toolchain.bzl",
"ruleClassName": "mongo_apple_cross_toolchain_config",
"attributes": {}
},
"mongo_apple_cross_toolchain_files": {
"bzlFile": "@@//bazel/toolchains/cc/mongo_apple_cross:mongo_apple_cross_toolchain.bzl",
"ruleClassName": "mongo_apple_cross_toolchain_files_config",
"attributes": {}
}
},
"recordedRepoMappingEntries": [
[
"",
"build_bazel_apple_support",
"apple_support~"
]
]
}
},
"//bazel/toolchains/cc/mongo_windows:mongo_toolchain.bzl%setup_mongo_windows_toolchain_extension": {
"general": {
"bzlTransitiveDigest": "dHLyfzMhaUTIIP7LsjQxVGJHz6n+z1RfQNYH+/O+1Iw=",

19
bazel/platforms/BUILD.bazel
View File

@ -93,6 +93,25 @@ platform(
],
)
# Cross-compilation platforms for building macOS binaries on Linux
# These use the same target constraints as native macOS but are intended
# for use when building from a Linux host.
platform(
name = "macos_arm64_cross",
constraint_values = [
"@platforms//cpu:arm64",
"@platforms//os:macos",
],
)
platform(
name = "macos_x86_64_cross",
constraint_values = [
"@platforms//cpu:x86_64",
"@platforms//os:macos",
],
)
platform(
name = "linux_ppc64le",
constraint_values = [

44
bazel/toolchains/cc/mongo_apple/BUILD.tmpl
View File

@ -7,7 +7,17 @@ load("@//bazel/toolchains/cc:mongo_defines.bzl", "MONGO_GLOBAL_DEFINES")
load(
"@//bazel/toolchains/cc:mongo_custom_features.bzl",
"FEATURES_ATTR_NAMES",
"get_common_features_attrs")
"get_common_features_attrs",
)
load(
"@//bazel/toolchains/cc:mongo_toolchain_selects.bzl",
"COMPRESS_DEBUG_ENABLED",
"COVERAGE_ENABLED",
"DEBUG_LEVEL",
"INTERNAL_THIN_LTO_ENABLED",
"LINKSTATIC_ENABLED",
"WARNINGS_AS_ERRORS_ENABLED",
)
# Helper target for the toolchain (see below):
filegroup(
@ -15,38 +25,6 @@ filegroup(
srcs = glob(["**/*"]),
)
DEBUG_LEVEL = select({
"@//bazel/config:gcc_or_clang_dbg_level_0": 0,
"@//bazel/config:gcc_or_clang_dbg_level_1": 1,
"@//bazel/config:gcc_or_clang_dbg_level_2": 2,
"@//bazel/config:gcc_or_clang_dbg_level_3": 3,
})
INTERNAL_THIN_LTO_ENABLED = select({
"@//bazel/config:thin_lto_enabled": True,
"@//conditions:default": False,
})
COVERAGE_ENABLED = select({
"@//bazel/config:gcov_enabled": True,
"@//conditions:default": False,
})
COMPRESS_DEBUG_ENABLED = select({
"@//bazel/config:compress_debug_compile_enabled": True,
"@//conditions:default": False,
})
WARNINGS_AS_ERRORS_ENABLED = select({
"@//bazel/config:warnings_as_errors_enabled": True,
"@//conditions:default": False,
})
LINKSTATIC_ENABLED = select({
"@//bazel/config:linkstatic_enabled": True,
"@//conditions:default": False,
})
[
cc_toolchain(
name = "cc-compiler-" + arch,

4
bazel/toolchains/cc/mongo_apple_cross/BUILD.bazel Normal file
View File

@ -0,0 +1,4 @@
# Cross-compilation toolchain for building macOS binaries on Linux
# This is a placeholder BUILD file; the actual toolchain is generated by the repository rule.
package(default_visibility = ["//visibility:public"])

143
bazel/toolchains/cc/mongo_apple_cross/BUILD.tmpl Normal file
View File

@ -0,0 +1,143 @@
"""Build file for macOS cross-compilation toolchain from Linux."""
package(default_visibility = ["//visibility:public"])
load(
"@//bazel/toolchains/cc/mongo_apple_cross:mongo_apple_cross_cc_toolchain_config.bzl",
"mongo_apple_cross_cc_toolchain_config",
)
load("@//bazel/toolchains/cc/mongo_apple_cross:mongo_apple_cross_toolchain.bzl", "get_supported_apple_cross_archs")
load("@//bazel/toolchains/cc:mongo_defines.bzl", "MONGO_GLOBAL_DEFINES")
load(
"@//bazel/toolchains/cc:mongo_custom_features.bzl",
"FEATURES_ATTR_NAMES",
"get_common_features_attrs",
)
load(
"@//bazel/toolchains/cc:mongo_toolchain_selects.bzl",
"COMPRESS_DEBUG_ENABLED",
"COVERAGE_ENABLED",
"DEBUG_LEVEL",
"INTERNAL_THIN_LTO_ENABLED",
"LINKSTATIC_ENABLED",
"WARNINGS_AS_ERRORS_ENABLED",
)
# Hermetic toolchain filegroups - these are the files Bazel ships to remote workers.
# LLVM tools + headers + SDK are symlinked into this repo by the repository rule.
filegroup(
name = "compiler_files",
srcs = glob([
"tools/clang",
"tools/clang++",
"tools/clang-*",
"include/**",
"sysroot/usr/include/**",
"sysroot/System/Library/Frameworks/**/*.h",
"sysroot/System/Library/Frameworks/**/*.modulemap",
]),
)
filegroup(
name = "linker_files",
srcs = glob([
"tools/clang",
"tools/clang++",
"tools/ld.lld",
"tools/ld64.lld",
"tools/lld",
"sysroot/usr/lib/**",
"sysroot/System/Library/Frameworks/**/*.tbd",
]),
)
filegroup(
name = "ar_files",
srcs = glob(["tools/llvm-ar"]),
)
filegroup(
name = "strip_files",
srcs = glob(["tools/llvm-strip"]),
)
filegroup(
name = "all_files",
srcs = [
":ar_files",
":compiler_files",
":linker_files",
":strip_files",
] + glob([
"tools/llvm-*",
]),
)
filegroup(
name = "empty",
srcs = [],
)
feature_attrs = get_common_features_attrs()
# Generate cc_toolchain for each supported architecture
[
cc_toolchain(
name = "cc-compiler-apple-cross-" + arch,
all_files = ":all_files",
ar_files = ":ar_files",
as_files = ":compiler_files",
compiler_files = ":compiler_files",
dwp_files = ":empty",
linker_files = ":linker_files",
objcopy_files = ":empty",
strip_files = ":strip_files",
supports_header_parsing = 1,
supports_param_files = 1,
toolchain_config = "cross_llvm_" + arch,
toolchain_identifier = "apple_cross_clang_toolchain_" + arch,
)
for arch, cpu in get_supported_apple_cross_archs().items()
]
# Generate toolchain configs for each architecture
[
mongo_apple_cross_cc_toolchain_config(
name = "cross_llvm_" + arch,
cpu = cpu,
compiler = "clang",
toolchain_identifier = "apple_cross_clang_toolchain_" + arch,
target_libc = "macosx",
abi_version = arch,
abi_libc_version = arch,
llvm_version = "%{llvm_version}",
min_macos_version = "%{min_macos_version}",
execroot_prefix = "%{execroot_prefix}",
tool_paths = {
"ar": "tools/llvm-ar",
"cpp": "tools/clang-cpp",
"ld": "tools/ld.lld",
"dwp": "tools/llvm-dwp",
"gcc": "tools/clang",
"g++": "tools/clang++",
"gcov": "tools/llvm-profdata",
"llvm-cov": "tools/llvm-cov",
"llvm-profdata": "tools/llvm-profdata",
"nm": "tools/llvm-nm",
"objcopy": "tools/llvm-objcopy",
"objdump": "tools/llvm-objdump",
"strip": "tools/llvm-strip",
},
optimization_level = feature_attrs[FEATURES_ATTR_NAMES.OPT_LEVEL],
debug_level = DEBUG_LEVEL,
supports_start_end_lib = False,
internal_thin_lto_enabled = INTERNAL_THIN_LTO_ENABLED,
coverage_enabled = COVERAGE_ENABLED,
compress_debug_enabled = COMPRESS_DEBUG_ENABLED,
warnings_as_errors_enabled = WARNINGS_AS_ERRORS_ENABLED,
linkstatic = LINKSTATIC_ENABLED,
global_defines = MONGO_GLOBAL_DEFINES,
)
for arch, cpu in get_supported_apple_cross_archs().items()
]

715
bazel/toolchains/cc/mongo_apple_cross/mongo_apple_cross_cc_toolchain_config.bzl Normal file
View File

@ -0,0 +1,715 @@
"""Cross-compilation toolchain configuration for building macOS binaries on Linux.
This toolchain is hermetic: all paths reference files within the generated external
repository (symlinked from LLVM and SDK), using execroot-relative paths so that both
local sandboxed builds and remote execution work correctly.
"""
load(
"@bazel_tools//tools/cpp:cc_toolchain_config_lib.bzl",
"artifact_name_pattern",
"feature",
"flag_group",
"flag_set",
"tool_path",
"variable_with_value",
)
load("@bazel_tools//tools/build_defs/cc:action_names.bzl", "ACTION_NAMES")
_OBJCPP_EXECUTABLE_ACTION_NAME = "objc++-executable"
all_compile_actions = [
ACTION_NAMES.c_compile,
ACTION_NAMES.cpp_compile,
ACTION_NAMES.linkstamp_compile,
ACTION_NAMES.assemble,
ACTION_NAMES.preprocess_assemble,
ACTION_NAMES.cpp_header_parsing,
ACTION_NAMES.cpp_module_compile,
ACTION_NAMES.cpp_module_codegen,
ACTION_NAMES.clif_match,
ACTION_NAMES.lto_backend,
]
all_cpp_compile_actions = [
ACTION_NAMES.cpp_compile,
ACTION_NAMES.linkstamp_compile,
ACTION_NAMES.cpp_header_parsing,
ACTION_NAMES.cpp_module_compile,
ACTION_NAMES.cpp_module_codegen,
ACTION_NAMES.clif_match,
]
# C-only compile actions (for C-specific workarounds)
all_c_compile_actions = [
ACTION_NAMES.c_compile,
]
all_link_actions = [
ACTION_NAMES.cpp_link_executable,
ACTION_NAMES.cpp_link_dynamic_library,
ACTION_NAMES.cpp_link_nodeps_dynamic_library,
]
lto_index_actions = [
ACTION_NAMES.lto_index_for_executable,
ACTION_NAMES.lto_index_for_dynamic_library,
ACTION_NAMES.lto_index_for_nodeps_dynamic_library,
]
def _get_target_triple(cpu):
"""Returns the target triple for the given CPU architecture."""
if cpu == "arm64":
return "arm64-apple-macos"
elif cpu == "x86_64":
return "x86_64-apple-macos"
else:
fail("Unsupported CPU: " + cpu)
def _sanitizer_feature(name = "", specific_compile_flags = [], specific_link_flags = []):
return feature(
name = name,
flag_sets = [
flag_set(
actions = all_compile_actions,
flag_groups = [
flag_group(flags = [
"-fno-omit-frame-pointer",
"-fno-sanitize-recover=all",
] + specific_compile_flags),
],
),
flag_set(
actions = all_link_actions,
flag_groups = [
flag_group(flags = specific_link_flags),
],
),
],
)
def _impl(ctx):
tool_paths = [
tool_path(name = name, path = path)
for name, path in ctx.attr.tool_paths.items()
]
action_configs = []
# Get target triple for cross-compilation
target_triple = _get_target_triple(ctx.attr.cpu)
# All paths are relative to the execroot. The external repo's files live at
# external/<repo_name>/ in the execroot. This prefix is computed by the
# repository rule and passed as an attribute.
prefix = ctx.attr.execroot_prefix
# Paths to key directories within the repo (execroot-relative)
libcxx_include = prefix + "/include/c++/v1"
clang_builtin_include = prefix + "/include/clang"
sdk_include = prefix + "/sysroot/usr/include"
sdk_lib = prefix + "/sysroot/usr/lib"
sdk_frameworks = prefix + "/sysroot/System/Library/Frameworks"
# Build include directories (for cxx_builtin_include_directories)
# These tell Bazel which directories the compiler considers "built-in" so it
# doesn't require explicit deps for headers found there.
sdk_include_dirs = [
prefix,
prefix + "/include/config_override",
libcxx_include,
clang_builtin_include,
# Bundled ICU headers must be listed before SDK to avoid namespace mismatch
"src/third_party/icu4c-57.1/source/common",
"src/third_party/icu4c-57.1/source/i18n",
sdk_include,
sdk_frameworks + "/CoreFoundation.framework/Headers",
sdk_frameworks + "/Security.framework/Headers",
sdk_frameworks + "/IOKit.framework/Headers",
sdk_frameworks + "/SystemConfiguration.framework/Headers",
sdk_frameworks,
]
# Sysroot path (execroot-relative)
sysroot = prefix + "/sysroot"
# Cross-compilation specific compile flags (shared by C and C++)
# Note: -stdlib=libc++ is C++ only (causes -Werror,-Wunused-command-line-argument in C)
# but -nostdinc++ and the include paths must be here to maintain correct include order
# (libc++ headers before clang builtins before SDK headers).
cross_compile_flags = [
"--target=" + target_triple + ctx.attr.min_macos_version,
"-isysroot",
sysroot,
"-mmacosx-version-min=" + ctx.attr.min_macos_version,
"-fPIC",
# Prevent clang from using SDK's libc++ headers; use LLVM's instead
# (LLVM's libc++ has full C++20 support including std::atomic_ref)
"-nostdinc++",
# Override directory with modified __config_site (enables vendor availability
# annotations so newer libc++ symbols like __hash_memory are inlined)
"-isystem",
prefix + "/include/config_override",
# LLVM's libc++ headers (for C++ standard library with full C++20 support)
# These contain wrapper headers like stddef.h that use #include_next
"-isystem",
libcxx_include,
# Clang builtin headers SECOND - defines size_t, ptrdiff_t in global namespace
# libc++ wrappers' #include_next will find these
"-isystem",
clang_builtin_include,
# Bundled ICU 57.1 headers BEFORE SDK headers. The macOS SDK ships its own
# ICU headers at usr/include/unicode/ which use a different namespace (plain
# "icu" vs bundled "icu_57") and unversioned C API symbols. If SDK headers
# are found first, the linker can't resolve them against the bundled library.
"-isystem",
"src/third_party/icu4c-57.1/source/common",
"-isystem",
"src/third_party/icu4c-57.1/source/i18n",
# SDK's C headers LAST (needed for system library)
"-isystem",
sdk_include,
]
# C++ specific compile flags - force include cstdlib for malloc/free
# (needed because fmt sets _LIBCPP_REMOVE_TRANSITIVE_INCLUDES which
# prevents transitive includes from libc++ headers)
cpp_force_includes = ["-include", "cstdlib"]
# Cross-compilation specific link flags
cross_link_flags = [
"--target=" + target_triple + ctx.attr.min_macos_version,
"-isysroot",
sysroot,
"-mmacosx-version-min=" + ctx.attr.min_macos_version,
"-L" + sdk_lib,
"-F" + sdk_frameworks,
"-lc++",
# Use lld linker with macOS flavor. -B adds the tools dir to clang's
# program search path so it finds ld64.lld there (clang resolves its
# own symlink and otherwise only searches next to the real binary).
"-B" + prefix + "/tools",
"-fuse-ld=lld",
"-Wl,-platform_version,macos," + ctx.attr.min_macos_version + ",14.0",
# Force-load all members of static libraries. This is required because
# MongoDB uses MONGO_INITIALIZER registrations via static constructors
# that have no direct symbol references - without this, ld64 drops the
# object files containing them. The GNU ld equivalent is --whole-archive
# but ld64 mode doesn't support positional wrapping; -all_load is global.
"-Wl,-all_load",
]
# Feature definitions
supports_pic_feature = feature(
name = "supports_pic",
enabled = True,
)
supports_start_end_lib_feature = feature(
name = "supports_start_end_lib",
enabled = True,
)
gcc_quoting_for_param_files_feature = feature(
name = "gcc_quoting_for_param_files",
enabled = True,
)
static_link_cpp_runtimes_feature = feature(
name = "static_link_cpp_runtimes",
enabled = False,
)
# Default compile flags feature
default_compile_flags_feature = feature(
name = "default_compile_flags",
enabled = True,
flag_sets = [
flag_set(
actions = all_compile_actions,
flag_groups = [
flag_group(flags = cross_compile_flags + [
"-Wall",
"-Wextra",
"-Werror=return-type",
"-fno-strict-aliasing",
"-fno-omit-frame-pointer",
]),
],
),
flag_set(
actions = all_cpp_compile_actions,
flag_groups = [
flag_group(flags = [
"-stdlib=libc++",
"-std=c++20",
# Fix ICU 57.1 header shadowing: when bundled ICU code includes
# "unicode/localpointer.h", the SDK's newer ICU headers may be
# found first. SDK headers use bare 'noexcept' while bundled ICU
# uses U_NOEXCEPT macro. Define it to ensure compatibility.
"-DU_NOEXCEPT=noexcept",
# Fix ICU 57.1 U_FINAL macro: when MongoDB source files (outside ICU)
# include ICU headers via angle brackets, the SDK's umachine.h may be
# found instead of the bundled one. Ensure U_FINAL expands to 'final'.
"-DU_FINAL=final",
] + cpp_force_includes),
],
),
# C-specific flags: Work around macOS SDK ICU headers using char16_t
# which isn't available in C (macOS lacks the <uchar.h> C11 header).
# The SDK's umachine.h unconditionally uses char16_t for internal ICU
# builds (when U_COMMON_IMPLEMENTATION etc. are defined), so we must
# define char16_t itself rather than just UCHAR_TYPE.
flag_set(
actions = all_c_compile_actions,
flag_groups = [
flag_group(flags = [
"-Dchar16_t=uint16_t",
]),
],
),
],
)
# Default link flags feature
default_link_flags_feature = feature(
name = "default_link_flags",
enabled = True,
flag_sets = [
flag_set(
actions = all_link_actions + lto_index_actions,
flag_groups = [
flag_group(flags = cross_link_flags),
],
),
],
)
# User compile flags (from copts)
user_compile_flags_feature = feature(
name = "user_compile_flags",
enabled = True,
flag_sets = [
flag_set(
actions = all_compile_actions,
flag_groups = [
flag_group(
expand_if_available = "user_compile_flags",
iterate_over = "user_compile_flags",
flags = ["%{user_compile_flags}"],
),
],
),
],
)
# User link flags (from linkopts)
user_link_flags_feature = feature(
name = "user_link_flags",
enabled = True,
flag_sets = [
flag_set(
actions = all_link_actions + lto_index_actions,
flag_groups = [
flag_group(
expand_if_available = "user_link_flags",
iterate_over = "user_link_flags",
flags = ["%{user_link_flags}"],
),
],
),
],
)
# Include paths feature
include_paths_feature = feature(
name = "include_paths",
enabled = True,
flag_sets = [
flag_set(
actions = all_compile_actions,
flag_groups = [
flag_group(
iterate_over = "quote_include_paths",
expand_if_available = "quote_include_paths",
flags = ["-iquote", "%{quote_include_paths}"],
),
flag_group(
iterate_over = "include_paths",
expand_if_available = "include_paths",
flags = ["-I%{include_paths}"],
),
flag_group(
iterate_over = "system_include_paths",
expand_if_available = "system_include_paths",
flags = ["-isystem", "%{system_include_paths}"],
),
],
),
],
)
# External include paths feature - handles dependencies like abseil-cpp
external_include_paths_feature = feature(
name = "external_include_paths",
enabled = True,
flag_sets = [
flag_set(
actions = all_compile_actions,
flag_groups = [
flag_group(
iterate_over = "external_include_paths",
expand_if_available = "external_include_paths",
flags = ["-isystem", "%{external_include_paths}"],
),
],
),
],
)
# Dependency file feature
dependency_file_feature = feature(
name = "dependency_file",
enabled = True,
flag_sets = [
flag_set(
actions = all_compile_actions,
flag_groups = [
flag_group(
expand_if_available = "dependency_file",
flags = ["-MD", "-MF", "%{dependency_file}"],
),
],
),
],
)
# Debug feature
dbg_feature = feature(
name = "dbg",
flag_sets = [
flag_set(
actions = all_compile_actions,
flag_groups = [
flag_group(flags = ["-g", "-O0"]),
],
),
],
implies = ["common"],
)
# Opt feature
opt_feature = feature(
name = "opt",
flag_sets = [
flag_set(
actions = all_compile_actions,
flag_groups = [
flag_group(flags = ["-O2", "-DNDEBUG"]),
],
),
],
implies = ["common"],
)
# Common feature
common_feature = feature(
name = "common",
implies = [
"default_compile_flags",
"default_link_flags",
],
)
# Sysroot feature
sysroot_feature = feature(
name = "sysroot",
enabled = True,
flag_sets = [
flag_set(
actions = all_compile_actions + all_link_actions + lto_index_actions,
flag_groups = [
flag_group(
expand_if_available = "sysroot",
flags = ["--sysroot=%{sysroot}"],
),
],
),
],
)
# Framework paths feature for macOS
framework_paths_feature = feature(
name = "framework_paths",
enabled = True,
flag_sets = [
flag_set(
actions = all_compile_actions + all_link_actions,
flag_groups = [
flag_group(
flags = [
"-F" + sdk_frameworks,
],
),
],
),
],
)
# Sanitizer features
asan_feature = _sanitizer_feature(
name = "asan",
specific_compile_flags = ["-fsanitize=address"],
specific_link_flags = ["-fsanitize=address"],
)
tsan_feature = _sanitizer_feature(
name = "tsan",
specific_compile_flags = ["-fsanitize=thread"],
specific_link_flags = ["-fsanitize=thread"],
)
ubsan_feature = _sanitizer_feature(
name = "ubsan",
specific_compile_flags = ["-fsanitize=undefined"],
specific_link_flags = ["-fsanitize=undefined"],
)
# Archiver flags feature
archiver_flags_feature = feature(
name = "archiver_flags",
enabled = True,
flag_sets = [
flag_set(
actions = [ACTION_NAMES.cpp_link_static_library],
flag_groups = [
flag_group(
expand_if_available = "output_execpath",
flags = ["rcsD", "%{output_execpath}"],
),
flag_group(
expand_if_available = "libraries_to_link",
iterate_over = "libraries_to_link",
flag_groups = [
flag_group(
expand_if_equal = variable_with_value(
name = "libraries_to_link.type",
value = "object_file",
),
flags = ["%{libraries_to_link.name}"],
),
flag_group(
expand_if_equal = variable_with_value(
name = "libraries_to_link.type",
value = "object_file_group",
),
iterate_over = "libraries_to_link.object_files",
flags = ["%{libraries_to_link.object_files}"],
),
],
),
],
),
],
)
# Output execpath feature for linking
output_execpath_flags_feature = feature(
name = "output_execpath_flags",
enabled = True,
flag_sets = [
flag_set(
actions = all_link_actions + lto_index_actions,
flag_groups = [
flag_group(
expand_if_available = "output_execpath",
flags = ["-o", "%{output_execpath}"],
),
],
),
],
)
# Library search directories feature
library_search_directories_feature = feature(
name = "library_search_directories",
enabled = True,
flag_sets = [
flag_set(
actions = all_link_actions + lto_index_actions,
flag_groups = [
flag_group(
expand_if_available = "library_search_directories",
iterate_over = "library_search_directories",
flags = ["-L%{library_search_directories}"],
),
],
),
],
)
# Libraries to link feature
libraries_to_link_feature = feature(
name = "libraries_to_link",
enabled = True,
flag_sets = [
flag_set(
actions = all_link_actions + lto_index_actions,
flag_groups = [
flag_group(
expand_if_available = "libraries_to_link",
iterate_over = "libraries_to_link",
flag_groups = [
flag_group(
expand_if_equal = variable_with_value(
name = "libraries_to_link.type",
value = "object_file_group",
),
iterate_over = "libraries_to_link.object_files",
flags = ["%{libraries_to_link.object_files}"],
),
flag_group(
expand_if_equal = variable_with_value(
name = "libraries_to_link.type",
value = "object_file",
),
flags = ["%{libraries_to_link.name}"],
),
flag_group(
expand_if_equal = variable_with_value(
name = "libraries_to_link.type",
value = "interface_library",
),
flags = ["%{libraries_to_link.name}"],
),
flag_group(
expand_if_equal = variable_with_value(
name = "libraries_to_link.type",
value = "static_library",
),
flags = ["%{libraries_to_link.name}"],
),
flag_group(
expand_if_equal = variable_with_value(
name = "libraries_to_link.type",
value = "dynamic_library",
),
flags = ["-l%{libraries_to_link.name}"],
),
flag_group(
expand_if_equal = variable_with_value(
name = "libraries_to_link.type",
value = "versioned_dynamic_library",
),
flags = ["-l:%{libraries_to_link.name}"],
),
],
),
],
),
],
)
# Collect all features
features = [
supports_pic_feature,
gcc_quoting_for_param_files_feature,
static_link_cpp_runtimes_feature,
common_feature,
default_compile_flags_feature,
default_link_flags_feature,
user_compile_flags_feature,
user_link_flags_feature,
include_paths_feature,
external_include_paths_feature,
dependency_file_feature,
dbg_feature,
opt_feature,
sysroot_feature,
framework_paths_feature,
asan_feature,
tsan_feature,
ubsan_feature,
archiver_flags_feature,
output_execpath_flags_feature,
library_search_directories_feature,
libraries_to_link_feature,
# Apply MONGO_GLOBAL_DEFINES (enterprise defines, sanitizer defines, etc.)
# as -D flags. This is a subset of get_common_features() - we only need the
# defines, not the warnings_as_errors or optimization features which would
# conflict with the cross-compilation toolchain's own settings.
feature(
name = "mongo_defines",
enabled = True,
flag_sets = [
flag_set(
actions = all_compile_actions,
flag_groups = [flag_group(
flags = ["-D" + define for define in ctx.attr.global_defines],
)],
),
],
),
] + ([supports_start_end_lib_feature] if ctx.attr.supports_start_end_lib else [])
# Artifact name patterns for macOS
artifact_name_patterns = [
artifact_name_pattern(
category_name = "executable",
prefix = "",
extension = "",
),
artifact_name_pattern(
category_name = "dynamic_library",
prefix = "lib",
extension = ".dylib",
),
artifact_name_pattern(
category_name = "static_library",
prefix = "lib",
extension = ".a",
),
]
return cc_common.create_cc_toolchain_config_info(
ctx = ctx,
features = features,
action_configs = action_configs,
artifact_name_patterns = artifact_name_patterns,
cxx_builtin_include_directories = sdk_include_dirs,
toolchain_identifier = ctx.attr.toolchain_identifier,
host_system_name = "x86_64-unknown-linux-gnu",
target_system_name = target_triple,
target_cpu = ctx.attr.cpu,
target_libc = ctx.attr.target_libc,
compiler = ctx.attr.compiler,
abi_version = ctx.attr.abi_version,
abi_libc_version = ctx.attr.abi_libc_version,
tool_paths = tool_paths,
builtin_sysroot = sysroot,
)
mongo_apple_cross_cc_toolchain_config = rule(
implementation = _impl,
attrs = {
"abi_libc_version": attr.string(mandatory = True),
"abi_version": attr.string(mandatory = True),
"compiler": attr.string(mandatory = True),
"cpu": attr.string(mandatory = True),
"llvm_version": attr.string(mandatory = True),
"min_macos_version": attr.string(mandatory = True),
"execroot_prefix": attr.string(mandatory = True),
"target_libc": attr.string(mandatory = True),
"tool_paths": attr.string_dict(mandatory = True),
"toolchain_identifier": attr.string(mandatory = True),
"supports_start_end_lib": attr.bool(default = False),
"optimization_level": attr.string(mandatory = False),
"debug_level": attr.int(mandatory = False),
"internal_thin_lto_enabled": attr.bool(default = False, mandatory = False),
"coverage_enabled": attr.bool(default = False, mandatory = False),
"compress_debug_enabled": attr.bool(default = False, mandatory = False),
"warnings_as_errors_enabled": attr.bool(default = True, mandatory = False),
"linkstatic": attr.bool(mandatory = True),
"global_defines": attr.string_list(default = []),
},
provides = [CcToolchainConfigInfo],
)

308
bazel/toolchains/cc/mongo_apple_cross/mongo_apple_cross_toolchain.bzl Normal file
View File

@ -0,0 +1,308 @@
"""Repository rules for MongoDB's macOS cross-compilation toolchain.
Two repos are created:
1. `mongo_apple_cross_toolchain` — lightweight, always fetched. Contains only
`toolchain()` registration targets with platform constraints. No downloads.
2. `mongo_apple_cross_toolchain_files` — lazily fetched (only when Bazel selects
the macOS cross-compilation toolchain). Downloads LLVM + macOS SDK from S3
(or uses local overrides via LLVM_PATH / MACOS_SDK_PATH env vars), then
generates cc_toolchain / cc_toolchain_config targets.
This split ensures that normal Linux builds never pay the download cost.
"""
load("@build_bazel_apple_support//configs:platforms.bzl", "APPLE_PLATFORMS_CONSTRAINTS")
# S3 bucket and paths for pre-built toolchain archives
_S3_BUCKET = "https://mdb-build-public.s3.amazonaws.com"
_S3_PREFIX = "macos_cross_compile"
# LLVM toolchain archive (stripped to only what cross-compilation needs)
_LLVM_URL = _S3_BUCKET + "/" + _S3_PREFIX + "/llvm-22.1.1-cross-linux-arm64.tar.xz"
_LLVM_SHA256 = "172159dbf9db36f6a45e0836126213a354be9d784b3717faeec27b9e5e905af9"
_LLVM_STRIP_PREFIX = "LLVM-22.1.1-Linux-ARM64"
_LLVM_VERSION = "22"
# macOS SDK archive
_MACOS_SDK_URL = _S3_BUCKET + "/" + _S3_PREFIX + "/macos-sdk-15.2.tar.xz"
_MACOS_SDK_SHA256 = "b090a2bd6b0566616da8bdb9a88ab84e842fd3f44ff4be6a3d795a599d462a0e"
_MACOS_SDK_STRIP_PREFIX = "MacOSX15.2.sdk"
# ---------------------------------------------------------------------------
# Shared constants
# ---------------------------------------------------------------------------
# Map of supported architectures for cross-compilation
_CROSS_COMPILE_ARCHS = {
"darwin_arm64": "arm64",
"darwin_x86_64": "x86_64",
}
def get_supported_apple_cross_archs():
"""Returns a map of supported Apple architectures for cross-compilation."""
return _CROSS_COMPILE_ARCHS
# ---------------------------------------------------------------------------
# Repo 1: lightweight toolchain registration (always fetched, no downloads)
# ---------------------------------------------------------------------------
def _apple_cross_toolchain_registration_impl(repository_ctx):
"""Generates toolchain() targets that point to the files repo."""
if "linux" not in repository_ctx.os.name.lower():
repository_ctx.file(
"BUILD.bazel",
"# Apple cross-compilation toolchain is only available on Linux\n",
)
return
entries = [
'package(default_visibility = ["//visibility:public"])\n',
]
for arch, cpu in _CROSS_COMPILE_ARCHS.items():
entries.append("""
toolchain(
name = "mongo_apple_cross_{arch}_toolchain",
exec_compatible_with = [
"@platforms//os:linux",
],
target_compatible_with = [
"@platforms//os:macos",
"@platforms//cpu:{cpu}",
],
toolchain = "@mongo_apple_cross_toolchain_files//:cc-compiler-apple-cross-{arch}",
toolchain_type = "@bazel_tools//tools/cpp:toolchain_type",
)
""".format(arch = arch, cpu = cpu))
repository_ctx.file("BUILD.bazel", "\n".join(entries))
mongo_apple_cross_toolchain_config = repository_rule(
implementation = _apple_cross_toolchain_registration_impl,
configure = True,
)
# ---------------------------------------------------------------------------
# Repo 2: actual toolchain files (lazily fetched — only on cross-compile)
# ---------------------------------------------------------------------------
def _download_llvm(repository_ctx):
"""Downloads and extracts the LLVM toolchain, returning its path."""
repository_ctx.report_progress("Downloading LLVM cross-compilation toolchain")
repository_ctx.download_and_extract(
url = _LLVM_URL,
sha256 = _LLVM_SHA256,
output = "llvm",
stripPrefix = _LLVM_STRIP_PREFIX,
)
return str(repository_ctx.path("llvm"))
def _download_macos_sdk(repository_ctx):
"""Downloads and extracts the macOS SDK, returning its path."""
repository_ctx.report_progress("Downloading macOS SDK")
repository_ctx.download_and_extract(
url = _MACOS_SDK_URL,
sha256 = _MACOS_SDK_SHA256,
output = "sdk",
stripPrefix = _MACOS_SDK_STRIP_PREFIX,
)
return str(repository_ctx.path("sdk"))
def _get_llvm_path(repository_ctx):
"""Gets the LLVM path: uses LLVM_PATH env var if set, otherwise downloads from S3."""
llvm_path = repository_ctx.os.environ.get("LLVM_PATH", "")
if llvm_path:
return llvm_path, None, False
# Download from S3 (statically-linked LLVM with all needed tools)
path = _download_llvm(repository_ctx)
return path, None, True
def _get_apple_sdk_path(repository_ctx):
"""Gets the Apple SDK path: uses MACOS_SDK_PATH env var if set, otherwise downloads from S3."""
sdk_path = repository_ctx.os.environ.get("MACOS_SDK_PATH", "")
if sdk_path:
return sdk_path, None, False
sdk_path = repository_ctx.os.environ.get("SDKROOT", "")
if sdk_path:
return sdk_path, None, False
# Download from S3 as default
path = _download_macos_sdk(repository_ctx)
return path, None, True
def _get_llvm_version(repository_ctx, llvm_path, downloaded):
"""Gets the LLVM version number."""
if downloaded:
return _LLVM_VERSION
result = repository_ctx.execute([llvm_path + "/bin/clang", "--version"])
if result.return_code == 0:
for line in result.stdout.split("\n"):
if "version" in line.lower():
parts = line.split(" ")
for i, part in enumerate(parts):
if part.lower() == "version" and i + 1 < len(parts):
version_str = parts[i + 1]
return version_str.split(".")[0]
return "14" # Default fallback
# LLVM tool binaries to symlink into the repo for hermetic builds
_LLVM_TOOLS = [
"clang",
"clang++",
"clang-cpp",
"ld.lld",
"ld64.lld",
"lld",
"llvm-ar",
"llvm-nm",
"llvm-objcopy",
"llvm-objdump",
"llvm-strip",
"llvm-dwp",
"llvm-profdata",
"llvm-cov",
]
def _symlink_toolchain_files(repository_ctx, llvm_path, llvm_version, sdk_path):
"""Symlinks LLVM tools, headers, and SDK into the repo for hermetic builds."""
# Symlink LLVM tool binaries
for tool in _LLVM_TOOLS:
src = llvm_path + "/bin/" + tool
if repository_ctx.execute(["test", "-e", src]).return_code == 0:
repository_ctx.symlink(src, "tools/" + tool)
# Ensure ld64.lld exists — clang looks for it (not ld.lld) when targeting
# macOS with -fuse-ld=lld. LLD is a multi-format linker; it picks the
# Mach-O backend automatically when invoked as "ld64.lld".
ld64_lld = repository_ctx.path("tools/ld64.lld")
if not ld64_lld.exists:
lld = repository_ctx.path("tools/lld")
if lld.exists:
repository_ctx.symlink(lld, "tools/ld64.lld")
# Symlink clang as a versioned name too (some internal lookups need it)
repository_ctx.symlink(llvm_path + "/bin/clang", "tools/clang-" + llvm_version)
# Symlink LLVM libc++ headers (needed for C++ standard library).
repository_ctx.symlink(llvm_path + "/include/c++/v1", "include/c++/v1")
# Override __config_site to enable vendor availability annotations.
config_site_content = repository_ctx.read(llvm_path + "/include/c++/v1/__config_site")
config_site_content = config_site_content.replace(
"#define _LIBCPP_HAS_VENDOR_AVAILABILITY_ANNOTATIONS 0",
"#define _LIBCPP_HAS_VENDOR_AVAILABILITY_ANNOTATIONS 1",
)
repository_ctx.file("include/config_override/__config_site", config_site_content)
# Symlink Clang builtin headers (needed for stddef.h, stdarg.h, etc.)
repository_ctx.symlink(
llvm_path + "/lib/clang/" + llvm_version + "/include",
"include/clang",
)
# Symlink macOS SDK directories
repository_ctx.symlink(sdk_path + "/usr/include", "sysroot/usr/include")
repository_ctx.symlink(sdk_path + "/usr/lib", "sysroot/usr/lib")
repository_ctx.symlink(sdk_path + "/System", "sysroot/System")
def _configure_cross_toolchain(repository_ctx):
"""Configures the cross-compilation toolchain files."""
build_file = "BUILD.bazel"
# Get LLVM path (downloads from S3 if no local override)
llvm_path, error, llvm_downloaded = _get_llvm_path(repository_ctx)
if error:
return False, error
# Get Apple SDK path (downloads from S3 if no local override)
sdk_path, error, sdk_downloaded = _get_apple_sdk_path(repository_ctx)
if error:
return False, error
# Get LLVM version
llvm_version = _get_llvm_version(repository_ctx, llvm_path, llvm_downloaded)
# Get minimum macOS version
min_macos_version = repository_ctx.os.environ.get("MACOS_MIN_VERSION", "14.0")
# Validate minimum macOS version is at least 14
min_version_major = int(min_macos_version.split(".")[0])
if min_version_major < 14:
fail("MACOS_MIN_VERSION must be at least 14.0, got: " + min_macos_version)
# Symlink LLVM tools, headers, and SDK into the repo for hermetic builds.
repository_ctx.report_progress("Symlinking LLVM tools and macOS SDK for hermetic toolchain")
_symlink_toolchain_files(repository_ctx, llvm_path, llvm_version, sdk_path)
# Compute the execroot-relative path prefix for this external repository.
repo_name = repository_ctx.name
execroot_prefix = "external/" + repo_name
repository_ctx.report_progress("Generating macOS cross-compilation toolchain build file")
build_template = Label("@//bazel/toolchains/cc/mongo_apple_cross:BUILD.tmpl")
repository_ctx.template(
build_file,
build_template,
{
"%{llvm_version}": llvm_version,
"%{min_macos_version}": min_macos_version,
"%{execroot_prefix}": execroot_prefix,
},
)
return True, ""
def _apple_cross_toolchain_files_impl(repository_ctx):
"""Downloads and configures the actual toolchain files."""
if "linux" not in repository_ctx.os.name.lower():
repository_ctx.file(
"BUILD.bazel",
"# Apple cross-compilation toolchain is only available on Linux\n",
)
return
success, error_msg = _configure_cross_toolchain(repository_ctx)
if not success:
repository_ctx.file(
"BUILD.bazel",
"""# Apple cross-compilation toolchain configuration failed:
# {}
# To enable macOS cross-compilation, either:
# 1. Just run: bazel build --config=macos-cross-arm64 (downloads LLVM + SDK from S3)
# 2. Or override with local paths: set LLVM_PATH and/or MACOS_SDK_PATH env vars
""".format(error_msg.replace("\n", "\n# ")),
)
mongo_apple_cross_toolchain_files_config = repository_rule(
environ = [
"LLVM_PATH",
"MACOS_SDK_PATH",
"SDKROOT",
"MACOS_MIN_VERSION",
],
implementation = _apple_cross_toolchain_files_impl,
configure = True,
)
# ---------------------------------------------------------------------------
# Module extension: sets up both repos
# ---------------------------------------------------------------------------
def setup_mongo_apple_cross_toolchain():
"""Sets up the macOS cross-compilation toolchain (both repos)."""
mongo_apple_cross_toolchain_config(
name = "mongo_apple_cross_toolchain",
)
mongo_apple_cross_toolchain_files_config(
name = "mongo_apple_cross_toolchain_files",
)
setup_mongo_apple_cross_toolchain_extension = module_extension(
implementation = lambda ctx: setup_mongo_apple_cross_toolchain(),
)

44
bazel/toolchains/cc/mongo_linux/mongo_toolchain.BUILD.tmpl
View File

@ -12,7 +12,8 @@ load("@mongo_toolchain_{version}//:mongo_toolchain_flags.bzl",
load(
"@//bazel/toolchains/cc:mongo_custom_features.bzl",
"FEATURES_ATTR_NAMES",
"get_common_features_attrs")
"get_common_features_attrs",
)
load(
"@//bazel/toolchains/cc:mongo_errors.bzl",
"REQUIRED_SETTINGS_SANITIZER_ERROR_MESSAGE",
@ -20,6 +21,15 @@ load(
"THREAD_SANITIZER_ERROR_MESSAGE",
)
load("@//bazel/toolchains/cc:mongo_defines.bzl", "MONGO_GLOBAL_DEFINES")
load(
"@//bazel/toolchains/cc:mongo_toolchain_selects.bzl",
"COMPRESS_DEBUG_ENABLED",
"COVERAGE_ENABLED",
"DEBUG_LEVEL",
"INTERNAL_THIN_LTO_ENABLED",
"LINKSTATIC_ENABLED",
"WARNINGS_AS_ERRORS_ENABLED",
)
package(default_visibility = ["//visibility:public"])
@ -64,11 +74,6 @@ CHOSEN_LINKER = select(
}
)
LINKSTATIC_ENABLED = select({
"@//bazel/config:linkstatic_enabled": True,
"@//conditions:default": False,
})
SHARED_ARCHIVE_ENABLED = select({
"@//bazel/config:shared_archive_enabled": True,
"@//conditions:default": False,
@ -85,13 +90,6 @@ FISSION_ENABLED = select({
"@//conditions:default": False,
})
DEBUG_LEVEL = select({
"@//bazel/config:gcc_or_clang_dbg_level_0": 0,
"@//bazel/config:gcc_or_clang_dbg_level_1": 1,
"@//bazel/config:gcc_or_clang_dbg_level_2": 2,
"@//bazel/config:gcc_or_clang_dbg_level_3": 3,
})
DISABLE_DEBUG_SYMBOLS = select({
"@//bazel/config:debug_symbols_disabled": True,
"@//conditions:default": False,
@ -189,26 +187,6 @@ IS_X86_64 = select({
"@//conditions:default": False,
})
INTERNAL_THIN_LTO_ENABLED = select({
"@//bazel/config:thin_lto_enabled": True,
"@//conditions:default": False,
})
COVERAGE_ENABLED = select({
"@//bazel/config:gcov_enabled": True,
"@//conditions:default": False,
})
COMPRESS_DEBUG_ENABLED = select({
"@//bazel/config:compress_debug_compile_enabled": True,
"@//conditions:default": False,
})
WARNINGS_AS_ERRORS_ENABLED = select({
"@//bazel/config:warnings_as_errors_enabled": True,
"@//conditions:default": False,
})
IS_DEBUG_BUILD = select({
"@//bazel/config:dbg_enabled": True,
"@//conditions:default": False,

33
bazel/toolchains/cc/mongo_toolchain_selects.bzl Normal file
View File

@ -0,0 +1,33 @@
"""Common select() definitions shared across toolchain BUILD templates."""
DEBUG_LEVEL = select({
"@//bazel/config:gcc_or_clang_dbg_level_0": 0,
"@//bazel/config:gcc_or_clang_dbg_level_1": 1,
"@//bazel/config:gcc_or_clang_dbg_level_2": 2,
"@//bazel/config:gcc_or_clang_dbg_level_3": 3,
})
INTERNAL_THIN_LTO_ENABLED = select({
"@//bazel/config:thin_lto_enabled": True,
"@//conditions:default": False,
})
COVERAGE_ENABLED = select({
"@//bazel/config:gcov_enabled": True,
"@//conditions:default": False,
})
COMPRESS_DEBUG_ENABLED = select({
"@//bazel/config:compress_debug_compile_enabled": True,
"@//conditions:default": False,
})
WARNINGS_AS_ERRORS_ENABLED = select({
"@//bazel/config:warnings_as_errors_enabled": True,
"@//conditions:default": False,
})
LINKSTATIC_ENABLED = select({
"@//bazel/config:linkstatic_enabled": True,
"@//conditions:default": False,
})

9
bazel/toolchains/cc/mongo_windows/mongo_toolchain.BUILD.tmpl
View File

@ -1,16 +1,15 @@
load("@//bazel/toolchains/cc/mongo_windows:mongo_windows_cc_toolchain_config.bzl", "mongo_windows_cc_toolchain_config")
load("@//bazel/toolchains/cc:mongo_defines.bzl", "MONGO_GLOBAL_DEFINES")
load(
"@//bazel/toolchains/cc:mongo_toolchain_selects.bzl",
"WARNINGS_AS_ERRORS_ENABLED",
)
SMALLER_BINARY = select({
"@//bazel/config:msvc_opt": True,
"//conditions:default": False,
})
WARNINGS_AS_ERRORS_ENABLED = select({
"@//bazel/config:warnings_as_errors_enabled": True,
"@//conditions:default": False,
})
OPT_DEBUG = select({
"@//bazel/config:windows_opt_debug": True,
"//conditions:default": False,

5
bazel/toolchains/python/python_toolchain.BUILD.tmpl
View File

@ -35,7 +35,10 @@ py_runtime_pair(
toolchain(
name = "python_toolchain",
exec_compatible_with = [{constraints}],
target_compatible_with = [{constraints}],
# NOTE: Do NOT set target_compatible_with for Python toolchains.
# Python is used for exec (build-time) actions only, not for producing
# target artifacts. Setting target_compatible_with breaks cross-compilation
# because Bazel would try to find a Python that runs on the target platform.
toolchain = ":py_runtime_pair",
toolchain_type = "@rules_python//python:toolchain_type",
)

20
etc/evergreen_yml_components/tasks/compile_tasks.yml
View File

@ -526,6 +526,21 @@ tasks:
--linkopt=-s
task_compile_flags: ${compile_all_but_not_unittests_flags|--linkopt=-s}
- name: compile_devcore
depends_on:
- name: version_expansions_gen
variant: generate-tasks-for-version
tags: ["assigned_to_jira_team_devprod_build", "auxiliary"]
commands:
- func: "bazel compile"
vars:
targets: install-devcore # Compile main targets most likely to fail
compiling_for_test: true
# The executables generated by the install-devcore target are not used after successfully being
# compiled. We therefore link them without debug symbols to compile them faster.
bazel_args: >-
--dbg=False
- name: compile_libraries_for_unittests
tags: ["assigned_to_jira_team_devprod_build", "auxiliary"]
depends_on:
@ -1624,6 +1639,11 @@ task_groups:
- archive_dist_test_debug
- run_pretty_printer_tests
- <<: *compile_task_group_template
name: compile_devcore_TG
tasks:
- compile_devcore
- <<: *compile_task_group_template
name: compile_all_but_not_unittests_TG
tasks:

16
etc/evergreen_yml_components/variants/macos/test_dev_master_branch_only.yml
View File

@ -30,3 +30,19 @@ buildvariants:
- name: sasl
- name: .crypt
- name: run_bazel_TG
# Build variant that uses cross compilation to prove out that compilation succeeds on MacOS without having to
# run on more expensive MacOS infrastructure.
- name: &enterprise-macos-arm64-cross-compile enterprise-macos-arm64-cross-compile
display_name: "Enterprise MacOS ARM64 Cross Compile"
tags: ["bazel_check"]
cron: "0 1 * * *" # From the ${project_nightly_cron} parameter, upgrade to required after stability is proven out.
run_on:
- ubuntu2204-arm64-m8g-4xlarge
expansions:
compile_variant: *enterprise-macos-arm64-cross-compile
evergreen_remote_exec: on
bazel_compile_flags: >-
--config=macos-cross-arm64
tasks:
- name: compile_devcore_TG

17
src/third_party/icu4c-57.1/source/BUILD.bazel vendored
View File

@ -17,7 +17,22 @@ ICU4C_DEFINES = [
"U_USING_ICU_NAMESPACE=0",
]
ICU4C_COPTS = select({
# On macOS (native or cross-compiled), the SDK includes system ICU headers
# under /usr/include/unicode/ that can shadow our bundled ICU 57.1 headers.
# We add explicit -I flags to ensure the bundled headers are found first.
# The -iquote flag ensures quoted includes like "unicode/localpointer.h"
# search our bundled directory before the system include paths.
ICU4C_MACOS_INCLUDE_COPTS = select({
"@platforms//os:macos": [
"-iquote",
"src/third_party/icu4c-57.1/source/common",
"-iquote",
"src/third_party/icu4c-57.1/source/i18n",
],
"//conditions:default": [],
})
ICU4C_COPTS = ICU4C_MACOS_INCLUDE_COPTS + select({
"//bazel/config:gcc_or_clang": [
# TODO: Omit this flag when a resolution for cxxopts vs copts issue is achieved.
"-w",