Compare commits

...

355 Commits
master ... v8.2

Author SHA1 Message Date
Cole Harbeck
bace90bea7 SERVER-103650: Add FTDC metrics for reconnect attempts in DBClientSession (#51054)
GitOrigin-RevId: 8fec0f1ac9d74c783c0645a0254f37607cf9ac09
2026-04-06 17:22:57 +00:00
Colin Stolley
ebc29a9614 SERVER-116272: backport schema bypass changes (#50645)
GitOrigin-RevId: 5bc720ead6732c7854f683e8814a6e66e9234012
2026-04-06 17:22:41 +00:00
Indy Prentice
24db206944 SERVER-94001: Continue with processing inactive sessions even if logi… (#48838)
GitOrigin-RevId: 779e7e14d2a71d7190b481d1c39f080d6f8a92d6
2026-04-01 16:47:22 +00:00
Andrew Bradshaw
a64c687f8d SERVER-108845 Fix failing rapidyaml (#50783)
GitOrigin-RevId: d4da2479adedbd9c8a7d80f55b209156200f5d32
2026-04-01 16:47:05 +00:00
Sam Frank
5cc1d48094 SERVER-111411: Fix resharding with forceDistribution on time series collections (#50466)
Co-authored-by: Abdul Qadeer <abdul.qadeer@mongodb.com>
GitOrigin-RevId: 1a2c62dc3bbf411f6c1b50b95fda1b332a49b9bd
2026-04-01 16:46:48 +00:00
Denis Trailin
5f7e594d07 SERVER-119766: fix race in OplogFetcher metric (#50492)
GitOrigin-RevId: 65db3184f703b0b208886b350849504fbdea15ab
2026-04-01 16:46:30 +00:00
Malik Endsley
49a07bb144 SERVER-119749: Change spill store isolation level to readCommitted [8.2] (#49610)
GitOrigin-RevId: 80a6d4b2c8c4774eb630519ebdfea6f7e827865f
2026-03-26 20:02:45 +00:00
Spencer Jackson
c382e41dde SERVER-119317 Improve object lifecycle of MD5 hash state (#49998)
GitOrigin-RevId: 0533d15b96bbd6e84b355b039ed542f271ae49f4
2026-03-19 22:57:41 +00:00
fotiniAlvanaki
11931d956f SERVER-119319 fix ExprCtx use-after-free for $lookup/$graphLookup clone (#49956)
Co-authored-by: Zixuan <leozzx@users.noreply.github.com>
GitOrigin-RevId: 61bea16b08ea8b5ff20ba7282dc6d8ab5bdcfb3b
2026-03-19 22:57:24 +00:00
Daniel Segel
36eb45c51b SERVER-121851 Handle empty pipelines on views in $rankFusion and $scoreFusion (#49909)
GitOrigin-RevId: 0f13b61d08f4e8206752b1ed0fe6d5cdec4900fd
2026-03-19 22:57:06 +00:00
Thomas Goyne
8eafdcf18d SERVER-119964 Only stop oplog manager thread when the current Oplog is destroyed (#49733)
GitOrigin-RevId: d1a18a1c66493eff272cd1407faf14a39dfa5cd1
2026-03-19 22:56:49 +00:00
wt-vendoring-bot[bot]
8fe8ccc913 Import wiredtiger: dab483768edbfd660e54a8f753b664b7f7c86bdc from branch mongodb-8.2 (#49637)
Co-authored-by: wt-vendoring-bot <wt-vendoring-bot@mongodb.com>
GitOrigin-RevId: ebc797b78f800667a81ee491425c9206d062a0b6
2026-03-19 22:56:30 +00:00
Pierre Turin
d77c2c1b4a SERVER-116284 In mongos, ensure we send commits to all shards on error (#49370)
GitOrigin-RevId: 732b3ed596b52b6639fbeccccca9a5750b5ed80b
2026-03-13 20:34:54 +00:00
Marcos Grillo
a803a0599c SERVER-85346 Prevent unique indexes with non simple collation in sharded collections (#48769) (#49245)
Co-authored-by: Meryama <meryama.nadim@mongodb.com>
GitOrigin-RevId: 1de747462945a4f52ba76addc8526e8cdaf240e9
2026-03-13 20:34:37 +00:00
Matt Olma
65bb179286 SERVER-94613: Find on collections with a multikey wildcard index handles metadata inconsistently (#49200)
GitOrigin-RevId: dca55ded3b35b8fe29b3285d295582529652c274
2026-03-13 20:34:20 +00:00
seanzimm
ad71b7376c SERVER-119413: Change configTerm from int to long long in idl and cod… (#48912)
GitOrigin-RevId: e23c0c72eef88abeebf1567048e8d01a3b39f8ec
2026-03-13 20:34:03 +00:00
Max Verbinnen
2b1ff713b8 SERVER-111928 Remove wait for suspendRangeDeletion in agg_orphan_filtering_renamed_shardkey.js (#48815)
GitOrigin-RevId: 1b2a44f2dc5b4b6368a1a32de7421b45cfdfcde6
2026-03-13 20:33:46 +00:00
wt-vendoring-bot[bot]
6ce4e14936 Import wiredtiger: ed5be22a19ee9e3cc5a8e3df1f4370782fd1b33b from branch mongodb-8.2 (#49229)
Co-authored-by: wt-vendoring-bot <wt-vendoring-bot@mongodb.com>
GitOrigin-RevId: c82baacaf99a1e92d9fb7f1415073ec47a61be8c
2026-03-09 15:29:49 +00:00
Abdul Qadeer
359b1260bd SERVER-103990 Skip range deletions when transitioning from embedded to dedicated config servers (#49138)
GitOrigin-RevId: 41cac11aa233fca81dd88a90659f006af57423a7
2026-03-09 15:29:32 +00:00
Joshua Siegel
cc819502e7 SERVER-113623 Add uassert when GRPC is used on unsupported build (#48837)
GitOrigin-RevId: 99e34f06e98579378abff83d86e6fbd2ad7bfbc9
2026-03-09 15:29:15 +00:00
Pierre Turin
87e057b66a SERVER-112273 Prevent repl internal operations from being killed (#48512)
Co-authored-by: Billy Donahue <BillyDonahue@users.noreply.github.com>
GitOrigin-RevId: 94a388fe3889b2f3a036dcf63a8aaab82fded20a
2026-03-09 15:28:58 +00:00
Moustafa Maher
1613813459 SERVER-113530 Address fragility in the transaction coordinator (#44211) (#48925)
GitOrigin-RevId: aac96b3a0a1cd6d2588dc070823eeadd3e001d08
2026-03-09 15:28:41 +00:00
Daniel Moody
590a3ee8e5 SERVER-119813 sync copybara config file (#48677)
GitOrigin-RevId: fc6c4eac0a8d930c3625827f3f1d378f52750897
2026-03-09 15:28:23 +00:00
Moustafa Maher
58a1f4e2c7 SERVER-118514 DelayableTimeoutCallback ignores earlier timeout values… (#48924)
GitOrigin-RevId: 056bc4e207314fe74288bcd42c71915c1ca5171c
2026-03-09 15:28:06 +00:00
Sulabh Mahajan
f90fbdb88e SERVER-120241 Close the connections before returning if opened by the ff util (#48599) (#48655)
GitOrigin-RevId: 6b29e6a7b44210733ac5a7c0060b90420fb49d83
2026-03-09 15:27:49 +00:00
wt-vendoring-bot[bot]
bd287221f6 Import wiredtiger: cb0e9e75ed5555fd3c520e10ebb2897050f5f06d from branch mongodb-8.2 (#48803)
Co-authored-by: wt-vendoring-bot <wt-vendoring-bot@mongodb.com>
GitOrigin-RevId: 32e6fc79c9c12bcbf9131b459cd0847b016046f7
2026-03-09 15:27:32 +00:00
Gabriel Marks
0b13f3b9d2 SERVER-119679 Add auth check for updating mechanisms of user (#48178) (#48534)
GitOrigin-RevId: 3e15fa285a88b114c8e4247a1979cc9b9aabe1eb
2026-03-09 15:27:14 +00:00
tarunsethi
e6120da285 SERVER-114165 Run Coverity scans as periodic builds instead of cron (#47289)
GitOrigin-RevId: 656cc031f1a5af12bc0c2f0b787f36d2f0aee0fa
2026-03-09 15:26:57 +00:00
Cole Harbeck
a5b88171e8 SERVER-115585 Introduce new parameter to stop crashing FTDC on BSONObjectTooLarge exceptions being thrown (#48196)
GitOrigin-RevId: 4e4f05541a1fa367b83f7552f7ac8567a8045163
2026-03-09 15:26:40 +00:00
Indy Prentice
413475e158 SERVER-115307: Always log optime when replication fails to apply an operation (#48086)
GitOrigin-RevId: 9f375ff8da4a702da3c5e6c734c853cef5b59f43
2026-03-09 15:26:24 +00:00
Erwin Pe
1e0e62bb30 SERVER-118428 Fix mongocryptd rejecting large messages (#48210)
GitOrigin-RevId: b698cc53e7b89c79e443282b5248e6f92187018f
2026-02-18 19:36:15 +00:00
Cole Harbeck
1de5943c67 SERVER-115584 detect huge procfs files during FTDC collection (#47923)
Co-authored-by: Billy Donahue <BillyDonahue@users.noreply.github.com>
GitOrigin-RevId: 43dc25b39b217c41e6c3cc0ab37e69f60134db3f
2026-02-18 19:35:56 +00:00
Alex Sarkesian
dcc27418ff SERVER-116327 & SERVER-118546: Correctly validate internal index types (#48204)
Co-authored-by: Benjamin Pearce <benjamin.pearce@mongodb.com>
GitOrigin-RevId: 2ae32bc103866f64de1cd10f3fa3c95e0f8c7063
2026-02-18 19:35:39 +00:00
Saman Memaripour
0b4a0e62c6 SERVER-119396 Adjust maximum buffer size for ingress network messages (#48035)
GitOrigin-RevId: 137a549e064d1c7c3412124f6389e72665536232
2026-02-18 19:35:21 +00:00
Benjamin Pearce
d4b6bdff20 SERVER-118711 Revert SERVER-113888 SERVER-113889 (#48090)
GitOrigin-RevId: ea633db31f853ce5f1f207d47fa150d0f4ca706d
2026-02-18 19:35:03 +00:00
Zac Codiamat
dcc1dbb423 SERVER-119689 Pin docker systemctl replacement to a specific commit (#48084)
GitOrigin-RevId: f0590710e1d98732dcc573425d7af3ce7d7691e4
2026-02-18 19:34:46 +00:00
Zixuan
c03686cacb SERVER-117854 Add assert on inf numberfor GranularityRounderPowersOfTwo (#47606)
GitOrigin-RevId: 1bdac3ba9f86f936d16cef4296e2e58d6e228138
2026-02-18 19:34:28 +00:00
Jordi Serra Torrens
aea6ff0e69 SERVER-118466 Prevent the TTL Monitor from spawning an unbounded amount of threads for sharding metadata recovery (#47972)
GitOrigin-RevId: 112c656050aba8f328e916b653a84843e7003a71
2026-02-18 19:34:10 +00:00
Silvia Surroca
4140d9d592 SERVER-117009 setClusterParameter silently resets to default when misspelling non-strict cluster parameter fields (#47827)
GitOrigin-RevId: ed1e6094c87dfcb69c60f807f955b3e7989c11bf
2026-02-18 19:33:53 +00:00
Jordi Serra Torrens
8ae6ea43ac SERVER-106390 Make apply ops trigger a sharding metadata recovery upon stale shard errors (#47892)
GitOrigin-RevId: f13ad657670efa40255a5a8857818102da16b26e
2026-02-18 19:33:34 +00:00
Ivan Fefer
e03c939eb6 SERVER-118849 Fix hash lookup when duplicate key causes spilling (#47636)
GitOrigin-RevId: 83751c5457fea4cdc4a4058bee4f3b8b08245b46
2026-02-18 19:33:16 +00:00
Ivan Fefer
ff55682afe SERVER-119091 Fix lookup hash table strings with collation spilling (#47779)
GitOrigin-RevId: aea49593e08fd39df872af0e0cbb8efb71b6da0b
2026-02-18 19:32:58 +00:00
blagoev
88fd06161a SERVER-114460 Add benchmarks_sep to v8.2 branch (#46774)
GitOrigin-RevId: f7d2b22cc8d6efe700c8512ef8171af6b7374e88
2026-02-18 19:32:40 +00:00
Joseph Prince
e2fec6eb56 SERVER-117564 Properly handle null device for logpath on Windows (#47618)
GitOrigin-RevId: 035f5404e0ef01c1b454c3b204a7d254e1ef1568
2026-02-18 19:32:22 +00:00
Kruti Shah
a6ab931c9a SERVER-116528 Balancer does not apply ForceJumbo::kForceBalancer to draining-shard migrations (#47810)
GitOrigin-RevId: 757f0f651586fee563bfafb76ba4b0bb4fbe15d0
2026-02-18 19:32:05 +00:00
Jordi Serra Torrens
f55f2b459f SERVER-117623 Revert SERVER-95330 + extend test for sharded txn and DDL interleaving (#47057)
GitOrigin-RevId: d4433faafb9ded7ac911e3c3116d43e4457421f8
2026-02-18 19:31:48 +00:00
Gregory Wlodarek
5710e50f4f SERVER-111573 Add configuration to return full WiredTiger data source metrics (#44030) (#47298)
GitOrigin-RevId: 3f6d4aa930a5504201f33c556bd7faa2a4b794f3
2026-02-04 22:14:49 +00:00
Pierlauro Sciarelli
5f8dbc8713 SERVER-113352 Implement command to persist range deletion task documents for unowned non-empty ranges (#47314)
GitOrigin-RevId: 6f7cec27927b50fe786d2c0ef8693ffdedba5d7a
2026-02-04 22:14:31 +00:00
Zack Winter
fc727a014e SERVER-117433 [v8.2] Add private third party dependency directory (#46743)
GitOrigin-RevId: 720af1193691c20585306d70a951dad9d9b1dfe7
2026-02-04 22:14:12 +00:00
Jordi Serra Torrens
63acb191e9 SERVER-116089 Add lock acquisition timeouts to migration's recipient critical section (#47124)
Co-authored-by: Meryama <meryama.nadim@mongodb.com>
GitOrigin-RevId: ec981c855389d72f8e5eabd94bc87f0f7208efa0
2026-02-04 22:13:54 +00:00
Steve McClure
a84c0fa6ea SERVER-117252: [8.2] Remove 6.0 from multiversion testing (#46651)
GitOrigin-RevId: 0cc02ac0f37d5ee9c6877c4320eb980a73dd1b72
2026-02-04 22:13:36 +00:00
Trevor Guidry
9b2d350c40 SERVER-117088 increase pymongo timeouts on asan variants (#46724)
GitOrigin-RevId: cef4b247f1e5fa2338339f565434f27eb46bce8e
2026-02-04 22:13:18 +00:00
Gregory Wlodarek
1e575782ca SERVER-117467 Skip running too_many_fds.js on macOS (#46655) (#47097)
GitOrigin-RevId: eb360d76aca024f0c593c42d346381cce3354ee0
2026-02-04 22:13:00 +00:00
Cole Harbeck
676409152e SERVER-117190 Add synchronization to ShutdownBehavior test in dbclient_grpc_stream_test.cpp to prevent race condition (#47001)
GitOrigin-RevId: 2fcc2408853c58c13f6b9c05e58f712d09b8f33b
2026-02-04 22:12:42 +00:00
seanzimm
290a3eb838 SERVER-106239: Relax assertion requirements for transaction readonly values (#46876)
GitOrigin-RevId: b1f08e2c9805c6bff8bf22728327ac72f5f14e4e
2026-02-04 22:12:22 +00:00
Erwin Pe
65de5d7e7a SERVER-115132 Fix Queryable Encryption mirrored reads (#45700)
GitOrigin-RevId: b54207964b2080769ae2d37a70ee52a7fdfc995d
2026-01-27 18:51:26 +00:00
Zack Winter
41edcff4b9 SERVER-117274 [v8.2] Update test_packages to download the dist binaries via evergreen API (#46929)
GitOrigin-RevId: e87248a17a296b5ea663968a92f41b1c36b7fae5
2026-01-27 18:51:08 +00:00
Zack Winter
38ab0ee533 SERVER-117755 [v8.2] Backport .gitignore from master (#46926)
GitOrigin-RevId: f9277f3caf3507b5a3aeb75c3a50af6bcadb2e2e
2026-01-27 18:50:50 +00:00
Zack Winter
f2d2c748ae SERVER-110304 [v8.2] Update BES Branch Keyword to use Evergreen Branch Name (#46924)
GitOrigin-RevId: e178dd40ce0c7bbbcb6fcd9b1b780d6a48b2700b
2026-01-27 18:50:31 +00:00
Patrick Freed
c2f432e02f SERVER-114890 Use more human-friendly ingress request rejection error message (#45811)
GitOrigin-RevId: 69d29e4d14a7a90f62d28678cd9430b3e5a2bedd
2026-01-22 18:03:24 +00:00
Patrick Freed
e142ec7e13 SERVER-113311 Attach NoWritesPerformed error label to IngressRequestRateLimiter rejections (#45805)
GitOrigin-RevId: 8763ea147e9b0e359bf6d5d1209cdabc669544ca
2026-01-22 18:03:06 +00:00
Jan
9d1682cb70 SERVER-110314 Make spilling test more resilient against background query activity (#40920) (#46643)
GitOrigin-RevId: 2e401825c3cf47d982bc55dde4e4b6107cf8be26
2026-01-22 18:02:48 +00:00
Pierlauro Sciarelli
c3a00304eb SERVER-113915 Remove viewless timeseries suites from v8.2 branch (#44766)
GitOrigin-RevId: 04065a6e17caf9828727a48690ec9a13e8a6b76a
2026-01-22 18:02:30 +00:00
Moustafa Maher
35bac269ba SERVER-113508 Should wait for old primary to complete stepdown in auto_reconfig_remove_newly_added_and_stepdown.js (#43616) (#46364)
GitOrigin-RevId: c9af51a3bb367249a76c2de58a0780e9958d826d
2026-01-22 18:02:12 +00:00
Moustafa Maher
455604a26c SERVER-113545 Wait for lastCommittedOpTime to be initialized after restart in api_version_prepare_txn.js (#43681) (#46365)
GitOrigin-RevId: 9cc97562ed12cab1ed54855e1939e36b5da95d35
2026-01-22 18:01:54 +00:00
Luke Chen
5bc14fbeec SERVER-117202 Add server-release as code owner for wiredtiger source folder (#46414)
GitOrigin-RevId: ec819e534a3803a2a9b1ae22c36b189e1064614e
2026-01-22 18:01:35 +00:00
Ivan Fefer
3c3fb98ec8 SERVER-114384 Share query-wide constant variables between Variables instances (#46288)
GitOrigin-RevId: 99ea4ce88252708fb2505d923b5c76d10608df66
2026-01-22 18:01:17 +00:00
wolfee
7a9dabec9c SERVER-116294 SetClusterParameterCoordinatorDocument crash on upgrade due to new non-optional field missing (#45965) (#46153)
GitOrigin-RevId: 95c9f75235bdc75c28dcb2a74cfef3eadcf207dc
2026-01-22 18:00:57 +00:00
Indy Prentice
45279cc75b SERVER-115326: Don't reset logical session cache reaper metrics prematurely (#46151)
GitOrigin-RevId: 0938e8144141fdf13b416258b5b61b6cf3e3b34b
2026-01-15 22:26:47 +00:00
Malik Endsley
701f29ddcc SERVER-110842: commitTimeseriesBucketForBatch and commitTimeseriesBucketsAtomically should set the collection catalog within the scope of acquireAndValidateBucketsCollection (#45620)
Co-authored-by: Stephanie <53684987+seristof@users.noreply.github.com>
GitOrigin-RevId: daeccb8cfad449fa2363a948904e6a27eeed3a0f
2026-01-15 22:26:29 +00:00
Joshua Siegel
0d05f10f27 SERVER-112786 Enable internalVectorSearchStoredSource query knob (#46123)
GitOrigin-RevId: 03a87330db556f2d2301d1c6e7499922d35f8506
2026-01-15 22:26:11 +00:00
Sean Lyons
f2b0e65789 SERVER-114974 Avoid conflicts with pipx-installed versions of db-contrib-tool (#45795)
GitOrigin-RevId: 7f84e9a9c5739a2883aca3a2d7f8a54a4aa95b0a
2026-01-15 22:25:53 +00:00
Allison Easton
3efaf08dc7 SERVER-111031 Cluster wide default read/write concerns are not honored during promotion to sharded (#45950)
Co-authored-by: Meryama Nadim <meryama.nadim@mongodb.com>
GitOrigin-RevId: d77e712fb7ccc37be1213d53f5d4b8d08e99f830
2026-01-15 22:25:34 +00:00
wt-vendoring-bot[bot]
bca052045b Import wiredtiger: bf55d99eb5b5f9fb9d51fb5c6bbb335bf569ef5f from branch mongodb-8.2 (#46180)
Co-authored-by: wt-vendoring-bot <wt-vendoring-bot@mongodb.com>
GitOrigin-RevId: afb70d316c668699d8b8fb5a9eebbecebe87ccf2
2026-01-15 22:25:17 +00:00
Tommaso Tocci
fa497b53f1 SERVER-113997 SERVER-114329 Fix legacy timeseries namespace translation in findAndModify explain command (#44558)
Co-authored-by: Joan Bruguera Micó (at MongoDB) <joan.bruguera-mico@mongodb.com>
GitOrigin-RevId: 299aeac990ac6467ae0d16a51fc9b0c68d5baa33
2026-01-15 22:24:58 +00:00
nicola cabiddu
3c3ed51901 SERVER-116344 Revert featureFlagSbeUpgradeBinaryTrees and featureFlagMultiPlanLimiter (#46012)
GitOrigin-RevId: 8a4066e569ecaad6c46e4df1ce15303d7954d1f2
2026-01-15 22:24:40 +00:00
Zack Winter
6ba665ccde SERVER-116078 Move AL2 Release Variant from compiling on AWS c-series to m-series (#45762)
GitOrigin-RevId: 55093301e05e1cb7a815892b0036c55a0d235082
2026-01-15 22:24:22 +00:00
Pierlauro Sciarelli
72629252a3 SERVER-115962 Balancer must progress when the most loaded shard is already balanced within its zones (#45948)
GitOrigin-RevId: 96fa001fc341864dee625501d849ed7ab6c675dd
2026-01-15 22:24:04 +00:00
Wei Hu
a5b3b8aa5b SERVER-114806 Add wildcard multikey metadata keys to sorter directly and deduplicate before bulk load (#45779)
GitOrigin-RevId: 628bab09cdacfd91a3442c74bf4aa4c6e4fe5799
2026-01-15 22:23:45 +00:00
Mike Nugent
cf18c7d602 SERVER-107966: Don't stop pinging when the ping result is "cancelled". (#45878)
GitOrigin-RevId: f71692173d9d3df47b9db4879be9eb2c84a50c52
2026-01-15 22:23:27 +00:00
Sean Lyons
c21ca3de19 SERVER-115576 Resolve commit hash for old versions from mongod (#45723)
GitOrigin-RevId: 28cf41a05f212d4f0d1988bd8617398ba36bb3ed
2026-01-15 22:23:09 +00:00
Xuerui Fa
6eba2d6cb5 SERVER-115594: Skip acquiring tickets when deleting txn coordinator d… (#45871)
GitOrigin-RevId: f028650a111606a7ae8cde8cf056501031c093ea
2026-01-15 22:22:50 +00:00
Daniel Moody
e4fcbbf92a SERVER-115917 set specific Visual Studio and MSVC install version (#45577)
GitOrigin-RevId: afa10fd1f222edcb0a059013b1ff522417c72370
2026-01-15 22:22:31 +00:00
Cheahuychou Mao
6a57b75a74 SERVER-108280 Prevent analyzeShardKeyNumRanges from being set to 1 (#45857)
GitOrigin-RevId: 866cd1b72efb878870f35b74196e049a861617ad
2026-01-07 16:14:41 +00:00
Wei Hu
ee8722fdf6 SERVER-103955 Primary only transitions to commitQuorumSatisified after itself has voted (#45227) (#45781)
GitOrigin-RevId: 45b146efe8c55f45b2de0976d9f6eb312f01e599
2026-01-07 16:14:22 +00:00
Allison Easton
77ef569881 Revert "SERVER-111031 Cluster wide default read/write concerns are not honored during promotion to sharded (#45673)" (#45844)
GitOrigin-RevId: 408c908b8216922002cbc0191282b48c89b1d091
2026-01-07 16:14:05 +00:00
Yuhong Zhang
ede7a0e5a2 SERVER-112539 Abort time-series write batch on Stale Version errors (#45733)
GitOrigin-RevId: 76bff1cd36c666382690654df98d2deb1c4cdfea
2026-01-07 16:13:47 +00:00
Logan Karbiner
7fc6b15695 SERVER-112267 Make RCString::create throw BSONObjectTooLarge (#43469)
GitOrigin-RevId: 5c662c7103827ec43b6aff3b2023b1215a0b816e
2026-01-07 16:13:29 +00:00
Allison Easton
d130eb1c50 SERVER-111031 Cluster wide default read/write concerns are not honored during promotion to sharded (#45673)
GitOrigin-RevId: 43937ccecd827c2d0ecf45d84b2b3668a0ddad66
2026-01-07 16:13:11 +00:00
Matt Kneiser
a0fa6de4bf SERVER-113484 Add test suite exclusion for timeseries_insert_respect_max_bson_size_too_big (#43656)
GitOrigin-RevId: f9c8a302af696aa0f4603336d249aa6f6212cf47
2026-01-07 16:12:53 +00:00
wt-vendoring-bot[bot]
0c65ad8423 Import wiredtiger: 607c3cd185afef9edfd56e180236a58062353e84 from branch mongodb-8.2 (#45664)
Co-authored-by: wt-vendoring-bot <wt-vendoring-bot@mongodb.com>
GitOrigin-RevId: 1d81b4b267c9ce07cbd28419f265982cc084b50c
2026-01-07 16:12:35 +00:00
Damian Wasilewicz
88b6828d24 SERVER-114363 SERVER-115398 Add metrics that capture the duration for which index builds can be stalled/block replication (#44965) (#45331)
GitOrigin-RevId: 3fb10a74bc0d33f83e94324fc6a7379dfcfb37f2
2026-01-07 16:12:16 +00:00
Mark Benvenuto
ed46e9c63a SERVER-115438 Make minimally sized buffers for uncompressed Messages (#45581)
Co-authored-by: Spencer Jackson <spencer.jackson@mongodb.com>
GitOrigin-RevId: 25381e40e8f1a2c00d03a93a93eb7e9cbd36ba21
2025-12-23 18:58:40 +00:00
Matt Kneiser
82f22ff185 SERVER-112342 ServerStatus should have a parameter to exclude all fields (#43748)
GitOrigin-RevId: a2c339a4a51a796e483da74b69ff2ac885eb3f08
2025-12-23 18:58:22 +00:00
Joan Bruguera Micó (at MongoDB)
0639250f67 SERVER-114666 Forbid tracking temporary collections in the global catalog (#45371)
Co-authored-by: Meryama <meryama.nadim@mongodb.com>
GitOrigin-RevId: f9c8a9984c932c35f62596ab3b9a4d381f09675c
2025-12-23 18:58:04 +00:00
wt-vendoring-bot[bot]
7ba4b31490 Import wiredtiger: 9fdd8f56a6b0b2a5c0369f3928b9353626b930a3 from branch mongodb-8.2 (#45561)
Co-authored-by: wt-vendoring-bot <wt-vendoring-bot@mongodb.com>
GitOrigin-RevId: b4975ebb11d209f09ff02bab5994be54717a3057
2025-12-23 18:57:47 +00:00
Niels Lohmann
05e40124fa SERVER-110338 backport add the option to set incremental features with the output of the getParameter to v8.2 (#44997)
Co-authored-by: Nicola Cabiddu <nicola.cabiddu@mongodb.com>
Co-authored-by: Victor Ghita <tomi.gvf@gmail.com>
Co-authored-by: Nicola Cabiddu <nicola.cabiddu@mongodb.com>
GitOrigin-RevId: 36a3c2bf2b116ce392df0d0ec8e268a6939544aa
2025-12-23 18:57:29 +00:00
Cole Harbeck
2c091b5ee4 SERVER-114695 Update default value for pending connections (#45200)
GitOrigin-RevId: 09816a4ce8de46cadc79e06e30a2cb26ee9e87e3
2025-12-23 18:57:12 +00:00
Sean Lyons
5ba0504741 SERVER-115245 core-analyzer: match program name from core dumps across new lines (#45514)
GitOrigin-RevId: a2c361260454dcffeb02957b148bb1ce420b47b0
2025-12-23 18:56:54 +00:00
Alya Carina Berciu
a2ccb4fe31 SERVER-114126 Reduce memory usage and add new guardrails to the boolean simplifier (#45180)
GitOrigin-RevId: 3082596a9b149841391efd8c1ab82234600a42bf
2025-12-23 18:56:36 +00:00
Ivan Fefer
3a9ddf5e84 SERVER-115716 Fix concurrency issue in server_status_multiplanner.js (#45500)
GitOrigin-RevId: ed7eb3925fd9095845396625635123254c5f7545
2025-12-23 18:56:19 +00:00
Jordi Olivares Provencio
dd9d82c6da SERVER-114838, SERVER-115296: Salt ResourceIds with a random value acquired at startup (#45118)
GitOrigin-RevId: 3517e9d98d24fcde732f6ab952527d049066f31f
2025-12-23 18:56:02 +00:00
Niels Lohmann
048a17c79f SERVER-115087 Re-enable featureFlagSbeUpgradeBinaryTrees and featureFlagMultiPlanLimiter (#44973)
GitOrigin-RevId: 756f6c29c333fe0867343574789de1aa03aa75e1
2025-12-23 18:55:44 +00:00
Niels Lohmann
da25fbc10b SERVER-114919 Refactor code to silence Coverity (#44975)
GitOrigin-RevId: 2a069cb22edc27c73675ff90e49971285c3437b7
2025-12-23 18:55:27 +00:00
Niels Lohmann
23dd776926 SERVER-114927 Fix overflow (#44974)
GitOrigin-RevId: 2dda5a7d3ce9b689495c473e01c91417afc404d4
2025-12-23 18:55:09 +00:00
Marcos Grillo
29367986fc SERVER-113376 Make check_metadata_consistency.js resilient to CSRS stepdowns (#44245) (#45358)
GitOrigin-RevId: cb2989a3c2a3ccb5cb8d9a813098ca9730206cb7
2025-12-23 18:54:52 +00:00
Adi Zaimi
e035a53c64 SERVER-113659 Add error label from chunk migration aborting when existing document is found (#45325)
GitOrigin-RevId: ee919e10aaafc58e808d5324eb45a6dc37dea2e7
2025-12-23 18:54:34 +00:00
Luke Chen
655ebb0fa1 SERVER-115539 Set wiredtiger-release as code owner for wiredtiger sources (#45285)
GitOrigin-RevId: dbbed4a4af314b2aa03e8e358baa1efd350837d8
2025-12-23 18:54:17 +00:00
Anna Veselova
61d51036f9 SERVER-113893: Improve memory handling in mozjs allocator (#45077)
GitOrigin-RevId: 453ee2003fd56823649f6d16f15fe5edbaadd2a9
2025-12-23 18:53:59 +00:00
Damian Wasilewicz
9a7c840349 SERVER-110442 SERVER-114337 Capture duration for last committed index build (#43531) (#45104)
Co-authored-by: Cedric Sirianni <cedric@sirianni.dev>
GitOrigin-RevId: 4cd153494e158413ec76b3b9b8838174aa0991d0
2025-12-23 18:53:42 +00:00
Evelyn Wu
73dadb8ec4 SERVER-102018 Add more information invariants in repl code base (#40211) (#44268)
Co-authored-by: Evelyn Wu <evelyn.wu@mongodb.com>
GitOrigin-RevId: 57ca066d7f2ef6bce7789053226034b7e32045f1
2025-12-23 18:53:24 +00:00
Benjamin Pearce
db35cc3354 SERVER-113888 SERVER-113889 Internal-only indices can only be created by the system (#44736)
GitOrigin-RevId: c3fefab8ee206a8b080f2b3d7a02fa9ededc37e6
2025-12-23 18:53:06 +00:00
adelinexchen
93f0027118 SERVER-111672 create throttle test clock source closer to when test begins (#41964) (v8.2 backport) (#44482)
GitOrigin-RevId: 42c43636c16f41dbe2761b5e1d5c03dfea4471e7
2025-12-23 18:52:49 +00:00
Silvia Surroca
84c733352b SERVER-112792 Report existance of compound wildcard indexes prefixed by the shard key (#44877)
GitOrigin-RevId: d546ecc6ef8455df3a29f88b56a994b4fe43e53a
2025-12-23 18:52:31 +00:00
fotiniAlvanaki
c87b7b2632 SERVER-113711 : Rate Limit Log unable to establish remote cursors (#44775)
Co-authored-by: kmznam <97981975+kmznam@users.noreply.github.com>
Co-authored-by: kmznam <97981975+kmznam@users.noreply.github.com>
GitOrigin-RevId: b24c927e7b3b6d1e24dc4ae3e92ecf30b9088ba6
2025-12-23 18:52:14 +00:00
Jan
48f10211e3 SERVER-86326 Increase regex max pattern length to 32764 characters (#44702)
GitOrigin-RevId: e9c1477062541d99b450688b036be72a5ccc87c1
2025-12-23 18:51:56 +00:00
Charlie Swanson
30e5202326 SERVER-111191 Discard profiler entries upon LockTimeout [v8.2] (#44812)
GitOrigin-RevId: 9e8dfb52e4121ea2a4825d9dd7ded4b2377fe2c3
2025-12-23 18:51:37 +00:00
Binh Vo
9ef9154b3c SERVER-114755 Handle endianness in control block generation in binary_reopen_test (#44957)
Co-authored-by: Benjamin Pearce <benjamin.pearce@mongodb.com>
GitOrigin-RevId: d35729b3aa5f642270ba23cc99f8a0487bf09904
2025-12-12 16:05:35 +00:00
adriangzz
f3f6fd6173 SERVER-112231 Fix race condition when stopping MockOCSPServer (#43417) (#44825)
GitOrigin-RevId: e4f1c4f4b9d970a28594a20a590d95f9f0bfcdd6
2025-12-12 16:05:17 +00:00
Charlie Swanson
19897d58bd SERVER-112952 Consider 'filter' in profile cmd auth check [v8.2] (#44639)
GitOrigin-RevId: c8f52d2388027d056aaf04f4f6c55c3a599f5631
2025-12-12 16:04:59 +00:00
Erwin Pe
e49562e8d6 SERVER-113550 Fix ticket exhaustion issue in FLE2 aggregation and explain (#44946)
GitOrigin-RevId: 4ee7aee06e13a640d400de56108ec676477ca7ef
2025-12-12 16:04:41 +00:00
Daniel Moody
e6a4d5e471 SERVER-114938 clean up auto header dirs (#44863)
GitOrigin-RevId: bcd5d6ce137d8429fd04728d13563a5c7996a51a
2025-12-12 16:04:23 +00:00
Militsa Sotirova
f0112f35f2 SERVER-113685 SERVER-113900 SERVER-113897 Disallow wildcard index creation where projection is only _id exclusion (#44569)
GitOrigin-RevId: 6841449b634f9cea3d404e7615a0fe6e9e927db4
2025-12-12 16:04:06 +00:00
Alexander Ignatyev
d2c8ab0fd0 SERVER-110803 Take into account sort direction in group stage when performing distinct scan optimization (#43768)
GitOrigin-RevId: b1b4fb208086acb69331ce5457e30e29a6c25395
2025-12-12 16:03:48 +00:00
Binh Vo
af92bd90f3 SERVER-111007 Validation distinguishes between root causes when finding mixed schema data (#44744)
GitOrigin-RevId: 38693f4d7f26af9c3fa7d3202ebb0e94faf5c89d
2025-12-12 16:03:31 +00:00
Henri Nikku
4e977bf676 SERVER-111876 Fix how the planner handles negations of $_internalExpr comparisons to null (#42180) (#44774)
GitOrigin-RevId: ba9f83b15a3146a819ef9133322052f5a905dcf2
2025-12-12 16:03:13 +00:00
Sean Lyons
991e1109a4 SERVER-114629 Upgrade db-contrib-tool to 2.1.0 (#44596)
GitOrigin-RevId: 3300da30a7f4e509bff1fc8045b30f858d1fe898
2025-12-12 16:02:55 +00:00
Jason Hills
97eec1df21 SERVER-114494 Improvements to SBOM automation [v8.2-staging] (#44588)
GitOrigin-RevId: cea36e68be3dff3e7dbebe40e7e10793ac61ded1
2025-12-12 16:02:37 +00:00
Ivan Fefer
382b41c9d6 SERVER-114196 Fix $unionWith system variables initialization (#44444)
GitOrigin-RevId: cacd16d51c46e07a2b1b427c155847a7856c0267
2025-12-12 16:02:19 +00:00
Trevor Guidry
d8cc9dd86b SERVER-114407 Delete antithesis tasks (#44637)
GitOrigin-RevId: af705e0baa6d9623c020774273bbf3eac1758622
2025-12-12 16:01:59 +00:00
Sean Lyons
f6411d0808 SERVER-111603 Use large distro for aggregation fuzzer tasks (#42365)
GitOrigin-RevId: 28b2219663059fb127c4db68c3150d797b6c53c1
2025-12-02 20:36:17 +00:00
wt-vendoring-bot[bot]
13fc9420f4 Import wiredtiger: baf8c83f285bb15e4a230068aa23958a7817e9fa from branch mongodb-8.2 (#44541)
Co-authored-by: wt-vendoring-bot <wt-vendoring-bot@mongodb.com>
GitOrigin-RevId: a04b90d0107eb4283425d0f698b4f145d5a614c9
2025-12-02 20:35:59 +00:00
Jordi Serra Torrens
cb23179906 SERVER-108256 Fix untracked secondary collections potentially being considered as local on non db-primary shards (#44527)
GitOrigin-RevId: 19a8975cd5cc88a2709161ba781c0d7ae3a19afd
2025-12-02 20:35:41 +00:00
Meryama
3145debd5e SERVER-114017 Fix move_primary_setFCV.js creating database at latest FCV instead of downgraded FCV (#44332) (#44335)
GitOrigin-RevId: cc22fb08d89fa855d18cba9b1a41fc0f6ac1dd1a
2025-12-02 20:35:23 +00:00
henrikedin
848dace7fc SERVER-100659 Generalize BSONColumnBuilder binary reopen (#44341)
Co-authored-by: Daniel Moody <dmoody256@gmail.com>
GitOrigin-RevId: 37753ad80748c2015a59c2099a3b6ffb8d14291f
2025-12-02 20:35:04 +00:00
HanaPearlman
c6587d9cb9 SERVER-113923: Change index bounds builder check for multkey paths with comparison from tassert to uassert (#44105)
GitOrigin-RevId: ec48110abf66660caf141ce62317eb151e2a841b
2025-11-25 19:37:40 +00:00
Udita
19707dbf10 SERVER-112955: backport, 8.2, emergency_release (#43792)
GitOrigin-RevId: d76e772d4b4c32967f61f840ec7d34fe0490b9f6
2025-11-25 19:37:22 +00:00
henrikedin
beca4ba8b2 SERVER-111292 Reenable validation for geo indexes (#44340)
GitOrigin-RevId: b47900b2576d1ebe47c3d80dedf4f01304ba479f
2025-11-25 19:37:04 +00:00
Jason Hills
6c6a5c3656 SERVER-111072 Automate SBOM Generation from Endor Labs API [v8.2] (#43851)
GitOrigin-RevId: 2c9ffad0948f0a48dd5aee75b6906e52a61dec7f
2025-11-25 19:36:47 +00:00
Andrew Bradshaw
a863c9fadc SERVER-108208 Update debian 10 urls when testing packages (#39146) (#44267)
GitOrigin-RevId: 67922d5e793670f668b6aa634222e9108c1327b8
2025-11-25 19:36:29 +00:00
David Storch
aeb19e6da3 SERVER-113319 fix match pushdown when expression root is a renamed field path expression (#43893) (#44063)
GitOrigin-RevId: 1b0c883f8ae6f8d89a7f37e7b3459728125b2c58
2025-11-25 19:36:12 +00:00
Zack Winter
455c919165 SERVER-113700 [v8.2] Update copybara to v20251110 (#43804) (#44162)
Co-authored-by: Udita <83602637+ubatmongo@users.noreply.github.com>
GitOrigin-RevId: a8c4e4d3a47cbe7059efb34601a349cc311ded0e
2025-11-25 19:35:54 +00:00
Pol Piñol Castuera
bcf1ea4b28 SERVER-112357 moveChunk with waitForDelete must not hang when range deleter is disabled (#44167)
GitOrigin-RevId: 720d45cbdf1495194807c6c1fcadab11e92af6af
2025-11-25 19:35:35 +00:00
Pol Piñol Castuera
ba235332c5 SERVER-112933 Escape CheckMetadataConsistency hook assertion from running in sharded clusters (#44165)
GitOrigin-RevId: 2aefebf4d0cadce39570de96756e310b3f806d8a
2025-11-25 19:35:17 +00:00
Benjamin Pearce
b559a3aab3 SERVER-108699 Added check for BSON object size to collection validation (#43997)
GitOrigin-RevId: b8fd156b1b8c27a874b121680f3e5f8b7de7a8c2
2025-11-25 19:34:59 +00:00
Damian Wasilewicz
bb03ca79ce SERVER-113887: Add timeseries_create_indexes.js to backports_required_for_multiversion (#44017)
GitOrigin-RevId: 547bc5f8af935fd827bdb486b167549e5ddd787b
2025-11-25 19:34:39 +00:00
Max Verbinnen
8ed73d25fd SERVER-113468 Filter out duplicate rawData field for merging command (#43765)
GitOrigin-RevId: 3ddd1c23420e1c4bcd0b2c819b5411883794b407
2025-11-19 17:52:22 +00:00
wt-vendoring-bot[bot]
375aac847f Import wiredtiger: 6d23f8b399935ca913c701ca6194ed3e2c0fd743 from branch mongodb-8.2 (#44027)
Co-authored-by: wt-vendoring-bot <wt-vendoring-bot@mongodb.com>
GitOrigin-RevId: 9b1d7d01f008644b806123dd4d50b08b4d53b8ed
2025-11-19 17:52:04 +00:00
Silvia Surroca
0422ebf9a5 SERVER-113361 Race condition on cluster time value in vector clock unit test (#43929)
GitOrigin-RevId: 0a0afe8f0b33d72d5bc4f5ba2fc4b865b4f5a92a
2025-11-19 17:51:46 +00:00
Pierlauro Sciarelli
05303d777f SERVER-90889 SERVER-110952 SERVER-112678 validate bucket collections metadata in checkMetadataConsistency (#43872)
Co-authored-by: Joan Bruguera Micó (at MongoDB) <joan.bruguera-mico@mongodb.com>
GitOrigin-RevId: f96650ceb4a4fc1876ecc8190a3a05274ed87476
2025-11-19 17:51:28 +00:00
Cole Harbeck
8dad976ba2 SERVER-106302, SERVER-112636: Provide visibility into the sizes of serverStatus sections if the aggregate FTDC object is too large (#43954)
Co-authored-by: Didier Nadeau <didier.nadeau@mongodb.com>
GitOrigin-RevId: 6463d8b8f906909e741dcae0b67528d16c7f7882
2025-11-19 17:51:10 +00:00
Cole Harbeck
74692da3cc Revert "SERVER-106302, SERVER-112636: Provide visibility into the sizes of serverStatus sections if the aggregate FTDC object is too large" (#43892)
GitOrigin-RevId: 6febcafd6bca497c03af65725513cb87a960cfbb
2025-11-19 17:50:52 +00:00
Anna Maria Nestorov
e6d7124994 SERVER-111423 Create setParameterOnAllNonConfigNodes to set parameter on all non config nodes and add tag to guarantee topology stability during test execution (#42368) (#43397)
GitOrigin-RevId: 68365d94e10c31844f8d7465ab8f0947f678a0d0
2025-11-19 17:50:33 +00:00
Cole Harbeck
1bff549d1f SERVER-106302, SERVER-112636: Provide visibility into the sizes of serverStatus sections if the aggregate FTDC object is too large (#43357)
Co-authored-by: Didier Nadeau <didier.nadeau@mongodb.com>
GitOrigin-RevId: e56171ec00d14c959cdd2b3acc1164a054ff76fb
2025-11-19 17:50:15 +00:00
Shin Yee Tan
a5aeb4d387 SERVER-104355 Report Spill Engine Storage Stats to CurOp (#40068) (#42189)
GitOrigin-RevId: c456e43a3283234997299314c8a552ae8ffdb99f
2025-11-19 17:49:57 +00:00
Philip Stoev
6707a21134 SERVER-113153 Make query multiversion fuzzers deterministic in LTS versions (#43770)
GitOrigin-RevId: dc14bcd682e7a864774cd324d5629d123e414a65
2025-11-19 17:49:38 +00:00
Gregory Wlodarek
18e9ee9904 SERVER-112991 Update error message to remove incorrect assumption that compact gets EBUSY due to cache pressure (#43184) (#43808)
GitOrigin-RevId: e7d4a29ff5071971e58b88c13a36e73aaffe37cf
2025-11-19 17:49:20 +00:00
Joan Bruguera Micó (at MongoDB)
30bb760be0 SERVER-107819 Concurrent createIndex+collMod can leave behind inconsistent collection options (#42830)
GitOrigin-RevId: 28a8bc1dbaabe13692042f2bb3e628da2207d0c6
2025-11-19 17:49:02 +00:00
Kruti Shah
989e1357f1 SERVER-113020 Relax majorityReplicationLagMillis check on donor only shards in resharding_commit_monitor_repl_lag.js (#43271) (#43678)
GitOrigin-RevId: f23b64dcef14740c420cd51d54fd25f3738cd1c5
2025-11-19 17:48:43 +00:00
gitpushjoe
47e6b69dce SERVER-110832 Oplog writer should not hold on to session through rollbacks (#43643)
GitOrigin-RevId: d30b6856465084805ccffbb393bf067a08dadaea
2025-11-19 17:48:24 +00:00
wt-vendoring-bot[bot]
d7a239f410 Import wiredtiger: 8aa4baf9e9d7a8398458c7ca253b7d959bfa374a from branch mongodb-8.2 (#43754)
Co-authored-by: wt-vendoring-bot <wt-vendoring-bot@mongodb.com>
GitOrigin-RevId: 58de201582888e501a0be011d3b341fcdca62e41
2025-11-19 17:48:06 +00:00
Max Hirschhorn
ae73e84ece SERVER-112250 Test shard mongod starting without --shardsvr. (#42633)
GitOrigin-RevId: 470fe8a966ae4185b061921a634df94a640e8235
2025-11-19 17:47:47 +00:00
Zack Winter
7277584dd6 SERVER-113431 Pin copybara to a specific tag (#43548) (#43549)
GitOrigin-RevId: 3e0f6085ff64675144b7f657493578ec7d26d2bc
2025-11-19 17:47:29 +00:00
wolfee
85729d2e4f SERVER-110008 Ignore ddlLockOptimisticRecoveryWaitTimeout in multiversion server parameter tests (#40711) (#43577)
GitOrigin-RevId: b242a3fd3bda9bb34eb8b3f3a4d98b30e83bf788
2025-11-19 17:47:10 +00:00
Tommaso Tocci
dd09d25acc SERVER-108801 Invalid collMod command with timeseries granularity change creates collection options inconsistency (#43557)
Co-authored-by: Allison Easton <allison.easton@mongodb.com>
GitOrigin-RevId: 864b42a92e20eccd62e08c46c51ac530d6708058
2025-11-19 17:46:48 +00:00
Andrew Bui
2ab5dad532 SERVER-107228 Update GHA comments with Regex and for clarity (#42851)
GitOrigin-RevId: c8bff7f958862ecb0ddccb7b3cfe6864e0af7611
2025-11-19 17:46:28 +00:00
Andrew Bui
0b31cdade8 SERVER-107228 Add Auto comment with Github Actions for Staging Branch (#39718)
GitOrigin-RevId: 285daa4d8b28bf8fb5128539d94154c5defccadd
2025-11-19 17:46:09 +00:00
ekovalets
ce63cdf537 SERVER-113503: Adding github org to repo name in the SilkBomb argument - 8.2 (#43609)
GitOrigin-RevId: b17e9f51cac9cd8d8f20e6017cd8c485e33d3426
2025-11-19 17:45:52 +00:00
Sean Lyons
e28ebc56bf SERVER-112491 Add binaries in the Evergreen working directory higher on the search path (#42835)
GitOrigin-RevId: 48240d47af9b30483f869dd596fd7ebf21e501b3
2025-11-19 17:45:32 +00:00
Malik Endsley
60c4ee2187 SERVER-109230: Use length of buffer when making std::string from RecordId (#43327)
Co-authored-by: Shin Yee Tan <shinyee.tan@mongodb.com>
GitOrigin-RevId: 37f2468c3e9d846f5e6f9e103122a71633d7fa46
2025-11-19 17:45:15 +00:00
ekovalets
2995019970 SERVER-112109: Evergreen Task and scripts to run Server SBOM upload to Google Drive at release - 8.2 backport (#43259)
GitOrigin-RevId: 8e7d55db8c55f73721cd21589aa3c39c0d315753
2025-11-19 17:44:58 +00:00
Joan Bruguera Micó (at MongoDB)
397eb0bcb4 SERVER-112969 Skip CheckReplDBHash index check for regular collection with accompanying buckets namespace (#43454)
GitOrigin-RevId: 21868c0e7bbe6f49b638e31e403092f0eb81f6a8
2025-11-19 17:44:37 +00:00
Janna Golden
3c9c2f2747 SERVER-106341 Disable remaining racy assertion in ReshardingOplogFetcherTest (#40429) (#43473)
GitOrigin-RevId: 7a29d2972b04728df26cacafd293d310a5957a90
2025-11-03 19:10:49 +00:00
Damian Wasilewicz
48ae227c07 SERVER-108354 Add StaleConfig as retryable error for timeseries_create_indexes.js (#39287) (#42018)
GitOrigin-RevId: 115cd28412f4840c7e1a48ffc2a4fc557bcb2edd
2025-10-29 21:51:35 +00:00
Evan Bergeron
d4511d28ec SERVER-110845 Use deleteMany instead of drop in index_key_expression.… (#43048)
GitOrigin-RevId: 594f839ceec1f4385be9a690131412d67b249a0a
2025-10-23 20:05:56 +00:00
clarissecheah
889e1451b1 SERVER-112846 Change in repl set reconfigure test (#43034)
GitOrigin-RevId: 31b7b6bbf424a62b56e7f95b21430fe2a3215745
2025-10-23 17:46:52 +00:00
clarissecheah
b7daf9f217 SERVER-112118 Track and log how long it takes oplog sampling to stop after its been interrupted (#42899)
Co-authored-by: Gregory Wlodarek <gregory.wlodarek@mongodb.com>
GitOrigin-RevId: 8d7caaf81e9d4b4fb8a1047fdfbee6fb0ed9ebe1
2025-10-23 17:46:34 +00:00
adelinexchen
2e5d23ff56 SERVER-112120 Check async oplog sampling does not block core operations (#42662)
Co-authored-by: Nic Hollingum <nic.hollingum@mongodb.com>
Co-authored-by: Gregory Wlodarek <gregory.wlodarek@mongodb.com>
GitOrigin-RevId: 947832b0bc0f33a729172a7d7e322e6145ee5000
2025-10-23 17:46:15 +00:00
Nic
58dca72b11 SERVER-112123: Test that async oplog sampling doesn't block replication configuration. (#42599)
GitOrigin-RevId: fc4b21358072074b7532d826ada3e5558fde1507
2025-10-23 17:45:57 +00:00
Nic
5981ffdf4f SERVER-112541: Report data scanned during oplog initial marker generation (#42728)
GitOrigin-RevId: 748745606ddf981f8e04796afa505c3f4267a7c6
2025-10-23 17:45:39 +00:00
adelinexchen
58fa16f921 SERVER-112119 hang OplogCapMaintainerThread during initial sampling, not after (#42726)
GitOrigin-RevId: e1e5c9586d607a8aaa8f71bf8c4babe4d888c287
2025-10-23 17:45:21 +00:00
Clarisse Cheah
e08e44656e SERVER-112183 Start async oplog sampling towards the end of startup (#42485)
GitOrigin-RevId: 34ecf551c9c36aa3b0abb56d173f8ff15c164425
2025-10-23 17:45:02 +00:00
Gregory Wlodarek
96f7a84d3f SERVER-112124 Test that async oplog sampling doesn't block shutdown (#42551)
GitOrigin-RevId: 615a518a03a19f1011355d6bab3951226bdd9943
2025-10-23 17:44:44 +00:00
Nic
aab726b545 SERVER-112117: Add yielding behaviour to async oplog sampling (#42493)
GitOrigin-RevId: 1636a99dc75b1ed9c1a1dba9c0789069298d4508
2025-10-23 17:44:26 +00:00
Gregory Wlodarek
d0e2146b8c SERVER-112121 Test that async oplog sampling doesn't block rollback-to-stable (#42550)
GitOrigin-RevId: dfd824bdd5503faa1649ab18cde53d6e2e4e9912
2025-10-23 17:44:08 +00:00
Gregory Wlodarek
2dc4bbb91a SERVER-112122 Test that async oplog sampling doesn't block FCBIS (#42546)
GitOrigin-RevId: 2da4fbce585c1adae80c322da949a47ae4f5e2a7
2025-10-23 17:43:49 +00:00
clarissecheah
9f2273839d SERVER-112182 Create a server parameter to simulate oplog sampling slowness (#42547)
GitOrigin-RevId: d589b51d533df2d05d346814d6d9914d07ea1123
2025-10-23 17:43:30 +00:00
Daniel Moody
cef94c64c9 SERVER-93265 support windows rc files in bazel (#42764)
GitOrigin-RevId: 3ac0a98ea7592082c103ae8a46bb597e45169a66
2025-10-22 22:11:06 +00:00
Militsa Sotirova
5410b0829b SERVER-110423, SERVER-111486: Prohibit group by distinct scan optimization if index is sparse (#42942)
GitOrigin-RevId: 0d79eb9797dc058b15851e223143e0a4a9fa0d71
2025-10-22 20:37:56 +00:00
Adi Zaimi
d528fdd2b4 SERVER-110953 Check and fail migration if invalid existing documents exist (#42843)
GitOrigin-RevId: 6f27f90c146221be2bddf8b93eb8593cf45a143d
2025-10-22 19:26:43 +00:00
Jan
3585bbee40 SERVER-112429 Make SBE hash lookup tests more efficient (#42602) (#42618)
GitOrigin-RevId: ed7c93f98afa2a5088083f0c49cd79f2d1d4ec3f
2025-10-22 19:26:24 +00:00
Jan
f53ae515e8 SERVER-112127 Fix ambiguous hash value handling in FieldPath (#42344) (#42934)
GitOrigin-RevId: f2bda24e635fa702d0bea7e5c500a1d4517de48e
2025-10-22 19:26:05 +00:00
Varun Ravichandran
6466b2fb48 SERVER-106946 Reject x.509 certificates with mismatched client|serverAuth EKU (#42974)
GitOrigin-RevId: f1aa7c70c6d003e2496ac51f7eba4904b3bb4159
2025-10-22 18:09:45 +00:00
Erin McNulty
0facf06066 SERVER-112810 Fix error code in ingress_request_rate_limiter_app_exemption.js (#42994)
GitOrigin-RevId: 7f821b3e0bfc1090c00737b201351ca2ec573e67
2025-10-22 14:52:02 +00:00
Saman Memaripour
5dcc6f66f7 SERVER-112467, SERVER-112035: Track and report the number of pending proxy connections (#42710)
Co-authored-by: David Goffredo <david.goffredo@mongodb.com>
Co-authored-by: Didier Nadeau <didier.nadeau@mongodb.com>
Co-authored-by: David Goffredo <david.goffredo@mongodb.com>
GitOrigin-RevId: 445735bd31ba7cbab4e84a9e50095fbff4edbe4e
2025-10-21 17:03:03 +00:00
Marcos Grillo
be44b88830 SERVER-106590 Relax db lock to MODE_IX on movePrimary's cloner code to prevent contention on migrations (#39184) (#42695)
Co-authored-by: Allison Easton
Co-authored-by: Allison Easton <allison.easton@mongodb.com>
GitOrigin-RevId: 762d47a43180102203e9ea2473e236643822ba36
2025-10-21 16:48:48 +00:00
Guillaume Racicot
1d26946296 SERVER-108318 Introduce new error label indicating a failure is unconditionally retryable (#42840)
GitOrigin-RevId: 21ca15bf2da061b81ade7f0f7d6bd966126f9459
2025-10-21 16:48:29 +00:00
Binh Vo
07d77e5061 SERVER-95726 Time-series meta-only delete and update should respect control.closed (#42099)
GitOrigin-RevId: aaad9d3cc324e407248411e1c4420634d85eeb12
2025-10-21 16:18:53 +00:00
Chi-I Huang
02c6a29bbc SERVER-111245 Add tassert to check _execPipeline (#41852)
GitOrigin-RevId: 696208507b27a3bd882ec5261600932757a27672
2025-10-21 16:18:35 +00:00
Adi Zaimi
1331603c57 SERVER-110345 Use equality to check config collection for range deleter document (#42638)
GitOrigin-RevId: 5cc90567f32c31d3fbb9b9f8c84feedc8b3878c4
2025-10-21 16:18:17 +00:00
Saman Memaripour
e2640d348f SERVER-111131 Create a metric to track delayed FTDC collections (#41886) (#42429)
Co-authored-by: Joseph Prince <57415450+josephdprince@users.noreply.github.com>
GitOrigin-RevId: b5b98586e2826374c64081da600f100e6d75eea0
2025-10-21 16:10:07 +00:00
Daniel Moody
27cc937065 SERVER-112229 update auto header ignore on old branches (#42449)
GitOrigin-RevId: a20d7851b6213c0c46ed612759e87cf9d32c6ea2
2025-10-21 16:01:03 +00:00
Sergei B
d87902c241 SERVER-103855 Add log counter metric (#41651) (#42608)
GitOrigin-RevId: 3f88ac574b950337f69808d079a8d7319143b2c4
2025-10-21 15:56:18 +00:00
Ted Tuckman
28ebe34b8d SERVER-109937 Change skipAuthenticationToSearchIndexManagementServer to default to false (#42673)
GitOrigin-RevId: f4df2e7df04b1a8d8d03d95337dd113b1cb5f4cd
2025-10-21 15:56:00 +00:00
Erin McNulty
7cf67a345e SERVER-111914 Client-based rate limiting exemption (#42289) (#42762)
Co-authored-by: Didier Nadeau <didier.nadeau@mongodb.com>
GitOrigin-RevId: 81b11cc8a03b8962c4d3c18659cb110ce77b0dbc
2025-10-21 15:55:41 +00:00
Max Hirschhorn
427194347f SERVER-110899 New "readBackupFile" auth privilege for $_backupFile. (#42675)
GitOrigin-RevId: 61239dacc4e623ee66069fc2095f2afd92192643
2025-10-21 15:55:22 +00:00
henrikedin
195c2b231f SERVER-112520 Fix index maintenance for 2dsphere indexes on timeseries collections (#42786) (#42837)
GitOrigin-RevId: 114592f1a809a818c5ab0457be2b8990075841b4
2025-10-21 15:40:14 +00:00
wt-vendoring-bot[bot]
6be67289bb Import wiredtiger: ead72399abea7ce909fd8e2212afeff694be57e3 from branch mongodb-8.2 (#42898)
Co-authored-by: wt-vendoring-bot <wt-vendoring-bot@mongodb.com>
GitOrigin-RevId: ae92a87e92030687567c55ce1c827b8bed90689d
2025-10-21 05:34:17 +00:00
wt-vendoring-bot[bot]
e0117e92da Import wiredtiger: 88ae30810857ef059d0b8526dc1280983f66e99f from branch mongodb-8.2 (#42865)
Co-authored-by: wt-vendoring-bot <wt-vendoring-bot@mongodb.com>
GitOrigin-RevId: 981852279e853ca502f8e15d79d02919a706bc4b
2025-10-21 05:33:58 +00:00
Didier Nadeau
cc3025436f Revert "SERVER-106302 Provide visibility into the sizes of serverStatus sections if the aggregate FTDC object is too large (#42716)" (#42841)
GitOrigin-RevId: 45e645d9364f476aa85efd3343667b14d52aab00
2025-10-17 23:48:28 +00:00
Cole Harbeck
829bda19ce SERVER-106302 Provide visibility into the sizes of serverStatus sections if the aggregate FTDC object is too large (#42716)
GitOrigin-RevId: 2bce77d5edd8c9221568ee297c5fdb99c123c52f
2025-10-16 21:50:33 +00:00
Blake Oler
31b06936c6 SERVER-111139 Prepend all calls to RateLimiter's token bucket with proper mutex acquisition (#42634)
GitOrigin-RevId: a5dfb7ab0c726cdef9ee9c9ba1c1834c20c1d18c
2025-10-16 21:31:10 +00:00
Binh Vo
4418d28265 SERVER-106854 Make timeseries reopen verification function properly handle meta in different order (#42702)
Co-authored-by: Damian Wasilewicz <33820523+DamianWasilewicz@users.noreply.github.com>
GitOrigin-RevId: 2e2cc2fdca4bf2b5b63d2b27ec9817451712b075
2025-10-16 21:14:12 +00:00
Silvia Surroca
3f23b647d6 SERVER-109003 Mark write_without_shard_key_with_moveChunk.js and timeseries_list_catalog.js with 'requires_getmore' (#42737)
GitOrigin-RevId: a4b1855644dc079ab1f18a87132fe6d3c3299600
2025-10-16 12:55:54 +00:00
Janna Golden
d9814c89f0 SERVER-107535 Set up TransactionRouter state if sub-router in RouterRole::route (#40055) (#42628)
GitOrigin-RevId: 99bba1d408a6ee3a5526a6840a689a8bf6b19156
2025-10-15 19:51:46 +00:00
Moustafa Maher
221f4e012d SERVER-108366 Prepared Transactions with apiVersion (#37458) (#42637)
GitOrigin-RevId: 49152e5477b52a0a32b137514c43a1e4648f7437
2025-10-15 18:38:45 +00:00
Pierre Turin
228193a940 SERVER-110083 Remove change_stream_serverless_fuzzer evergreen tasks (#41772)
GitOrigin-RevId: f4a58e3cd8fd9c0cf64fae89e90976b5296ecb8d
2025-10-15 18:38:26 +00:00
James H
f913620192 SERVER-108002: Correct replayWorkloadRecordingFile validation (8.2) (#39376)
GitOrigin-RevId: 278ad65cd078f0f68ce16d0cdebf0196fb542a08
2025-10-15 15:16:03 +00:00
Silvia Surroca
a5eeb8939e SERVER-103774 Disallow compound wildcard indexes from being used as a shard key index (#40518) (#42275)
GitOrigin-RevId: 724234dd26a77de8da08b8e0cd24829bca6ddda4
2025-10-15 14:51:54 +00:00
Jan
7d97af9679 SERVER-111406 Fix change stream results for top-level $v fields (#42273)
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
GitOrigin-RevId: 4398df22b0422da5739f4537fc1b95c03b6a4289
2025-10-15 14:51:35 +00:00
ekovalets
4be6f775aa SERVER-110425: SBOM upload and SCA automation 8.2 (#41877)
GitOrigin-RevId: 40c0e396749b7f323644b76d4f6ad752d4773f53
2025-10-14 21:49:36 +00:00
wolfee
9b7e9473b8 SERVER-111794 Add ddlLockOptimisticRecoveryWaitTimeout parameter (#42062) (#42234)
GitOrigin-RevId: 5afb864548e8107d309aebd8e83836d580b2e150
2025-10-13 20:45:52 +00:00
Daniel Segel
f69d078407 SERVER-109677 Destroy PinnedConnectionTaskExecutor before shutting down the underlying executor (#42165)
GitOrigin-RevId: ef11d455fad2879db19c30bbfbc130a6fd12f00e
2025-10-13 19:51:46 +00:00
Naafiyan Ahmed
937e7161b8 SERVER-110916: Fix use-after-move in targetShardsAndMergeCursor() (#42382)
GitOrigin-RevId: 4b8c691dbba61f60749a181fd263026af6e8e6c9
2025-10-13 19:46:34 +00:00
Evan Bergeron
56dbb1009d SERVER-102364 Periodically check $reduce accumulated depth less than maxAllowableBSONDepth (#40108)
Co-authored-by: Mihai Andrei <mihai.andrei@mongodb.com>
GitOrigin-RevId: 51eb58a705317d12d191dff64140a5a887685a26
2025-10-13 17:52:55 +00:00
Pol Piñol Castuera
098a61f2a3 SERVER-107973 Revert "SERVER-107176 Introduce placeholder configureBackgroundTask command" (#40019)
GitOrigin-RevId: 20b90b5792ed75a1272f36cb722e3b819df5b75e
2025-10-13 16:09:12 +00:00
David Goffredo
d29e20a451 SERVER-107161: FTDC jstests: retry getDiagnosticData when fields are missing (#42477)
GitOrigin-RevId: 9ac529daad2966bfc5db36fc133b4e32ef6fb8cf
2025-10-13 14:09:49 +00:00
Tianyu Wang
9b4c631916 SERVER-108547: configure retry attempts for evergreen_activate_gen_tasks.py (#39976)
Co-authored-by: Steve McClure <steve.mcclure@mongodb.com>
GitOrigin-RevId: 079524e881dd394fe8516bad89536e9771c16d39
2025-10-13 12:57:35 +00:00
Jonas Bergler
16bdaa6b69 SERVER-109276 KVDropPendingIdentReaper::dropIdentsOlderThan should check for interrupt (#42341)
Co-authored-by: Gregory Wlodarek <gregory.wlodarek@mongodb.com>
GitOrigin-RevId: f512677cedb25c9db2f0029b53ef560d0536d1a7
2025-10-13 12:57:16 +00:00
Brad Cater
2d5f499469 SERVER-107546 Dump wt state when there are many write conflicts (#40396) to v8.2 (#42424)
Co-authored-by: Solomon Lifshits <solomon.lifshits@mongodb.com>
GitOrigin-RevId: e8f95b94c4e9600662ebdc5654234e8f06e62310
2025-10-13 12:56:57 +00:00
wt-vendoring-bot[bot]
3facd5a845 Import wiredtiger: 0164fd130bf034bb3414db97ce6f1258ecc53103 from branch mongodb-8.2 (#42541)
Co-authored-by: wt-vendoring-bot <wt-vendoring-bot@mongodb.com>
GitOrigin-RevId: d9fead707d6235197072320acdc3916d9ba837b1
2025-10-13 02:58:11 +00:00
Cheahuychou Mao
40180020c2 SERVER-109137 Fix integer overflow when calculating in 'expireAt' in QueryAnalysisWriter (#42418)
GitOrigin-RevId: 2921a8849660982529666799537ebe10122cae4b
2025-10-09 20:29:01 +00:00
Lee Maguire
9b4f468908 SERVER-110890 Remove logically dead code in module_loader (#41709) (#41916)
GitOrigin-RevId: 9d0fc9c8874ab6e661ab09ad4bd8a51e9e71eb12
2025-10-09 20:12:09 +00:00
Silvia Surroca
9e04e35005 SERVER-99623 Do not immediately retry when range deleter encounters an index not found error (#42274)
Co-authored-by: Randolph Tan <randolph@10gen.com>
GitOrigin-RevId: 3593ec907940bdb17ba6be2bda2a2701e5327049
2025-10-09 20:01:03 +00:00
Gregory Wlodarek
0048f016e8 SERVER-103807 Reset _storageExecutionTime when caching sessions (#40756) (#41373)
GitOrigin-RevId: 75c2e0fe928700fe4e52c146fb760388c1d58df5
2025-10-09 19:26:10 +00:00
Mariano Shaar
c2c5491b61 SERVER-108052 Set sorting metadata for bounded sorts [8.2] (#39265)
Co-authored-by: Lee Maguire lee.maguire@mongodb.com
GitOrigin-RevId: 6cece96d93eb2e5c0b110c2f7bfbb74820b7c47e
2025-10-08 20:41:31 +00:00
Erwin Pe
a8b0cb11c0 SERVER-107873 Report number of Queryable Encryption collections using each index type in serverStatus (#42303)
GitOrigin-RevId: e14ecda06812a4fdb7d490484c22c0ab952f51bf
2025-10-08 20:41:13 +00:00
ben-gawel
e32ffb05e4 SERVER-109541 Coverity analysis defect 149896: Data race condition (#42259)
GitOrigin-RevId: d208ecad3e32b4ef979c41d639383ffa39f76dea
2025-10-08 20:36:17 +00:00
Erwin Pe
7666e8fea6 SERVER-110716 Authenticate before issuing getParameter in commands_lib.js (#42322)
GitOrigin-RevId: a5cefca67596aebe094858876a05196edf4db56d
2025-10-08 18:36:10 +00:00
Gregory Noma
298218ea4b SERVER-110291 Add serverStatus metric for spill WT open sessions (#41092)
GitOrigin-RevId: 3fae0bdbc3ed26b5c0599d4b60701bb880014618
2025-10-07 21:57:19 +00:00
Ronald Steinke
6adf0f08a7 SERVER-111058 Lock throw_hook server parameters (#41482) (#41938)
GitOrigin-RevId: 490f481460d5daf722ffb2ec7f8774500db8bc7b
2025-10-07 21:56:59 +00:00
Matt Olma
a165502772 SERVER-105047: Query using $or and $ne on sharded collection can incl… (#41365)
GitOrigin-RevId: c3579b8c8661091b73b885baf8e7dff59257306a
2025-10-07 21:51:23 +00:00
adriangzz
3a37c48ae6 SERVER-109515 Disable oidc_e2e_azure_machine tests (#40737) (#41354)
GitOrigin-RevId: beae84e670af8bde7145a835386a9d38e229927d
2025-10-07 21:51:03 +00:00
Zack Winter
7af4a575d1 SERVER-109836 [v8.2] Add BES evergreen variables in Server that are currently present in MMS (#40565) (#40573)
GitOrigin-RevId: c75feba73a756667e1d584fc4a1effb162fb635d
2025-10-07 21:35:19 +00:00
Jason Chan
7f60803f1d SERVER-106908 Enable ingress request rate limiting by default (#40225) (#41129)
Co-authored-by: Guillaume Racicot <guillaume.racicot@mongodb.com>
GitOrigin-RevId: a64ffbc4526123a96d281ef945cdc5203674fe50
2025-10-07 21:35:00 +00:00
Carlos Alonso
33c0745c8a SERVER-109213 enhance logging for invalid bounds (#40026) (#40761)
GitOrigin-RevId: 614dc0b5fc543b3cb848a8869586a31256e6f79c
2025-10-07 21:34:41 +00:00
Wei Hu
c9a651fe6a SERVER-109355 Reconcile pins after reopening CollectionCatalog in recoverToStableTimestamp (#40852)
Co-authored-by: Gregory Noma <gregory.noma@gmail.com>
GitOrigin-RevId: 571052438bcf553471c541ac3069a1c0be439d15
2025-10-07 21:29:14 +00:00
Gabriel Marks
527bc506a2 SERVER-109033 Fix double lock in SSLManagerCoordinator::rotate() (#39… (#40321)
GitOrigin-RevId: 4fa70b9c6c8fb59249eff5f772c441192136c6ae
2025-10-07 21:22:34 +00:00
Ronald Steinke
138e6a6a1a SERVER-109493 Disable uncaught exception coverity check. (#40270) (#41269)
GitOrigin-RevId: a7feae5fdf879bf615c40fd711258fcab86d7fff
2025-10-07 21:08:22 +00:00
Henri Nikku
e70e212e13 SERVER-108945 Handle $elemMatch with empty path under a contained $or correctly (#39824) (#40915)
GitOrigin-RevId: 2c7e210d9047bcd86a6f46e0768584b8ca70faa9
2025-10-07 21:04:26 +00:00
Daniel Segel
5b2863df43 SERVER-108243 $unionWith serialization may use the wrong namespace (#40223)
GitOrigin-RevId: b03d8e4ff86b909503a2e219b4e85e8014077b47
2025-10-07 21:04:08 +00:00
Andrew Bradshaw
aacfc54b8d SERVER-108916 [v8.2] Remove unused pgo and bolt builds (#39749)
GitOrigin-RevId: 71cb9a20933958f1c27045b9379bd576ff4bc3ed
2025-10-07 21:03:49 +00:00
Zixuan
94f72ca06b SERVER-108581 Add maxTime to DeadlineState to avoid double count execution time (#39909)
GitOrigin-RevId: ad61bc40a54527145a892fe4502fa865557054f9
2025-10-07 20:57:08 +00:00
Saman Memaripour
1fe2b708a9 SERVER-108723 Synchronize before reading from FaultFacet (#39661) (#40561)
GitOrigin-RevId: 777a65940224a278cb70169e5a67573d88e94cfd
2025-10-07 20:42:50 +00:00
Wei Hu
d2abc347ac SERVER-108626 Move fileIteratorsMaxNum definition to the proper subclass (#40228)
GitOrigin-RevId: 4cc6ca18708a380fd497666a74ed110552a9faa1
2025-10-07 20:42:32 +00:00
Jordi Olivares Provencio
37ef6eff43 SERVER-108364 Allow an untimestamped write to the minvalid document to avoid timestamp violations (#40066)
GitOrigin-RevId: ab2fb25217702e2a305e9cee082299972ee99b9b
2025-10-07 20:42:13 +00:00
Yuhong Zhang
d8e609493f SERVER-95570 Avoid deleting the same document twice in BatchedDeleteStage (#39247) (#39421)
Co-authored-by: Wei Hu <wei.hu@mongodb.com>
Co-authored-by: Max Hirschhorn <max.hirschhorn@mongodb.com>
GitOrigin-RevId: 0b11ebf1954025b0b191e80e0d61dede396db84e
2025-10-07 20:41:54 +00:00
Allison Easton
781e2aa218 SERVER-107532 Ensure new Mongo shell connection creation returns informative error code (#39676)
GitOrigin-RevId: a9b11ce0df5023ea9a301ecf20273e2affc0b634
2025-10-07 20:26:47 +00:00
Dylan Richardson
f7f3d170c3 SERVER-111429: fix crypt_push dependencies on windows (#41777)
GitOrigin-RevId: 25c0ce30ec1326f95cd4f17b8f699359cda1e968
2025-10-07 20:13:42 +00:00
Zixuan
def54dbbaf SERVER-104007 Track delinquent checkForInterrupt (#41510)
GitOrigin-RevId: e7d6a5925d0939a105aee7e5914023c6fe713287
2025-10-07 19:51:45 +00:00
huayu-ouyang
da0fc3c1bc SERVER-110184 Fix command_registration_startup_behavior_fcv.js now that command registry properly works on older versions (#41058)
GitOrigin-RevId: dacf5321781d5f6da0d91f6a873ef3ddc9ae2604
2025-10-07 17:31:14 +00:00
HanaPearlman
fc0636057c SERVER-110254: Support swapping $match before "complex" renames when a flag says there are no arrays (#41196)
Co-authored-by: Andi Wang andi.wang@mongodb.com
Co-authored-by: David Storch david.storch@mongodb.com
GitOrigin-RevId: 16a355ba0654dfbc527cb301dee154ca59fc06e1
2025-10-07 16:51:05 +00:00
Gabriel Marks
12a8c902f3 SERVER-110118 Remove unneccesary xvalue function implem… (#42024)
GitOrigin-RevId: 6287aa3ea8275d13bffb849b389b0c3bd8073477
2025-10-07 16:43:49 +00:00
Ivan Fefer
a020406ab9 SERVER-107837 Replace tassert with uassert on SBE spilling failure (#42036)
GitOrigin-RevId: 99390f6f6ed39ecf4a577406f0e5480ed380bfda
2025-10-07 15:16:35 +00:00
adriangzz
fd4848d554 SERVER-108040 Update markFailed to check current step (#41221) (#41357)
GitOrigin-RevId: 6b56e1dc0a457399af37b618026fa41c0a1e75b3
2025-10-07 15:16:16 +00:00
wolfee
bb2dadf241 SERVER-101758 Adding a new local-only task executor for node local tasks (#40780) (#42121)
GitOrigin-RevId: 47e8eb91b9d556095429567ae598a11e8167a726
2025-10-07 15:15:57 +00:00
Zack Winter
eb9af3a1da SERVER-111295 [v8.2] Set python as formatter in format_multirun (#41680)
GitOrigin-RevId: e47fadfd8bfc0835d7d121df0ce2073d43f2e898
2025-10-07 14:51:00 +00:00
Saman Memaripour
3c713fd09f SERVER-111427 Load SASL before locking code pages (#41776) (#41924)
GitOrigin-RevId: ff014d545de3d405b4328fdcf34c61aa3383737a
2025-10-07 14:50:41 +00:00
Erin McNulty
6d56055c32 SERVER-105713 Allow callers to provide TickSource to RateLimiter for unit testing (#42173)
GitOrigin-RevId: 6930a411d24e6230b21edfec6f897bea396f7f5d
2025-10-07 14:50:23 +00:00
Alberto Massari
4211210a74 SERVER-110215 Add retry mechanism to test_api_version_compatibility task (#41925)
Co-authored-by: Jan <jsteemann@users.noreply.github.com>
GitOrigin-RevId: ac3a2db8e354368c84c2ce2e9f2c476c3b816cdd
2025-10-07 14:50:05 +00:00
Sean Lyons
277257c1ed SERVER-110480 Use large distro for s390x/PPC64LE core analysis tasks (#41945)
GitOrigin-RevId: 07aa4cebfd4897bc676d2ba18008f00f8906fa89
2025-10-07 14:49:46 +00:00
Gil Alon
5c24c5da7c SERVER-109490 Use unique collection names in hybrid search integration tests (#40267) (#42229)
GitOrigin-RevId: a5696a48a6032bf683920b7c20f7abab0aedd270
2025-10-06 16:20:24 +00:00
Pol Piñol Castuera
afcaf66903 SERVER-109299 Allow transient errors for update operations after a WouldChangeOwningShard error in refine_collection_shard_key_crud_ops.js (#42163)
GitOrigin-RevId: 04b36f208f742eace1f1a17fb4c202003c8712df
2025-10-06 08:36:58 +00:00
Erin McNulty
4c2cee8153 SERVER-111349 Rewrite RateLimitIsValidAfterQueueing unit test (#42011)
GitOrigin-RevId: e761053b2375e0fe04e349ce568d00cd5b0ce694
2025-10-02 19:19:52 +00:00
Erin McNulty
4c99b3f706 SERVER-108016 Update additional assertions in RateLimitIsValidAfterQueueing for slow machines (#41706)
GitOrigin-RevId: c652ff2f7d30f124524166df70b130ad17f033fd
2025-09-29 21:47:31 +00:00
Andrew Bui
d86f99ed7e SERVER-111469 Exempt coverity only changes from Server Release gate (#41825)
GitOrigin-RevId: 32bc55365e2057f1babbbefa302ef30a6445a827
2025-09-29 21:47:12 +00:00
Marcos Grillo
a3edac5a71 SERVER-106162 Remove writeConcern check from fsm ddl response (#40909) (#41867)
GitOrigin-RevId: d6827204b28630fbbfd41bba6595e7802ade5307
2025-09-29 16:17:22 +00:00
Vishnu K
f1496796d8 SERVER-111135 retry configureFailPoint in setFastGetMoreEnabled in rollback_test.js (#41760)
GitOrigin-RevId: bfab7bf68e3bbbeecdae669777661ca1c51c4247
2025-09-25 20:53:53 +00:00
Ben Shteinfeld
722d015456 SERVER-105873 Ensure lifetime of catalog-owned pointers stored in QueryPlannerParams (#40502) (#40876)
GitOrigin-RevId: 3312bdcf28aa65f5930005e21c2cb130f648b8c3
2025-09-24 19:28:30 +00:00
Erin McNulty
2e536df639 SERVER-107924 Make RateLimitIsValidAfterQueueing test resilient to slow machines (#41656)
GitOrigin-RevId: 1ac25655a8d9a1a5e3a219955b8e23e513b6a0bb
2025-09-22 19:24:38 +00:00
Evan Fetsko
a082c84cc9 SERVER-111140: use authenticated Jira client for SAST report generation (#41583) (#41599)
GitOrigin-RevId: 86121a818763e5dc07ba7ab651ad977d65f78bf0
2025-09-22 19:09:13 +00:00
kmznam
f0b5ab5e38 SERVER-110161 SERVER-109980 correct $_startAt behavior with deleted record id (#40994)
GitOrigin-RevId: b475651641b473f53fb6fd14925732592ff63ab7
2025-09-22 17:22:38 +00:00
Alyssa Clark
aaab85f098 SERVER-109913 Add query knob to override FCV-gating for rankFusion features (#41256)
GitOrigin-RevId: 6a750be286793541c31a2f74bbb25b329ba1db87
2025-09-19 14:46:26 +00:00
Gregory Noma
e72a10d4b8 SERVER-110250 Use existing session when dropping spill WT ident (#41091)
GitOrigin-RevId: d2647aa33002a03f942dee9fa3a9a3e6ba71523e
2025-09-18 20:41:11 +00:00
Didier Nadeau
9a81256d9c SERVER-106469 Handle null device for logpath on Windows (#41284) (#41346)
GitOrigin-RevId: b63f5d77b45487f5920850565b4967b8f1e0b4b0
2025-09-18 17:30:47 +00:00
Madison Hoover
2191f30bf4 SERVER-107074: Update man pages for 8.2 (#39851) (#40489)
GitOrigin-RevId: 3b4d4cd53e35a5f85dd82c8bc9e9ee0b150c79ec
2025-09-18 16:54:23 +00:00
Matt Kneiser
68b52d3865 SERVER-108565 Check bucket size before writing for ordered time-series inserts (#40339)
GitOrigin-RevId: 540c149276d01fc1a0bf843c00e3e0e38df0d46b
2025-09-18 16:54:04 +00:00
Drew Paroski
07bfb33e54 SERVER-109309 Fix multi-planning issue when cached plan has decisionReads=0 (#40191)
GitOrigin-RevId: d7f2b5c545c81a6d71a9b37a9926dd604fd349b9
2025-09-18 16:13:12 +00:00
Evan Fetsko
d3f99007aa SERVER-110489: use github_commit expansion and cd into 10gen/mongo to get commit datetime for SAST report generation (#41108)
GitOrigin-RevId: c4c66f27ecf7d99c88947b5de0642cdc2db6af58
2025-09-18 16:12:54 +00:00
Cheahuychou Mao
91f7205349 SERVER-109323 Disable featureFlagReshardingSkipCloningAndApplyingIfApplicable (#40129)
GitOrigin-RevId: df494dce92f69ffee2544e7535648e5f62d3db6f
2025-09-18 16:12:34 +00:00
Daniel Moody
21e335fe3e SERVER-108544 update to latest coverity version (#40738) (#40816)
GitOrigin-RevId: 86fdc8791640c67039c4f73d0d11575a35f0ecd6
2025-09-17 17:34:48 +00:00
wt-vendoring-bot[bot]
627d3cecf7 Import wiredtiger: a300655f291261468d87ed8461494be5c78ddcdb from branch mongodb-8.2 (#41375)
Co-authored-by: wt-vendoring-bot <wt-vendoring-bot@mongodb.com>
GitOrigin-RevId: de69aae102b2c44e50e93773ed3f801322fca93b
2025-09-16 23:15:17 +00:00
Jan
a402b4d130 SERVER-109626 Adjust behavior for emitting additional fields in change streams (#41310)
GitOrigin-RevId: 756c683a6f52a6b304843b322308c20873e6d25a
2025-09-16 16:07:12 +00:00
clarissecheah
fa6bd7d221 SERVER-101672 Remove async sampling feature flag (#41318)
Co-authored-by: Gregory Wlodarek <gregory.wlodarek@mongodb.com>
GitOrigin-RevId: c0b6e4868aae177a72b5354b9b62eb162eec38bf
2025-09-16 15:35:24 +00:00
Clarisse Cheah
1bc69b035c SERVER-101672 Remove async sampling feature flag
GitOrigin-RevId: 806be28349f08716ee1bbd3f3b063bcd16793cb7
2025-09-16 15:35:06 +00:00
clarissecheah
1b098d48d8 SERVER-110649 Move maintainer thread to start/stop storage controls (#41152)
Co-authored-by: Gregory Wlodarek <gregory.wlodarek@mongodb.com>
GitOrigin-RevId: 5b4df9850057bcca0bf0a66783847d4b0df6657a
2025-09-16 15:34:48 +00:00
adelinexchen
f1ef2c7dae SERVER-106211 Adding metrics to oplog sampling (#38137)
Co-authored-by: Clarisse Cheah <clarisse.cheah@mongodb.com>
GitOrigin-RevId: d6cda34bbbebab5982898cc989ce67f77b9949ce
2025-09-16 15:34:29 +00:00
clarissecheah
85f50fb465 SERVER-101673 Documentation Updates (#40293)
GitOrigin-RevId: a1e51012d36abf04ac231cbcb95938b7242ff2e4
2025-09-16 15:34:11 +00:00
clarissecheah
ed6a5c217e SERVER-109207 Fix bug in oplog cap maintainer thread run (#40001)
GitOrigin-RevId: b19533124bb495cfd0a6b54edf392cf3e5e8fc72
2025-09-16 15:33:53 +00:00
clarissecheah
05c03f3e8a SERVER-94681 Move sampling to the oplog cap maintainer thread (#38511)
GitOrigin-RevId: eaabe67799a0c96d4ea1f8d5622d07f1e3589a4c
2025-09-16 15:33:34 +00:00
adelinexchen
4751478dab SERVER-107100 add var to track sampling progress (#38735)
GitOrigin-RevId: 451be43e97dbdc49d48393a651f0e0b2effccdbb
2025-09-16 15:33:16 +00:00
adelinexchen
e83005b5d4 SERVER-106210 add server parameter to turn on async oplog (#38510)
GitOrigin-RevId: 0a4a835d3955a6c7e6d903a516bfc2a66fbdafcb
2025-09-16 15:32:58 +00:00
adelinexchen
57257d9ad6 SERVER-106315 Functional correctness testing of async oplog sampling (#38457)
Co-authored-by: Clarisse Cheah <clarisse.cheah@mongodb.com>
GitOrigin-RevId: e79b4852fba0230d9b1604d7f38afbe6a4464f60
2025-09-16 15:32:39 +00:00
Ian Boros
ca8a87ef77 SERVER-110378 Increase maxTimeMS in flaky explain_shell_helpers.js test (#41180)
Co-authored-by: Jan <jsteemann@users.noreply.github.com>
GitOrigin-RevId: fc44ebfac4a585078e1c67ecaee2090684453de0
2025-09-12 16:22:15 +00:00
Dylan Richardson
c1064af11a SERVER-110362: modify push task dependencies on non-windows variants (#40956)
GitOrigin-RevId: cc019f91378bbc1f492f7b08f61023635e950baa
2025-09-12 15:52:20 +00:00
Zack Winter
78711fa1b6 SERVER-108219 [v8.2] Increase size of the Amazon 2023 ARM compile (#39165) (#41059)
GitOrigin-RevId: 0de9e8a2772b31f89d6c22ec85e540cdbb0890c4
2025-09-09 19:59:54 +00:00
Jason Hills
4e07a389ae SERVER-107852 SBOM rebuild to v8.2 (#39929)
GitOrigin-RevId: aec26739ea31c57e8d56d4b0f23f61ad339c6ae6
2025-09-08 14:21:36 +00:00
Tommaso Tocci
c3f463cc86 SERVER-105424 Handle yield and restore in direct shard connection warnings (#40763)
GitOrigin-RevId: dc58dd228457d0a21db54a574dbea3d10f0a5a47
2025-09-03 10:07:11 +00:00
Binh Vo
c9876d0fa9 SERVER-107721 Use majority read concern and disable migrations during multi updates in timeseries_deletes_and_inserts.js (#40082)
GitOrigin-RevId: edb12a1109b62185e9f8ddadde5fc0337da97110
2025-09-02 18:46:21 +00:00
Alyssa Clark
de9fef5886 SERVER-109892 Update expected error codes for multiversion rank fusion tests (#40662)
GitOrigin-RevId: 96154a0bc1183a20eff98525bbbc8edf7d337c6a
2025-08-29 14:26:10 +00:00
Shin Yee Tan
faf12e9eae SERVER-107929 Allows duplicate key write error when inserting to bucket in timeseries_crud_operations_respect_control_closed.js (#38986) (#40102)
GitOrigin-RevId: c280aacab9b435dd461ef3978ce65b30d39a5cfb
2025-08-21 17:14:37 +00:00
Gabriel Marks
9c4b726869 SERVER-108896 Split mixed_mode_sharded_transition to prevent timeouts… (#40338)
GitOrigin-RevId: 2ee36935288dc2a2f8985305981989f88e70ef5e
2025-08-20 22:14:59 +00:00
Britt Snyman
b993867dce SERVER-107081 Bump commit for 8.2.0 GA (#40134)
GitOrigin-RevId: 13e629eeccd63f00d17568fc4c12b7530fa34b54
2025-08-15 21:51:47 +00:00
Andrew Bradshaw
12dc03761c SERVER-108960 [8.2] Fix missing git commit in version info for windows mongod (#39806)
GitOrigin-RevId: 7939a34a7480a8d40a4e31087de26d3a9526a505
2025-08-08 18:35:53 +00:00
Lynne Wang
2d43d9c8df SERVER-108959: Revert SERVER-104422 Support a "searchRootDocumentId" meta variable from mongot (#39793)
GitOrigin-RevId: 39c98613306d2abd3ee31605f4ac27f981c6cfab
2025-08-08 16:03:07 +00:00
Daniel Moody
6ee0dcb9a0 SERVER-108778 Fix coverity task to use evergreen MONGO_VERSION (#39613) (#39720)
GitOrigin-RevId: 94f342c6662f71276ab428b3fcf8e3cc1ac9c698
2025-08-06 22:18:09 +00:00
Binh Vo
65be29203b SERVER-108909 Readd jstests for mixed schema fix (#39732)
GitOrigin-RevId: 98494390a6fc09c85e216417054ce7f893168f17
2025-08-06 21:38:37 +00:00
Will Buerger
f0cb4751ff SERVER-108870: "searchScore"/"vectorSearchScore" warning log should only be logged when invalid usage (#39703)
GitOrigin-RevId: 2eccff6039929394453ffca59f30a6d1cba738ba
2025-08-06 17:46:05 +00:00
Enrico Golfieri
8f2c658842 Revert "SERVER-105921 Replace usage of AutoGetCollectionForRead under the db/commands path (#37170)" (#39649)
GitOrigin-RevId: 2a02e38ade3bdaab26dea534f46e2a2b8cac5afe
2025-08-05 17:12:29 +00:00
Jan
b0f36602de SERVER-108341 Add missing detach/reattach calls to DocumentSourceLookup::_resolvedIntrospectionPipeline (#39280) (#39349)
GitOrigin-RevId: e0cd9fbdaef2db5f2aaf73715e48ec8ff82589fe
2025-08-05 15:35:23 +00:00
Andrew Bradshaw
a53de12948 SERVER-108655 Use our own architectures for naming the top level folder in release packages (#39602)
Co-authored-by: Daniel Moody <dmoody256@gmail.com>
GitOrigin-RevId: d7644705d817516b2db60baca94af29223281a09
2025-08-05 04:50:49 +00:00
Binh Vo
0be07af779 SERVER-107361 Fix mixed schema detection in rollover determination (#39471)
GitOrigin-RevId: 1213e08050f437234c6c53e8eec4b1ada71ddb52
2025-08-04 21:41:53 +00:00
Evan Fetsko
64b6ddd767 SERVER-108449: remove allowed_requesters from task definition for publish-sast-report (#39416)
GitOrigin-RevId: 74016702f07bb118d4d78f0a86c5ffe28d8f340d
2025-08-04 13:43:16 +00:00
Will Buerger
67dc54bb93 SERVER-108468: Unavailable searchScore and vectorSearchScore log a warning instead of throwing an exception (#39476)
GitOrigin-RevId: 891e030c83dbfced2fd75b16a656df2366ae05fa
2025-08-01 20:18:27 +00:00
Sean Lyons
b517e4c8a6 SERVER-108053 Match task names by regex in task timeout overrides (#39030)
GitOrigin-RevId: a56684818e6e92d20185d9cbbf7b7e5624c3a51a
2025-08-01 13:39:33 +00:00
joe-mongodb
675d0b61d2 SERVER-82020: BACKPORT-25616 [v8.2]: Enable hybrid search feature flag and upgrade/downgrade testing (#39468)
Co-authored-by: Finley Lau <finley.lau@mongodb.com>
Co-authored-by: Mariano Shaar <mariano.shaar@mongodb.com>
GitOrigin-RevId: 6e5c812d221048df48014948cc841e034a4873fc
2025-07-31 21:06:13 +00:00
Austin
8387f4eb6a SERVER-107913 Revert "SERVER-104988 Use a container for jstestfuzz (#36006)" (#39295)
GitOrigin-RevId: 40b98acc49c7b4f85cebda294543c70e1dcba5c1
2025-07-31 18:07:07 +00:00
Gabriel Marks
df068ab24f SERVER-108185 Assert encryptedTokens is the correct length for text s… (#39240)
GitOrigin-RevId: b00a4e468eccb33b841278e192b9d823804db2de
2025-07-31 16:21:32 +00:00
Britt Snyman
5cc52b5a90 SERVER-108509 Add ibm variants to evergreen yml so they are included in 8.2 releases (#39444)
GitOrigin-RevId: a1c9d09490069cb2f213f1d98d96e2126e565339
2025-07-31 16:21:15 +00:00
adam r
9bd8c9267d SERVER-107880 Update OpenSSL 3 FIPS Module Initialization (#39346)
GitOrigin-RevId: cae818bb3d719a9281b4a94e1fd25710968702ca
2025-07-31 13:40:26 +00:00
Erwin Pe
213fce2e3e SERVER-108051 Adjust FLE2 substringPreview per-field max length limit (#39322)
GitOrigin-RevId: 0417aa53f9bcada88661221e8168508db89112e9
2025-07-31 13:40:07 +00:00
Joshua Siegel
4909b2b67b SERVER-107803 Add view validation to search queries (#39127) (#39330)
GitOrigin-RevId: 2ecc9d661f7628756ba2f695b6a6137f261b5882
2025-07-30 20:08:50 +00:00
Finley Lau
91e5dcf79c SERVER-107879: BACKPORT-25612: [v8.2] Best-effort ban hybrid search on timeseries collections inside of $lookup and $unionWiths (#39303)
GitOrigin-RevId: d1a6b73fdd6d8a880b2cfe4f2ccba5e1f189f156
2025-07-30 18:05:21 +00:00
Adithi Raghavan
812035b596 SERVER-105543: BACKPORT-25609: [v8.2] Strengthen rankFusion scoreDetails testing (#39319)
GitOrigin-RevId: 3a9d05edd8548be468cee1fadf033c9ab8c30d10
2025-07-29 22:16:18 +00:00
Ruchitha Rajaghatta
8a3d441ee9 SERVER-107962: Enable oplogApplicationEnforcesSteadyStateConstraints=true in resmoke based test fixtures (#39049)
GitOrigin-RevId: 7b91a7c3a2d4b890d5586719cb9c9be88306b17d
2025-07-29 20:41:55 +00:00
Joshua Siegel
0423879028 SERVER-107265 Add FCV condition to internalVectorSearchStoredSource (#38265) (#39294)
GitOrigin-RevId: 32eb053de665b4d6fef14f7013919bd596f6aa53
2025-07-29 20:36:53 +00:00
Daniel Hill
ef178b3ce5 SERVER-108106 expanded symbols for 8.2 (#39239)
Co-authored-by: jawwad.asghar <jawwad.asghar@mongodb.com>
GitOrigin-RevId: 95b80c02c080e64cac25d06b8960dfe78f03bf48
2025-07-29 20:36:35 +00:00
Adithi Raghavan
dc5795b510 SERVER-104725: BACKPORT-25617: [v8.2] Enable scoreFusion on views (#39262)
Co-authored-by: Joe Shalabi <joseph.shalabi@mongodb.com>
GitOrigin-RevId: 4ac7d5e3cedf454ec15a09cdb53a64f9934aaa66
2025-07-29 20:36:17 +00:00
Adithi Raghavan
04b67683bf SERVER-107586: BACKPORT-25611: [v8.2] Correct rankFusion.scoreDetails when no documents output (#39259)
GitOrigin-RevId: 0e6172c7f5dd290cf9599ea6c822e6bd1df514a1
2025-07-29 20:35:59 +00:00
Santiago Roche
e033c93689 SERVER-108179 Convert encrypted text predicates in '$expr' to a match expression tag disjunction in FLE rewrite (#39145) (#39243)
GitOrigin-RevId: e2098525189e85fbc4945fb10073e509d3fe5d56
2025-07-29 20:35:41 +00:00
Adithi Raghavan
a1d1cbbc54 SERVER-107881: BACKPORT-25613: [v8.2] Always perform LPP validation in ClusterAggregate::runAggregate(...) (#39224)
GitOrigin-RevId: 19a039191df510a699ca18fb6dc2c0d7658d6174
2025-07-29 20:35:22 +00:00
Erwin Pe
ec9b18ec4e SERVER-107730 Move FLE2 tag limits check in createCollection under FCV check (#39210)
GitOrigin-RevId: 743d5f5e32793390a0821a12391b56e2510aecc0
2025-07-29 20:35:04 +00:00
Finley Lau
deec3224eb SERVER-104736: BACKPORT-25614: [v8.2] Enable referencing collection level variables in $scoreFusion expression combination option without 'docs.' prefix (#39125)
Co-authored-by: Joe Shalabi <joseph.shalabi@mongodb.com>
GitOrigin-RevId: ce98b9d3edec9ab88202d1acf264689629be90cd
2025-07-29 20:34:46 +00:00
Louis Williams
d1722270a9 SERVER-108257 Lower WiredTiger max sessions for the spill engine (#39218)
GitOrigin-RevId: 52a3cc06b3e76d09a7392e40282b481988965c87
2025-07-29 20:34:27 +00:00
Mariano Shaar
47bd14d734 SERVER-100360: BACKPORT-25615 [v8.2] Add test for $rankFusion with $sample (#39230)
Co-authored-by: Adithi Raghavan adithi.raghavan@mongodb.com
GitOrigin-RevId: 27f1bd87b526f926afbb07052820fb2c432730da
2025-07-29 20:34:09 +00:00
Mariano Shaar
34eee63cd5 SERVER-105639: BACKPORT-25615 [v8.2] Verify proper behavior when having multiple scoring stages inside $scoreFusion (#39223)
GitOrigin-RevId: 9242e9d2156b6eab79bd782db01761d54777479d
2025-07-29 20:33:51 +00:00
Daniel Moody
db2e25a73a SERVER-107933 fix download.py on windows (#38901) (#39084)
GitOrigin-RevId: 1f29e96cb2fc8c9d3937cacecd81e6f670c9d7ae
2025-07-29 20:33:33 +00:00
Alya Carina Berciu
01575d0911 SERVER-107804 Add FLE-only $in $expr rewrite for 2nd param field (#39107)
Co-authored-by: Santiago Roche <santiago.roche@mongodb.com>
GitOrigin-RevId: e443b1c4a6e813300e50a6a9e62a05781aa07d74
2025-07-29 20:33:15 +00:00
wt-vendoring-bot[bot]
4f200e4524 Import wiredtiger: 70797c57f32d1b30410d062aeecd8fd5982b3848 from branch mongodb-8.2 (#39097)
Co-authored-by: wt-vendoring-bot <wt-vendoring-bot@mongodb.com>
Co-authored-by: Will Korteland <will.korteland@mongodb.com>
GitOrigin-RevId: c784162c0979805dfcb2d901b1595c74d51ec45b
2025-07-29 20:32:56 +00:00
Gregory Noma
a5fd97e029 SERVER-106596 Truncate spill table before drop (#39071)
GitOrigin-RevId: 622dec41f582b708cfd068c0323aca9ee7ad9606
2025-07-29 20:32:37 +00:00
Gregory Noma
e409c348a1 SERVER-107733 Create spill WT instance when using in-memory engine (#39028)
GitOrigin-RevId: 5780aee387b466553ad7e690bf09d87169484280
2025-07-29 20:32:19 +00:00
Gregory Noma
8cc63860eb SERVER-107753 Prevent snapshot being held across SpillTable APIs (#39027)
GitOrigin-RevId: 92842d7af8939fef35f588904732607b5aa3078a
2025-07-29 20:32:01 +00:00
Dylan Richardson
97e19d9bc4 SERVER-108027: adjust packaging script for 8.2 (#38964) (#38971)
GitOrigin-RevId: 59cf6aaffa7a87b52c8f1a20cc3706f2c3ec6e0d
2025-07-29 20:31:42 +00:00
Zack Winter
5b527bfc7b SERVER-107089 Add server-release as a codeowner on the v8.2 branch (#38924)
GitOrigin-RevId: 5831cb648764e9f6aa86adfbfa0356e36f37b789
2025-07-29 20:31:24 +00:00
Steve McClure
7a1b05c781 SERVER-107073: 8.2 Build and Verification Branching Task (#38920)
GitOrigin-RevId: e1ed2e7bda6f8a7b926a672c3115671c8cc12451
2025-07-29 20:31:05 +00:00
1525 changed files with 95610 additions and 62805 deletions

View File

@ -5,6 +5,10 @@ src/third_party/protobuf/dist
src/third_party/re2/dist
src/third_party/tcmalloc/dist
src/third_party/wiredtiger/dist
bazel/auto_header/.auto_header
src/mongo/db/modules/atlas/.auto_header
src/mongo/db/modules/enterprise/.auto_header
.git
# Ignore node_modules due to the following error
# ERROR: in verify_node_modules_ignored:

View File

@ -88,6 +88,8 @@ common:macos --repo_env=LLVM_VERSION=19
# Pin down the Microsoft Visual compiler. If you would like to use
# the default compiler version installed in this host, comment the line.
common:windows --repo_env=BAZEL_VS="C:/Program Files/Microsoft Visual Studio/2022/Professional"
common:windows --repo_env=BAZEL_VC="C:/Program Files/Microsoft Visual Studio/2022/Professional/VC"
common:windows --repo_env=BAZEL_VC_FULL_VERSION=14.31.31103
# Default the Visual C Redistribution to v14.3 for Windows installer.
@ -505,9 +507,6 @@ common:fission --remote_download_regex=.*\.dwo$
# Avoid failing builds when BES metadata fails to upload.
common --bes_upload_mode=fully_async
# Default Mongo Version if a version is not specified.
common --define=MONGO_VERSION=8.2.0-alpha
# Default distmod if not specified.
common --define=MONGO_DISTMOD=""
@ -523,6 +522,9 @@ try-import %workspace%/.bazelrc.evergreen
# local default dev settings
try-import %workspace%/.bazelrc.common_bes
# local default dev settings
try-import %workspace%/.bazelrc.mongo_variables
# local git version info
try-import %workspace%/.bazelrc.git

1
.gitattributes vendored
View File

@ -6,6 +6,7 @@
external rules-lint-ignored=true
**/*.tpl.h rules-lint-ignored=true
**/*.tpl.cpp rules-lint-ignored=true
rpm/*.spec rules-lint-ignored=true
src/mongo/bson/column/bson_column_compressed_data.inl rules-lint-ignored=true
*.idl linguist-language=yaml

3219
.github/CODEOWNERS vendored

File diff suppressed because it is too large Load Diff

35
.gitignore vendored
View File

@ -1,6 +1,7 @@
/build
/src/mongo/db/modules/*
/src/mongo/db/modules/*/
!/src/mongo/db/modules/enterprise/
!/src/mongo/db/modules/atlas/
/.jsdbshell
/.cache
/.dbshell
@ -11,7 +12,10 @@
/perf.data
/perf.data.old
/massif.out.*
/merged_decls.json
/modules_dump.yaml
/.tmp
!/.tmp/_placeholder_
venv
*~
@ -95,6 +99,7 @@ scratch
/mongoshim*
/mongosniff*
/mongotrafficreader*
/bin
# artifacts from db-contrib-tool
/ksdecode*
@ -116,6 +121,9 @@ scratch
/optimizer_gdb_test_program-*
/pretty_printer_test-*.py
/pretty_printer_test_program-*
/pretty_printer_test_launcher*
/install_compass*
/multiversion-config.yml
*.tgz
*.zip
@ -224,6 +232,7 @@ selected_tests_config
codereview.rc
# Python venvs and virtualenvs
python-venv
python3-venv
python2-venv
@ -259,7 +268,7 @@ all_feature_flags.txt
# generated by clang-tidy buildscripts
clang_tidy_fixes
.clang-tidy
/.clang-tidy
#SCons runtime configuration
scons_env.env
@ -280,10 +289,6 @@ package-lock.json
# jstestfuzz generated test directory
/jstestfuzz/
# docker volumes mapped in lint_fuzzer_sanity_patch.py
/jstestfuzzinput
/jstestfuzzoutput
# Bazel-related work
# (Note that each string does NOT terminate with a '/' so that gitignore works properly)
bazel-*
@ -295,11 +300,16 @@ buildozer
.bazelrc.gitinfo
.bazelrc.workstation
.bazelrc.common_bes
.bazelrc.mongo_variables
.bazelrc.mongo_version
.bazelrc.compiledb
.bazelrc.sync
.bazelrc.wrapper_hook
.bazelrc.engflow_creds
.compiledb
.bazelrc.xcode
.bazelrc.bazelisk
.bazelrc.exec_log_file
*.bazel_info_for_ninja.txt
.ninja_last_command_line_targets.txt
bazel/coverity/analysis/BUILD.bazel
@ -308,8 +318,10 @@ src/mongo/db/modules/enterprise/autogenerated_targets/BUILD.bazel
.bazel_header_list_cache
.bazel_real
.mongo_checks_module_path
MODULE.bazel
MODULE.bazel.lock
**/MODULE.bazel.lock
!/MODULE.bazel.lock
.auto_header
# generated configs for external fixture suites
docker_compose/
@ -319,6 +331,7 @@ buildscripts/antithesis/base_images/mongo_binaries/bin
buildscripts/antithesis/base_images/mongo_binaries/lib
buildscripts/antithesis/base_images/mongo_binaries/libvoidstar.so
buildscripts/antithesis/base_images/mongo_binaries/tsan.suppressions
buildscripts/antithesis/base_images/mongo_binaries/src
buildscripts/antithesis/base_images/workload/bin
buildscripts/antithesis/base_images/workload/lib
@ -341,3 +354,9 @@ etc/trimmed_system_perf.yml
/version_expansions.yml
/engflow.*
/.bazelrc.evergreen
# Generated Typescript type declarations from IDLs
src/mongo/shell/*_gen.d.ts
# devcontainer uses pnpm which results in a large amount of environment files that aren't intended for git commits
.pnpm-store

View File

@ -40,6 +40,9 @@ version_expansions.yml
# Ignore all formatting in third_party/*
src/third_party
# this file is automatically generated and conforms to formatting requirements
README.third_party.md
# Ignore anything in the build output directories
build
bazel-*

View File

@ -220,13 +220,13 @@ mongo_install(
"//conditions:default": ["//src/mongo/db:mongod"],
}),
package_extract_name = select({
"//bazel/config:build_enterprise_linux_enabled": "mongodb-linux-{TARGET_CPU}-enterprise-{MONGO_DISTMOD}-{MONGO_VERSION}",
"//bazel/config:build_enterprise_linux_disabled": "mongodb-linux-{TARGET_CPU}-{MONGO_DISTMOD}-{MONGO_VERSION}",
"//bazel/config:build_enterprise_windows_enabled": "mongodb-win32-{TARGET_CPU}-enterprise-{MONGO_DISTMOD}-{MONGO_VERSION}",
"//bazel/config:build_enterprise_windows_disabled": "mongodb-win32-{TARGET_CPU}-{MONGO_DISTMOD}-{MONGO_VERSION}",
"//bazel/config:build_enterprise_mac_enabled": "mongodb-macos-{TARGET_CPU}-enterprise-{MONGO_DISTMOD}-{MONGO_VERSION}",
"//bazel/config:build_enterprise_mac_disabled": "mongodb-macos-{TARGET_CPU}-{MONGO_DISTMOD}-{MONGO_VERSION}",
"//conditions:default": "mongodb-{TARGET_CPU}-{MONGO_DISTMOD}-{MONGO_VERSION}",
"//bazel/config:build_enterprise_linux_enabled": "mongodb-linux-{MONGO_ARCH}-enterprise-{MONGO_DISTMOD}-{MONGO_VERSION}",
"//bazel/config:build_enterprise_linux_disabled": "mongodb-linux-{MONGO_ARCH}-{MONGO_DISTMOD}-{MONGO_VERSION}",
"//bazel/config:build_enterprise_windows_enabled": "mongodb-win32-{MONGO_ARCH}-enterprise-{MONGO_DISTMOD}-{MONGO_VERSION}",
"//bazel/config:build_enterprise_windows_disabled": "mongodb-win32-{MONGO_ARCH}-{MONGO_DISTMOD}-{MONGO_VERSION}",
"//bazel/config:build_enterprise_mac_enabled": "mongodb-macos-{MONGO_ARCH}-enterprise-{MONGO_DISTMOD}-{MONGO_VERSION}",
"//bazel/config:build_enterprise_mac_disabled": "mongodb-macos-{MONGO_ARCH}-{MONGO_DISTMOD}-{MONGO_VERSION}",
"//conditions:default": "mongodb-{MONGO_ARCH}-{MONGO_DISTMOD}-{MONGO_VERSION}",
}),
publish_debug_in_stripped = select({
"@platforms//os:windows": True,

1027
MODULE.bazel.lock generated Normal file

File diff suppressed because it is too large Load Diff

View File

@ -68,9 +68,12 @@ filters:
approvers:
- 10gen/devprod-correctness
- 10gen/devprod-build
- "README.third_party.md":
approvers:
- 10gen/code-review-team-ssdlc
- "sbom.json":
approvers:
- 10gen/server-security
- 10gen/code-review-team-ssdlc
- "MODULE.bazel*":
approvers:
- 10gen/devprod-build

View File

@ -1,6 +1,6 @@
# ![Logo](docs/leaf.svg) MongoDB README
Welcome to MongoDB!
Welcome to MongoDB 8.2!
## Components

View File

@ -21,132 +21,101 @@ not authored by MongoDB, and has a license which requires reproduction,
a notice will be included in
`THIRD-PARTY-NOTICES`.
| Name | License | Vendored Version | Emits persisted data | Distributed in Release Binaries |
| ---------------------------------------------------- | --------------------------------------------------------------------------------------------------- | ---------------------------------------- | -------------------- | ------------------------------- |
| [Abseil] | Apache-2.0 | 20250512.1 | | ✗ |
| [arximboldi/immer] | BSL-1.0 | Unknown | | ✗ |
| [Asio C++ Library] | BSL-1.0 | 1.12.2 | | ✗ |
| [aws-sdk - the AWS SDK client library] | Apache-2.0 | 1.11.471 | | ✗ |
| [benchmark] | Apache-2.0 | v1.5.2 | | |
| [Boost C++ Libraries - boost] | BSL-1.0 | 1.88.0 | | ✗ |
| [c-ares] | MIT | 1.27.0 | | ✗ |
| [concurrencytest] | GPL-3.0-or-later | 0.1.2 | unknown | |
| [Cyrus SASL] | BSD-Attribution-HPND-disclaimer | 2.1.28 | unknown | |
| [dcleblanc/SafeInt] | MIT | 3.0.26 | | ✗ |
| [derickr/timelib] | MIT | 2022.13 | | ✗ |
| [discover] | BSD-3-Clause | 0.4.0 | unknown | |
| [fmtlib/fmt] | MIT | 11.1.3 | | ✗ |
| [folly] | Apache-2.0 | v2025.04.21.00 | | ✗ |
| [google-re2] | BSD-3-Clause | 2023-11-01 | | ✗ |
| [google-snappy] | BSD-3-Clause | 1.1.10 | ✗ | ✗ |
| [google/s2geometry] | Apache-2.0 | Unknown | ✗ | ✗ |
| [gperftools] | BSD-3-Clause | 2.9.1 | | ✗ |
| [grpc] | Apache-2.0 | 1.59.5 | | ✗ |
| [ICU for C/C++ (ICU4C)] | BSD-3-Clause, MIT v2 with Ad Clause License, Public Domain, BSD-2-Clause | 57.1 | ✗ | ✗ |
| [Intel Decimal Floating-Point Math Library] | BSD-3-Clause | v2.0 U1 | | ✗ |
| [jbeder/yaml-cpp] | MIT | 0.6.3 | | ✗ |
| [JSON-Schema-Test-Suite] | Unknown License | Unknown | | |
| [libmongocrypt] | Apache-2.0 | 1.14.0 | ✗ | ✗ |
| [librdkafka - the Apache Kafka C/C++ client library] | BSD-3-Clause, Xmlproc License, ISC, MIT, Public Domain, Zlib, BSD-2-Clause, Andreas Stolcke License | 2.0.2 | | ✗ |
| [LibTomCrypt] | WTFPL, Public Domain | 1.18.2 | ✗ | ✗ |
| [libunwind/libunwind] | MIT | v1.8.1 | | ✗ |
| [linenoise] | BSD-2-Clause | Unknown | | ✗ |
| [MongoDB C Driver] | Apache-2.0 | 1.28.1 | ✗ | ✗ |
| [Mozilla Firefox] | MPL-2.0 | 128.11.0esr | unknown | ✗ |
| [nlohmann-json] | MIT | 3.11.3 | ✗ | |
| [nlohmann.json.decomposed] | MIT | 3.10.5 | unknown | |
| [node] | ISC | 22.1.0 | unknown | |
| [ocspbuilder] | MIT | 0.10.2 | | |
| [ocspresponder] | Apache-2.0 | 0.5.0 | | |
| [opentelemetry-cpp] | Apache-2.0 | 1.17 | ✗ | |
| [opentelemetry-proto] | Apache-2.0 | 1.3.2 | ✗ | |
| [PCRE2] | BSD-3-Clause, Public Domain | 10.40 | | ✗ |
| [Protobuf] | BSD-3-Clause | v4.25.0 | | ✗ |
| [pyiso8601] | MIT | 2.1.0 | unknown | |
| [RoaringBitmap/CRoaring] | Unknown License | v3.0.1 | | ✗ |
| [SchemaStore/schemastore] | Apache-2.0 | Unknown | | |
| [smhasher] | Unknown License | Unknown | unknown | ✗ |
| [Snowball Stemming Algorithms] | BSD-3-Clause | 7b264ffa0f767c579d052fd8142558dc8264d795 | ✗ | ✗ |
| [subunit] | BSD-3-Clause, Apache-2.0 | 1.4.4 | unknown | |
| [tcmalloc] | Apache-2.0 | 20230227-snapshot-093ba93c | | ✗ |
| [testing-cabal/extras] | MIT | 0.0.3 | unknown | |
| [testscenarios] | BSD-3-Clause, Apache-2.0 | 0.4 | unknown | |
| [testtools] | MIT | 2.7.1 | unknown | |
| [unicode-data] | Unicode-DFS-2016 | 8.0 | ✗ | ✗ |
| [valgrind] | GPL-2.0-or-later | Unknown | | ✗ |
| [zlib] | Zlib | v1.3.1 | ✗ | ✗ |
| [zstd] | BSD-3-Clause, GPL-2.0-or-later | 1.5.5 | ✗ | ✗ |
| Name | License | Vendored Version | Emits persisted data | Distributed in Release Binaries |
| ---------------------------------------------------- | --------------------------------- | ---------------------------------------- | -------------------- | ------------------------------- |
| [Abseil Common Libraries (C++)] | Apache-2.0 | 20250512.1 | | ✗ |
| [Asio C++ Library] | BSL-1.0 | 1.12.2 | | ✗ |
| [AWS SDK for C++] | Apache-2.0 | 1.11.471 | | ✗ |
| [benchmark] | Apache-2.0 | 1.5.2 | | |
| [Boost C++ Libraries] | BSL-1.0 | 1.88.0 | | ✗ |
| [c-ares] | MIT | 1.27.0 | | ✗ |
| [CRoaring] | Apache-2.0 OR MIT | 3.0.1 | | ✗ |
| [Cyrus SASL] | BSD-Attribution-HPND-disclaimer | 2.1.28 | | |
| [fmt] | MIT | 11.1.3 | | ✗ |
| [folly] | Apache-2.0 | 2023.12.25.00 | | ✗ |
| [gperftools] | BSD-3-Clause | 2.9.1 | | ✗ |
| [gRPC (C++)] | Apache-2.0 | 1.59.5 | | ✗ |
| [ICU4C - International Components for Unicode C/C++] | Unicode-3.0 | 57.1 | ✗ | ✗ |
| [immer] | BSL-1.0 | 0b3aaf699b9d6f2e89f8e2b6d1221c307e02bda3 | | ✗ |
| [Intel® Decimal Floating-Point Math Library] | BSD-3-Clause | 2.0.1 | | ✗ |
| [JSON Schema Store] | Apache-2.0 | 6847cfc3a17a04a7664474212db50c627e1e3408 | | |
| [JSON-Schema-Test-Suite] | MIT | 728066f9c5c258ba3b1804a22a5b998f2ec77ec0 | | |
| [libmongocrypt] | Apache-2.0 | 1.14.0 | ✗ | ✗ |
| [librdkafka - The Apache Kafka C/C++ library] | BSD-2-Clause | 2.0.2 | | ✗ |
| [LibTomCrypt] | Unlicense | 1.18.2 | ✗ | ✗ |
| [libunwind] | MIT | 1.8.1 | | ✗ |
| [linenoise] | BSD-2-Clause | 6cdc775807e57b2c3fd64bd207814f8ee1fe35f3 | | ✗ |
| [MongoDB C Driver] | Apache-2.0 | 1.28.1 | ✗ | ✗ |
| [Mozilla Firefox ESR] | MPL-2.0 | 128.11.0esr | | ✗ |
| [MurmurHash3] | Public Domain | a6bd3ce7be8ad147ea820a7cf6229a975c0c96bb | | ✗ |
| [nlohmann/json] | MIT | 3.11.3 | ✗ | |
| [node] | ISC | 22.1.0 | | |
| [opentelemetry-cpp] | Apache-2.0 | 1.17.0 | ✗ | |
| [opentelemetry-proto] | Apache-2.0 | 1.3.2 | ✗ | |
| [PCRE2 - Perl-Compatible Regular Expressions] | BSD-3-Clause WITH PCRE2-exception | 10.40 | | ✗ |
| [Protobuf] | BSD-3-Clause | v25.0 | | ✗ |
| [pypi/ocspbuilder] | MIT | 0.10.2 | | |
| [pypi/ocspresponder] | Apache-2.0 | 0.5.0 | | |
| [re2] | BSD-3-Clause | 2023-11-01 | | ✗ |
| [S2 Geometry Library] | Apache-2.0 | a25c502bda9d7e0274b9e2b7825fbddf13cc0306 | ✗ | ✗ |
| [SafeInt] | MIT | 3.0.26 | | ✗ |
| [snappy] | BSD-3-Clause | 1.1.10 | ✗ | ✗ |
| [Snowball Stemming Algorithms (libstemmer)] | BSD-3-Clause | 1.0.0 | ✗ | ✗ |
| [tcmalloc] | Apache-2.0 | f3b20f9a07e175c5d897df7b49d9830d4efa6110 | | ✗ |
| [timelib] | MIT | 2022.13 | | ✗ |
| [Unicode Character Database] | Unicode-DFS-2016 | 8.0.0 | ✗ | ✗ |
| [valgrind.h] | BSD-4-Clause | 093bef43d69236287ccc748591c9560a71181b0a | | ✗ |
| [WiredTiger] | GPL-2.0-only OR GPL-3.0-only | 12.0.0 | ✗ | ✗ |
| [yaml-cpp] | MIT | 0.6.3 | | ✗ |
| [zlib] | Zlib | 1.3.1 | ✗ | ✗ |
| [Zstandard (zstd)] | BSD-3-Clause OR GPL-2.0-only | 1.5.5 | ✗ | ✗ |
[Abseil]: https://github.com/abseil/abseil-cpp
[Asio C++ Library]: https://github.com/chriskohlhoff/asio
[Boost C++ Libraries - boost]: http://www.boost.org/
[Cyrus SASL]: https://www.cyrusimap.org/sasl/
[ICU for C/C++ (ICU4C)]: http://site.icu-project.org/download/
[Intel Decimal Floating-Point Math Library]: https://software.intel.com/en-us/articles/intel-decimal-floating-point-math-library
[JSON-Schema-Test-Suite]: https://github.com/json-schema-org/JSON-Schema-Test-Suite
[LibTomCrypt]: https://github.com/libtom/libtomcrypt/releases
[MongoDB C Driver]: https://github.com/mongodb/mongo-c-driver
[Mozilla Firefox]: https://www.mozilla.org/en-US/security/known-vulnerabilities/firefox-esr
[PCRE2]: http://www.pcre.org/
[Protobuf]: https://github.com/protocolbuffers/protobuf
[RoaringBitmap/CRoaring]: https://github.com/RoaringBitmap/CRoaring
[SchemaStore/schemastore]: https://www.schemastore.org/json/
[Snowball Stemming Algorithms]: https://github.com/snowballstem/snowball
[arximboldi/immer]: https://github.com/arximboldi/immer
[aws-sdk - the AWS SDK client library]: https://github.com/aws/aws-sdk-cpp
[benchmark]: https://github.com/google/benchmark
[c-ares]: https://c-ares.org/
[concurrencytest]: https://pypi.org/project/concurrencytest/
[dcleblanc/SafeInt]: https://github.com/dcleblanc/SafeInt
[derickr/timelib]: https://github.com/derickr/timelib
[discover]: https://pypi.org/project/discover/
[fmtlib/fmt]: http://fmtlib.net/
[folly]: https://github.com/facebook/folly
[google-re2]: https://github.com/google/re2
[google-snappy]: https://github.com/google/snappy/releases
[google/s2geometry]: https://github.com/google/s2geometry
[gperftools]: https://github.com/gperftools/gperftools
[grpc]: https://github.com/grpc/grpc
[jbeder/yaml-cpp]: https://github.com/jbeder/yaml-cpp/releases
[libmongocrypt]: https://github.com/mongodb/libmongocrypt
[librdkafka - the Apache Kafka C/C++ client library]: https://github.com/confluentinc/librdkafka
[libunwind/libunwind]: http://www.github.com/libunwind/libunwind
[AWS SDK for C++]: https://github.com/aws/aws-sdk-cpp.git
[Abseil Common Libraries (C++)]: https://github.com/abseil/abseil-cpp.git
[Asio C++ Library]: https://github.com/chriskohlhoff/asio.git
[Boost C++ Libraries]: https://github.com/boostorg/boost.git
[CRoaring]: https://github.com/roaringbitmap/croaring.git
[Cyrus SASL]: https://github.com/cyrusimap/cyrus-sasl.git
[ICU4C - International Components for Unicode C/C++]: https://github.com/unicode-org/icu.git
[Intel® Decimal Floating-Point Math Library]: https://www.netlib.org/misc/intel/
[JSON Schema Store]: https://github.com/schemastore/schemastore.git
[JSON-Schema-Test-Suite]: https://github.com/json-schema-org/JSON-Schema-Test-Suite.git
[LibTomCrypt]: https://github.com/libtom/libtomcrypt.git
[MongoDB C Driver]: https://github.com/mongodb/mongo-c-driver.git
[Mozilla Firefox ESR]: https://github.com/mozilla-firefox/firefox.git
[MurmurHash3]: https://github.com/aappleby/smhasher/blob/a6bd3ce/
[PCRE2 - Perl-Compatible Regular Expressions]: https://github.com/pcre2project/pcre2.git
[Protobuf]: https://github.com/protocolbuffers/protobuf.git
[S2 Geometry Library]: https://github.com/google/s2geometry.git
[SafeInt]: https://github.com/dcleblanc/safeint.git
[Snowball Stemming Algorithms (libstemmer)]: http://github.com/snowballstem/snowball.git
[Unicode Character Database]: https://www.unicode.org/Public/8.0.0/
[WiredTiger]: https://github.com/wiredtiger/wiredtiger.git
[Zstandard (zstd)]: https://github.com/facebook/zstd.git
[benchmark]: https://github.com/google/benchmark.git
[c-ares]: https://github.com/c-ares/c-ares.git
[fmt]: https://github.com/fmtlib/fmt.git
[folly]: https://github.com/facebook/folly.git
[gRPC (C++)]: https://github.com/grpc/grpc.git
[gperftools]: https://github.com/gperftools/gperftools.git
[immer]: https://github.com/arximboldi/immer.git
[libmongocrypt]: https://github.com/mongodb/libmongocrypt.git
[librdkafka - The Apache Kafka C/C++ library]: https://github.com/confluentinc/librdkafka.git
[libunwind]: https://github.com/libunwind/libunwind.git
[linenoise]: https://github.com/antirez/linenoise
[nlohmann-json]: https://github.com/open-telemetry/opentelemetry-proto
[nlohmann.json.decomposed]: https://www.nuget.org/packages/nlohmann.json.decomposed
[node]: https://nodejs.org/en/blog/release
[ocspbuilder]: https://github.com/wbond/ocspbuilder
[ocspresponder]: https://github.com/threema-ch/ocspresponder
[opentelemetry-cpp]: https://github.com/open-telemetry/opentelemetry-cpp/
[nlohmann/json]: https://github.com/nlohmann/json.git
[node]: https://nodejs.org/
[opentelemetry-cpp]: https://github.com/open-telemetry/opentelemetry-cpp.git
[opentelemetry-proto]: https://github.com/open-telemetry/opentelemetry-proto
[pyiso8601]: https://pypi.org/project/iso8601/
[smhasher]: https://github.com/aappleby/smhasher/blob/a6bd3ce/
[subunit]: https://github.com/testing-cabal/subunit
[tcmalloc]: https://github.com/google/tcmalloc
[testing-cabal/extras]: https://github.com/testing-cabal/extras
[testscenarios]: https://pypi.org/project/testscenarios/
[testtools]: https://github.com/testing-cabal/testtools
[unicode-data]: http://www.unicode.org/versions/enumeratedversions.html
[valgrind]: http://valgrind.org/downloads/current.html
[zlib]: https://zlib.net/
[zstd]: https://github.com/facebook/zstd
## WiredTiger Vendored Test Libraries
The following Python libraries are transitively included by WiredTiger,
and are used by that component for testing. They don't appear in
released binary artifacts.
| Name |
| ------------------------ |
| concurrencytest |
| discover |
| nlohmann.json.decomposed |
| pyiso8601 |
| subunit |
| testing-cabal/extras |
| testscenarios |
| testtools |
[pypi/ocspbuilder]: https://pypi.org/project/ocspbuilder/
[pypi/ocspresponder]: https://pypi.org/project/ocspresponder/
[re2]: https://github.com/google/re2.git
[snappy]: https://github.com/google/tcmalloc.git
[tcmalloc]: https://github.com/google/tcmalloc.git
[timelib]: https://github.com/derickr/timelib.git
[valgrind.h]: https://sourceware.org/git/valgrind.git
[yaml-cpp]: https://github.com/jbeder/yaml-cpp.git
[zlib]: https://zlib.net/fossils/
## Dynamically Linked Libraries

View File

@ -1,7 +1,7 @@
load("@bazel_tools//tools/cpp:toolchain_utils.bzl", "find_cpp_toolchain")
load("@bazel_tools//tools/build_defs/cc:action_names.bzl", "ACTION_NAMES")
load("//bazel/config:configs.bzl", "sdkroot_provider")
load("//bazel:mongo_src_rules.bzl", "write_target")
load("//bazel:utils.bzl", "write_target")
def generate_config_header_impl(ctx):
cc_toolchain = find_cpp_toolchain(ctx)

View File

@ -1,4 +1,4 @@
load("//bazel:mongo_src_rules.bzl", "write_target")
load("//bazel:utils.bzl", "write_target")
def render_template_impl(ctx):
python = ctx.toolchains["@bazel_tools//tools/python:toolchain_type"].py3_runtime

View File

@ -1,5 +1,5 @@
py_binary(
name = "generate_coverity_command",
srcs = ["generate_coverity_command.py"],
name = "generate_coverity_targets",
srcs = ["generate_coverity_targets.py"],
visibility = ["//visibility:public"],
)

View File

@ -17,45 +17,27 @@ bazel_cache = os.path.expanduser(args.bazel_cache)
# the cc_library and cc_binaries in our build. There is not a good way from
# within the build to get all those targets, so we will generate the list via query
# https://sig-product-docs.synopsys.com/bundle/coverity-docs/page/coverity-analysis/topics/building_with_bazel.html#build_with_bazel
proc = subprocess.run(
cmd = (
[
bazel_executable,
bazel_cache,
"aquery",
]
+ bazel_cmd_args
+ [args.bazel_query],
+ [args.bazel_query]
)
print(f"Running command: {cmd}")
proc = subprocess.run(
cmd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
print(proc.stderr)
targets = set()
for line in proc.stdout.splitlines():
if line.startswith(" Target: "):
targets.add(line.split()[-1])
enterprise_coverity_dir = os.path.join("src", "mongo", "db", "modules", "enterprise", "coverity")
os.makedirs(enterprise_coverity_dir, exist_ok=True)
with open(os.path.join(enterprise_coverity_dir, "BUILD.bazel"), "w") as buildfile:
buildfile.write("""\
load("@rules_coverity//coverity:defs.bzl", "cov_gen_script")
cov_gen_script(
name="enterprise_coverity_build",
testonly=True,
tags=["coverity"],
deps=[
""")
for target in targets:
buildfile.write(
"""\
"%s",
"""
% target
)
buildfile.write("""\
],
)
""")
with open("coverity_targets.list", "w") as f:
for line in proc.stdout.splitlines():
if line.startswith(" Target: "):
f.write(line.split()[-1] + "\n")

View File

@ -35,6 +35,7 @@ format_multirun(
graphql = "//:prettier",
html = "//:prettier",
markdown = "//:prettier",
python = "@aspect_rules_lint//format:ruff",
shell = "@shfmt//:shfmt",
sql = "//:prettier",
starlark = "@buildifier_prebuilt//:buildifier",

View File

@ -33,6 +33,9 @@ load(
)
load("@local_host_values//:local_host_values_set.bzl", "NUM_CPUS")
load("@evergreen_variables//:evergreen_variables.bzl", "UNSAFE_COMPILE_VARIANT", "UNSAFE_VERSION_ID")
load("//bazel/toolchains/cc/mongo_windows:mongo_windows_cc_toolchain_config.bzl", "MIN_VER_MAP")
load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo")
load("//bazel/config:generate_config_header.bzl", "generate_config_header")
# These will throw an error if the following condition is not met:
# (libunwind == on && os == linux) || libunwind == off || libunwind == auto
@ -1117,27 +1120,6 @@ idl_generator_rule = rule(
fragments = ["py"],
)
def write_target_impl(ctx):
out = ctx.actions.declare_file(ctx.label.name + ".gen_source_list")
ctx.actions.write(
out,
"//" + ctx.label.package + ":" + ctx.attr.target_name,
)
return [
DefaultInfo(
files = depset([out]),
),
]
write_target = rule(
write_target_impl,
attrs = {
"target_name": attr.string(
doc = "the name of the target to record",
),
},
)
def idl_generator(name, tags = [], **kwargs):
idl_generator_rule(
name = name,
@ -1464,3 +1446,155 @@ def mongo_cc_fuzzer_test(
exec_properties = exec_properties,
**kwargs
)
def _compute_win_defines(ver):
if ver not in MIN_VER_MAP:
fail("Unsupported windows_version_minimal=%r; supported: %r" %
(ver, sorted(MIN_VER_MAP.keys())))
m = MIN_VER_MAP[ver]
return [
"_WIN32_WINNT=%s" % m["win"],
"BOOST_USE_WINAPI_VERSION=%s" % m["win"],
"NTDDI_VERSION=%s" % m["ddi"],
]
def _rc_impl(ctx):
# No-op on non-Windows
if not ctx.target_platform_has_constraint(
ctx.attr._os_windows[platform_common.ConstraintValueInfo],
):
li = cc_common.create_linker_input(
owner = ctx.label,
user_link_flags = [],
additional_inputs = depset(), # and declare it as a link input
)
new_cc_shared_info = CcSharedLibraryInfo(
dynamic_deps = depset(),
exports = [],
linker_input = li,
link_once_static_libs = {},
)
return [DefaultInfo(files = depset([])), CcInfo(), new_cc_shared_info]
new_cc_shared_info = CcSharedLibraryInfo(
dynamic_deps = depset(),
exports = [],
linker_input = None,
link_once_static_libs = {},
)
# On Windows we need an rc executable
if ctx.executable.rc == None:
fail("windows_rc: 'rc' tool not provided. Pass rc = '@mongo_windows_toolchain//:rc' (or your wrapper).")
out = ctx.actions.declare_file(ctx.label.name + ".res")
defines = []
if ctx.attr.windows_version_minimal:
ver = ctx.attr.windows_version_minimal[BuildSettingInfo].value
defines += _compute_win_defines(ver)
defines += ctx.attr.defines
include_dirs = ["src", ctx.file.src.dirname]
# Add $(GENDIR)/src (use backslashes for rc.exe friendliness)
gen_src = ctx.var["GENDIR"] + "\\src"
include_dirs.append(gen_src)
args = ctx.actions.args()
args.add("/nologo")
for inc in include_dirs:
args.add("/I", inc)
for d in defines:
args.add("/d", d)
args.add("/fo" + out.path) # /fo must be joined
args.add(ctx.file.src.path)
ctx.actions.run(
executable = ctx.executable.rc, # your wrapper
arguments = [args],
inputs = [ctx.file.src] + ctx.files.resources,
outputs = [out],
mnemonic = "WindowsRC",
)
li = cc_common.create_linker_input(
owner = ctx.label,
user_link_flags = [out.path], # put .res on the link line
additional_inputs = depset([out]), # and declare it as a link input
)
lc = cc_common.create_linking_context(linker_inputs = depset([li]))
new_cc_shared_info = CcSharedLibraryInfo(
dynamic_deps = depset(),
exports = [],
linker_input = li,
link_once_static_libs = {},
)
return [DefaultInfo(files = depset([out])), CcInfo(linking_context = lc), new_cc_shared_info]
windows_rc_rule = rule(
implementation = _rc_impl,
attrs = {
"src": attr.label(mandatory = True, allow_single_file = [".rc"]),
"resources": attr.label_list(allow_files = True),
"defines": attr.string_list(),
"rc": attr.label(executable = True, cfg = "exec", allow_files = True),
"windows_version_minimal": attr.label(),
"_os_windows": attr.label(
default = Label("@platforms//os:windows"),
providers = [platform_common.ConstraintValueInfo],
),
},
provides = [CcInfo],
fragments = ["cpp"],
)
def windows_rc(name, src, manifest_in = None, icon = None):
if manifest_in:
# Turn "foo.manifest.in" into "foo.manifest"
out_manifest = manifest_in[:-3] if manifest_in.endswith(".in") else manifest_in
generate_config_header(
name = name + "_manifest_gen",
checks = "//src/mongo/util:version_constants_gen.py",
cpp_defines = [],
cpp_linkflags = [],
cpp_opts = [],
extra_definitions = {
"MONGO_DISTMOD": "$(MONGO_DISTMOD)",
"MONGO_VERSION": "$(MONGO_VERSION)",
"GIT_COMMIT_HASH": "$(GIT_COMMIT_HASH)",
} | select({
"//bazel/config:js_engine_mozjs": {"js_engine_ver": "mozjs"},
"//conditions:default": {"js_engine_ver": "none"},
}) | select({
"//bazel/config:tcmalloc_google_enabled": {"MONGO_ALLOCATOR": "tcmalloc-google"},
"//bazel/config:tcmalloc_gperf_enabled": {"MONGO_ALLOCATOR": "tcmalloc-gperf"},
"//conditions:default": {"MONGO_ALLOCATOR": "system"},
}) | select({
"//bazel/config:build_enterprise_enabled": {"build_enterprise_enabled": "1"},
"//conditions:default": {},
}),
logfile = name + "_manifest_gen.log",
output = out_manifest,
template = manifest_in,
)
resources = ["//src/mongo/util:rc_constants_gen"]
if manifest_in:
resources.append(name + "_manifest_gen")
if icon:
resources.append(icon)
windows_rc_rule(
name = name,
src = src,
resources = resources,
rc = select({
"@platforms//os:windows": "@mongo_windows_toolchain//:rc",
"//conditions:default": None,
}),
windows_version_minimal = "//bazel/config:win_min_version",
)

View File

@ -39,6 +39,14 @@ mongo_windows_cc_toolchain_config(
supports_parse_showincludes = True,
)
exports_files(["rc_wrapper.cmd"])
alias(
name = "rc",
actual = "rc_wrapper.cmd",
visibility = ["//visibility:public"]
)
cc_toolchain(
name = "mongo_win_cc_toolchain",
all_files = ":all_files",

View File

@ -11,6 +11,41 @@ load(
"setup_vc_env_vars",
)
def _compute_rc_path(ctx, env_vars):
# Allow an explicit override from CI/dev
rc_path = ctx.os.environ.get("RC_PATH")
if rc_path and ctx.path(rc_path).exists:
return rc_path.replace("\\", "/")
winsdkdir = (env_vars.get("WINDOWSSDKDIR") or "").rstrip("\\/")
sdkver = (env_vars.get("WindowsSDKVersion") or "").strip("\\/")
# Typical locations in recent SDKs
candidates = []
if winsdkdir:
if sdkver:
candidates += [
"%s\\bin\\%s\\x64\\rc.exe" % (winsdkdir, sdkver),
"%s\\bin\\%s\\x86\\rc.exe" % (winsdkdir, sdkver),
]
candidates += [
"%s\\bin\\x64\\rc.exe" % winsdkdir,
"%s\\bin\\x86\\rc.exe" % winsdkdir,
]
# Fallback to LLVM if present
llvm = ctx.os.environ.get("BAZEL_LLVM")
if llvm:
candidates += [
"%s\\bin\\llvm-rc.exe" % llvm,
"%s\\bin\\rc.exe" % llvm, # some layouts
]
for c in candidates:
if ctx.path(c).exists:
return c.replace("\\", "/")
return None
def _impl_gen_windows_toolchain_build_file(ctx):
if "windows" not in ctx.os.name:
ctx.file(
@ -85,6 +120,21 @@ def _impl_gen_windows_toolchain_build_file(ctx):
# purpose.
ctx.file("windows_toolchain_config.json", json.encode(substitutions), executable = False)
# --- expose rc.exe/llvm-rc.exe as a stable label in this repo ---
# Ask VS env for SDK location/version (setup_vc_env_vars already did that).
# Add WindowsSDKVersion to your setup_vc_env_vars() if not already included.
rc_path = _compute_rc_path(ctx, vars)
if not rc_path:
auto_configure_fail(
"Could not locate rc.exe (or llvm-rc.exe). Set RC_PATH or ensure Windows SDK is installed and WindowsSDKVersion is exported.",
)
ctx.file(
"rc_wrapper.cmd",
"@echo off\r\nsetlocal\r\nset \"INCLUDE=%s\"\r\nset \"PATH=%s\"\r\n\"%s\" %%*\r\n" % (vars["INCLUDE"], vars["PATH"], rc_path),
executable = True,
)
ctx.report_progress("Generating toolchain build file")
ctx.template(
"BUILD.bazel",

View File

@ -36,6 +36,16 @@ load(
"with_feature_set",
)
# The values are populated from the following link:
# https://learn.microsoft.com/en-us/cpp/porting/modifying-winver-and-win32-winnt?view=msvc-170
# For future versions, please add it in the map.
MIN_VER_MAP = {
"10": {
"win": "0x0A00",
"ddi": "0x0A000000",
},
}
all_compile_actions = [
ACTION_NAMES.c_compile,
ACTION_NAMES.cpp_compile,
@ -85,25 +95,15 @@ all_link_actions = [
]
def get_windows_mimimun_version_feature(ctx):
# The values are populated from the following link:
# https://learn.microsoft.com/en-us/cpp/porting/modifying-winver-and-win32-winnt?view=msvc-170
# For future versions, please add it in the map.
min_ver_map = {
"10": {
"win": "0x0A00",
"ddi": "0x0A000000",
},
}
if BuildSettingInfo not in ctx.attr.windows_version_minimal:
fail("windows_version_minimal attribute value is not a build flag.")
ver = ctx.attr.windows_version_minimal[BuildSettingInfo].value
if ver not in min_ver_map:
error_msg = "Windows mininum version {} does not exist. These are the minimum versions that are supported: {}".format(ver, min_ver_map.keys())
if ver not in MIN_VER_MAP:
error_msg = "Windows mininum version {} does not exist. These are the minimum versions that are supported: {}".format(ver, MIN_VER_MAP.keys())
fail(error_msg)
min_ver = min_ver_map[ver]
min_ver = MIN_VER_MAP[ver]
return feature(
name = "windows_version_minimum",
enabled = True,

View File

@ -351,7 +351,7 @@ def setup_vc_env_vars(repository_ctx, vc_path, envvars = [], allow_empty = False
dictionary of the envvars
"""
if not envvars:
envvars = ["PATH", "INCLUDE", "LIB", "WINDOWSSDKDIR"]
envvars = ["PATH", "INCLUDE", "LIB", "WINDOWSSDKDIR", "WindowsSDKVersion"]
vcvars_script = _find_vcvars_bat_script(repository_ctx, vc_path)
if not vcvars_script:

View File

@ -1,6 +1,27 @@
# General starlark utility functions
load("//bazel/platforms:normalize.bzl", "ARCH_NORMALIZE_MAP")
def write_target_impl(ctx):
out = ctx.actions.declare_file(ctx.label.name + ".gen_source_list")
ctx.actions.write(
out,
"//" + ctx.label.package + ":" + ctx.attr.target_name,
)
return [
DefaultInfo(
files = depset([out]),
),
]
write_target = rule(
write_target_impl,
attrs = {
"target_name": attr.string(
doc = "the name of the target to record",
),
},
)
def retry_download_and_extract(ctx, tries, **kwargs):
sleep_time = 1
for attempt in range(tries):

View File

@ -10,6 +10,7 @@ py_library(
"install_modules.py",
"lint.py",
"plus_interface.py",
"set_mongo_variables.py",
"wrapper_debug.py",
"wrapper_hook.py",
],

View File

@ -11,6 +11,8 @@ import sys
REPO_ROOT = pathlib.Path(__file__).parent.parent.parent
sys.path.append(str(REPO_ROOT))
from bazel.wrapper_hook.set_mongo_variables import write_mongo_variables_bazelrc
def run_pty_command(cmd):
stdout = None
@ -43,6 +45,8 @@ def run_pty_command(cmd):
def generate_compiledb(bazel_bin, persistent_compdb, enterprise):
# compiledb ignores command line args so just make a version rc file in anycase
write_mongo_variables_bazelrc([])
if persistent_compdb:
info_proc = subprocess.run(
[bazel_bin, "info", "output_base"], capture_output=True, text=True

View File

@ -22,6 +22,7 @@ def write_workstation_bazelrc(args):
commit = "Unknown"
user = "Unknown"
hostname = "Unknown"
base_branch = "Unknown"
try:
repo = git.Repo()
except Exception:
@ -63,6 +64,10 @@ def write_workstation_bazelrc(args):
if os.environ.get("CI") is not None:
user = os.environ.get("author_email", "Unknown")
# This is the branch that the PR is merging into
base_branch = os.environ.get("github_pr_base_branch", "Unknown")
# Replace the branch with the head branch if this is a PR check / merge queue run
branch = os.environ.get("github_pr_head_branch", branch)
try:
hostname = socket.gethostname()
@ -71,7 +76,7 @@ def write_workstation_bazelrc(args):
filtered_args = args[1:]
if "--" in filtered_args:
filtered_args = filtered_args[:filtered_args.index("--")] + ["--", "(REDACTED)"]
filtered_args = filtered_args[: filtered_args.index("--")] + ["--", "(REDACTED)"]
developer_build = os.environ.get("CI") is None
filtered_command_line = " ".join(filtered_args)
@ -87,6 +92,7 @@ common --bes_keywords=engflow:BuildScmBranch={branch}
common --bes_keywords=engflow:BuildScmRevision={commit}
common --bes_keywords=engflow:BuildScmStatus={status}
common --bes_keywords=rawCommandLineBase64={b64_cmd_line}
common --bes_keywords=base_branch={base_branch}
"""
if developer_build:

View File

@ -0,0 +1,37 @@
import os
import sys
import shutil
from concurrent.futures import ThreadPoolExecutor
TARGET = ".auto_header"
def delete_dir(path: str):
# Don't follow/delete symlinked dirs
if os.path.islink(path):
return False
try:
shutil.rmtree(path, ignore_errors=True)
return True
except Exception:
return False
def find_targets(root: str):
for dirpath, dirnames, _ in os.walk(root):
# If TARGET exists directly under this directory
if TARGET in dirnames:
target_path = os.path.join(dirpath, TARGET)
yield target_path
# Prevent os.walk from descending into it
dirnames.remove(TARGET)
def clean_up_auto_header_dirs(ROOT):
targets = list(find_targets(ROOT))
if not targets:
return
workers = min(32, (os.cpu_count() or 8) * 2)
with ThreadPoolExecutor(max_workers=workers) as ex:
_ = list(ex.map(delete_dir, targets))

View File

@ -0,0 +1,48 @@
import hashlib
import os
import pathlib
import platform
import subprocess
ARCH_NORMALIZE_MAP = {
"amd64": "x86_64",
"x86_64": "x86_64",
"arm64": "aarch64",
"aarch64": "aarch64",
"ppc64le": "ppc64le",
"s390x": "s390x",
}
def get_mongo_arch(args):
arch = platform.machine().lower()
if arch in ARCH_NORMALIZE_MAP:
return ARCH_NORMALIZE_MAP[arch]
else:
return arch
def get_mongo_version(args):
proc = subprocess.run(["git", "describe", "--abbrev=0"], capture_output=True, text=True)
return proc.stdout.strip()[1:]
def write_mongo_variables_bazelrc(args):
mongo_version = get_mongo_version(args)
mongo_arch = get_mongo_arch(args)
repo_root = pathlib.Path(os.path.abspath(__file__)).parent.parent.parent
version_file = os.path.join(repo_root, ".bazelrc.mongo_variables")
existing_hash = ""
if os.path.exists(version_file):
with open(version_file, encoding="utf-8") as f:
existing_hash = hashlib.md5(f.read().encode()).hexdigest()
bazelrc_contents = f"""
common --define=MONGO_ARCH={mongo_arch}
common --define=MONGO_VERSION={mongo_version}
"""
current_hash = hashlib.md5(bazelrc_contents.encode()).hexdigest()
if existing_hash != current_hash:
with open(version_file, "w", encoding="utf-8") as f:
f.write(bazelrc_contents)

View File

@ -22,6 +22,10 @@ def main():
from bazel.wrapper_hook.engflow_check import engflow_auth
from bazel.wrapper_hook.generate_common_bes_bazelrc import write_workstation_bazelrc
from bazel.wrapper_hook.plus_interface import check_bazel_command_type, test_runner_interface
from bazel.wrapper_hook.remove_auto_header_dirs import clean_up_auto_header_dirs
from bazel.wrapper_hook.set_mongo_variables import write_mongo_variables_bazelrc
clean_up_auto_header_dirs(REPO_ROOT)
# This is used to autogenerate a BUILD.bazel that creates
# Filegroups for select tags - used to group targets for installing
@ -38,10 +42,15 @@ def main():
)
args += ["--build_enterprise=False", "--config=local"]
if any(arg.startswith("--include_mongot") for arg in args):
os.makedirs("mongot-localdev", exist_ok=True)
engflow_auth(args)
write_workstation_bazelrc(args)
write_mongo_variables_bazelrc(args)
args = test_runner_interface(
sys.argv[1:],
autocomplete_query=os.environ.get("MONGO_AUTOCOMPLETE_QUERY") == "1",

View File

@ -150,6 +150,10 @@ py_binary(
"jsonschema",
group = "build-metrics",
),
dependency(
"license-expression",
group = "lint",
),
],
)

View File

@ -7,8 +7,15 @@ import sys
import click
import structlog
from pydantic.main import BaseModel
from urllib3.util import Retry
from evergreen.api import EvergreenApi, RetryingEvergreenApi
from evergreen.api import (
DEFAULT_HTTP_RETRY_ATTEMPTS,
DEFAULT_HTTP_RETRY_BACKOFF_FACTOR,
DEFAULT_HTTP_RETRY_CODES,
EvergreenApi,
RetryingEvergreenApi,
)
# Get relative imports to work when the package is not installed on the PYTHONPATH.
if __name__ == "__main__" and __package__ is None:
@ -137,6 +144,14 @@ def main(expansion_file: str, evergreen_config: str, verbose: bool) -> None:
enable_logging(verbose)
expansions = EvgExpansions.from_yaml_file(expansion_file)
evg_api = RetryingEvergreenApi.get_api(config_file=evergreen_config, log_on_error=True)
evg_api._http_retry = Retry(
# this is a way to reuse all of Evergreen's logic, but to bump up the number of attempts
total=DEFAULT_HTTP_RETRY_ATTEMPTS + 10,
backoff_factor=DEFAULT_HTTP_RETRY_BACKOFF_FACTOR,
status_forcelist=DEFAULT_HTTP_RETRY_CODES,
raise_on_status=False,
raise_on_redirect=False,
)
activate_task(expansions, evg_api)

View File

@ -6,6 +6,7 @@ from __future__ import annotations
import argparse
import math
import os
import re
import shlex
import sys
from datetime import timedelta
@ -115,7 +116,7 @@ class TimeoutOverrides(BaseModel):
overrides = [
override
for override in self.overrides.get(build_variant, [])
if override.task == task_name
if re.search(override.task, task_name)
]
if overrides:
if len(overrides) > 1:

View File

@ -61,8 +61,8 @@ def generate_version_expansions():
raise ValueError("Unable to parse version from stdin and no version.json provided")
if version_parts[0]:
expansions["suffix"] = "latest"
expansions["src_suffix"] = "latest"
expansions["suffix"] = "v8.2-latest"
expansions["src_suffix"] = "v8.2-latest"
expansions["is_release"] = "false"
else:
expansions["suffix"] = version_line

View File

@ -31,10 +31,11 @@ import logging
import os
import shutil
import sys
from subprocess import check_output
from subprocess import CalledProcessError, check_output
from typing import List
from packaging.version import Version
from retry import retry
# Get relative imports to work when the package is not installed on the PYTHONPATH.
if __name__ == "__main__" and __package__ is None:
@ -104,6 +105,11 @@ def get_tags() -> List[str]:
return list(gen_tags())
@retry(tries=3, delay=5)
def _show_with_retry(tag: str, path: str):
return check_output(["git", "show", f"{tag}:{path}"])
def make_idl_directories(tags: List[str], destination: str) -> None:
"""For each tag, construct a source tree containing only its IDL files."""
LOGGER.info("Clearing destination directory '%s'", destination)
@ -116,7 +122,12 @@ def make_idl_directories(tags: List[str], destination: str) -> None:
if not path.endswith(".idl"):
continue
contents = check_output(["git", "show", f"{tag}:{path}"]).decode()
try:
contents = _show_with_retry(tag, path).decode()
except CalledProcessError as e:
LOGGER.error("Failed to run git show command after multiple retries: %s", str(e))
raise e
output_path = os.path.join(directory, path)
os.makedirs(os.path.dirname(output_path), exist_ok=True)
with open(output_path, "w+") as fd:

View File

@ -71,10 +71,6 @@ def install_bazel(binary_directory: str) -> str:
is_bazelisk_supported = normalized_arch not in ["ppc64le", "s390x"]
binary_filename = "bazelisk"
binary_path = os.path.join(binary_directory, binary_filename)
if os.path.exists(binary_path):
print(f"{binary_filename} already exists ({binary_path}), skipping download")
_set_bazel_permissions(binary_path)
return binary_path
if is_bazelisk_supported:
print(f"Downloading {binary_filename}...")

View File

@ -1,6 +1,7 @@
# This script validates the msi can be installed, uninstalled, and checks for the default install location and some of the possible install files
import glob
import os
import re
import subprocess
import sys
import tempfile
@ -87,6 +88,8 @@ def validate_files(is_enterprise):
print(f"File exists: {file_match[0]}")
if file_match[0].endswith(".exe"):
validate_help(file_match[0])
if file_match[0].endswith("mongod.exe"):
validate_version(file_match[0])
else:
print(f"Error: {file_path} could not be found.")
sys.exit(1)
@ -103,6 +106,26 @@ def validate_help(exe_path):
sys.exit(1)
# Make sure we have a proper git version in the windows release
def validate_version(exe_path):
try:
version_command = [exe_path, "--version"]
print(f"Calling '{exe_path}' with command: {' '.join(version_command)}...")
result = subprocess.run(version_command, check=True, stdout=subprocess.PIPE, text=True)
print(f"{exe_path} called version successfully.")
match = re.search('.*"gitVersion": "[0-9a-fA-F]{40}".*', result.stdout)
if match:
print("Found a valid git version.")
return
else:
print("--version command did not contain a valid git version in gitVersion. Stdout:")
print(result.stdout)
sys.exit(1)
except subprocess.CalledProcessError as e:
print(f"Error while calling version for {exe_path}: {e}")
sys.exit(1)
def main():
if len(sys.argv) != 2:
print("Usage: python msi_validation.py <path_to_msi>")

View File

@ -14,6 +14,7 @@ import uuid
from concurrent import futures
from pathlib import Path
from typing import Any, Dict, Generator, List, Optional, Set, Tuple
from urllib.parse import urlparse
import docker
import docker.errors
@ -24,6 +25,8 @@ from docker.models.images import Image
from retry.api import retry_call
from simple_report import Report, Result
from buildscripts.resmokelib.utils import evergreen_conn
root = logging.getLogger()
root.setLevel(logging.DEBUG)
@ -33,6 +36,66 @@ formatter = logging.Formatter("[%(asctime)s]%(levelname)s:%(message)s")
handler.setFormatter(formatter)
root.addHandler(handler)
def download_packages_from_build(build_id: str, download_dir: Path) -> Path:
"""
Download the packages artifact from the Evergreen API.
This is needed because private artifacts require authenticated access.
The Evergreen API client handles authentication and downloads the file
to a local path that can be mounted into Docker containers.
Args:
build_id: The Evergreen build ID to search for the package task
download_dir: Directory where the packages tarball should be downloaded
Returns:
The local path to the downloaded packages tarball
"""
logging.info("Fetching packages artifact from Evergreen API for build: %s", build_id)
evg_api = evergreen_conn.get_evergreen_api()
tasks = evg_api.tasks_by_build(build_id)
package_task = None
for task in tasks:
if task.display_name == "package":
package_task = task
break
if package_task is None:
raise RuntimeError(f"Could not find 'package' task in build {build_id}")
logging.info("Found package task: %s", package_task.task_id)
packages_url = None
for artifact in package_task.artifacts:
if artifact.name == "Packages":
packages_url = artifact.url
logging.info("Found Packages artifact URL: %s", packages_url)
break
if packages_url is None:
raise RuntimeError(
f"Could not find 'Packages' artifact for package task {package_task.task_id}"
)
# Download the packages file
download_dir.mkdir(parents=True, exist_ok=True)
local_path = download_dir / "packages.tgz"
logging.info("Downloading packages to: %s", local_path)
response = requests.get(packages_url, stream=True, timeout=300)
response.raise_for_status()
with open(local_path, "wb") as f:
for chunk in response.iter_content(chunk_size=8192):
f.write(chunk)
logging.info("Downloaded packages successfully: %s", local_path)
return local_path
PACKAGE_MANAGER_COMMANDS = {
"apt": {
"update": "export DEBIAN_FRONTEND=noninteractive && apt-get update -y",
@ -249,6 +312,16 @@ VERSIONS_TO_SKIP: Set[str] = set(
)
DISABLED_TESTS: Set[Tuple[str, str]] = set()
VALID_TAR_DIRECTORY_ARCHITECTURES = [
"linux-aarch64",
"linux-x86_64",
"linux-ppc64le",
"linux-s390x",
"macos-aarch64",
"macos-x86_64",
"windows-x86_64",
]
@dataclasses.dataclass
class Test:
@ -350,7 +423,7 @@ def run_test(test: Test, client: DockerClient) -> Result:
"yum -y install yum-utils epel-release",
"yum-config-manager --enable epel",
]
if test.os_name.startswith("debian92"):
if test.os_name.startswith("debian92") or test.os_name.startswith("debian10"):
# Adapted from https://stackoverflow.com/questions/76094428/debian-stretch-repositories-404-not-found
# Debian92 renamed its repos to archive
# The first two sed commands are to replace debian92's sources list to archive repo
@ -370,8 +443,28 @@ def run_test(test: Test, client: DockerClient) -> Result:
commands.append(f"ln -s {test.python_command} /usr/bin/python3")
os.makedirs(log_external_path.parent, exist_ok=True)
# Transform file:// URLs to use Docker-mounted paths
docker_packages_urls = []
for url in test.packages_urls:
parsed_url = urlparse(url)
if parsed_url.scheme == "file":
host_path = Path(parsed_url.path)
# Check if the path is within the test_external_root (buildscripts dir)
try:
relative_path = host_path.relative_to(test_external_root)
# Transform to Docker-mounted path
docker_path = Path(test_docker_root) / relative_path
docker_packages_urls.append(f"file://{docker_path}")
except ValueError:
# Path is not relative to test_external_root
docker_packages_urls.append(url)
else:
docker_packages_urls.append(url)
commands.append(
f"python3 /mnt/package_test/package_test_internal.py {log_docker_path} {' '.join(test.packages_urls)}"
f"python3 /mnt/package_test/package_test_internal.py {log_docker_path} "
f"{' '.join(docker_packages_urls)}"
)
logging.debug(
"Attempting to run the following docker commands:\n\t%s",
@ -524,6 +617,16 @@ def get_edition_alias(edition_name: str) -> str:
return edition_name
def validate_top_level_directory(tar_name: str):
command = f"tar -tf {tar_name} | head -n 1 | awk -F/ '{{print $1}}'"
proc = subprocess.run(command, capture_output=True, shell=True, text=True)
top_level_directory = proc.stdout.strip()
if all(os_arch not in top_level_directory for os_arch in VALID_TAR_DIRECTORY_ARCHITECTURES):
raise Exception(
f"Found an unexpected os-arch pairing as the top level directory. Top level directory: {top_level_directory}"
)
arches: Set[str] = set()
oses: Set[str] = set()
editions: Set[str] = set()
@ -600,6 +703,13 @@ branch_test_parser.add_argument(
branch_test_parser.add_argument(
"-v", "--server-version", type=str, help="Server version being tested", required=True
)
branch_test_parser.add_argument(
"--evg-build-id",
type=str,
help="Evergreen build ID. If provided, the packages URL will be fetched "
"from the Evergreen API (required for private artifacts).",
default=None,
)
args = parser.parse_args()
if args.command == "release":
@ -624,9 +734,22 @@ tests: List[Test] = []
urls: List[str] = []
if args.command == "branch":
# If evg-build-id is provided, download the packages locally using the Evergreen API
# This is required for private artifacts which need authenticated access
local_packages_path: Optional[Path] = None
if args.evg_build_id:
download_dir = Path(__file__).parent / "downloaded_packages"
local_packages_path = download_packages_from_build(args.evg_build_id, download_dir)
for test_pair in args.test:
test_os = test_pair[0]
urls = [test_pair[1]]
# Use local path if packages were downloaded, otherwise use the URL from --test argument
if local_packages_path:
# Use file:// protocol for local files - package_test_internal.py will handle this
package_url = f"file://{local_packages_path.resolve()}"
else:
package_url = test_pair[1]
urls = [package_url]
if test_os not in OS_DOCKER_LOOKUP:
logging.error(
"We have not seen this OS %s before, please add it to OS_DOCKER_LOOKUP", test_os
@ -663,6 +786,8 @@ if args.command == "branch":
)
)
validate_top_level_directory("mongo-binaries.tgz")
if not args.skip_enterprise_check:
logging.info(
"Checking the source files used to build the binaries, use --skip-enterprise-check to skip this check."

View File

@ -9,6 +9,7 @@ import pathlib
import platform
import pwd
import re
import shutil
import subprocess
import sys
import tarfile
@ -31,8 +32,12 @@ root.addHandler(stdout_handler)
root.addHandler(file_handler)
DOCKER_SYSTEMCTL_REPO = "https://raw.githubusercontent.com/gdraheim/docker-systemctl-replacement"
SYSTEMCTL_URL = DOCKER_SYSTEMCTL_REPO + "/master/files/docker/systemctl3.py"
JOURNALCTL_URL = DOCKER_SYSTEMCTL_REPO + "/master/files/docker/journalctl3.py"
SYSTEMCTL_URL = (
DOCKER_SYSTEMCTL_REPO + "/eb2a963a7d8413119b432bcb6151af6076b65f84/files/docker/systemctl3.py"
)
JOURNALCTL_URL = (
DOCKER_SYSTEMCTL_REPO + "/eb2a963a7d8413119b432bcb6151af6076b65f84/files/docker/journalctl3.py"
)
TestArgs = Dict[str, Union[str, int, List[str]]]
@ -49,11 +54,22 @@ def run_and_log(cmd: str, end_on_error: bool = True):
def download_extract_package(package: str) -> List[str]:
# Use wget here because using urllib we get errors like the following
# https://stackoverflow.com/questions/27835619/urllib-and-ssl-certificate-verify-failed-error
run_and_log('wget -q "{}"'.format(package))
downloaded_file = package.split("/")[-1]
if not package.endswith(".tgz"):
# Handle local files (file:// protocol) - these are pre-downloaded by package_test.py
# when using --evg-build-id for private artifacts
if package.startswith("file://"):
local_path = package[7:] # Remove "file://" prefix
logging.info("Using local file: %s", local_path)
downloaded_file = os.path.basename(local_path)
# Copy the file to the current directory if it's not already here
if local_path != downloaded_file and local_path != os.path.join(".", downloaded_file):
shutil.copy(local_path, downloaded_file)
else:
# Use wget here because using urllib we get errors like the following
# https://stackoverflow.com/questions/27835619/urllib-and-ssl-certificate-verify-failed-error
run_and_log('wget -q "{}"'.format(package))
downloaded_file = package.split("/")[-1]
if not downloaded_file.endswith(".tgz"):
return [downloaded_file]
extracted_paths = [] # type: List[str]

View File

@ -48,6 +48,23 @@ ARCH_CHOICES = ["x86_64", "arm64", "aarch64", "s390x"]
# Made up names for the flavors of distribution we package for.
DISTROS = ["suse", "debian", "redhat", "ubuntu", "amazon", "amazon2", "amazon2023"]
unexpected_lts_release_series = ("8.2",)
def get_suffix(version, stable_name: str, unstable_name: str) -> str:
parts = version.split(".")
major = int(parts[0])
minor = int(parts[1])
series = f"{major}.{minor}"
if major >= 5:
is_stable_version = minor == 0 or series in unexpected_lts_release_series
return stable_name if is_stable_version else unstable_name
else:
return stable_name if minor % 2 == 0 else unstable_name
class Spec(object):
"""Spec class."""
@ -104,12 +121,9 @@ class Spec(object):
# e.g., "1.8.2" < "1.8.10", "1.8.2" < "1.8.2-rc1"
return self.ver > version_string
def suffix(self):
def suffix(self) -> str:
"""Return suffix."""
if int(self.ver.split(".")[0]) >= 5:
return "-org" if int(self.ver.split(".")[1]) == 0 else "-org-unstable"
else:
return "-org" if int(self.ver.split(".")[1]) % 2 == 0 else "-org-unstable"
return get_suffix(self.ver, "-org", "-org-unstable")
def prelease(self):
"""Return pre-release verison suffix."""

View File

@ -54,11 +54,8 @@ class EnterpriseSpec(packager.Spec):
"""EnterpriseSpec class."""
def suffix(self):
return packager.get_suffix(self.ver, "-enterprise", "-enterprise-unstable")
"""Suffix."""
if int(self.ver.split(".")[0]) >= 5:
return "-enterprise" if int(self.ver.split(".")[1]) == 0 else "-enterprise-unstable"
else:
return "-enterprise" if int(self.ver.split(".")[1]) % 2 == 0 else "-enterprise-unstable"
def move_required_contents(self):
"""Move the required contents to the current working directory.

View File

@ -28,5 +28,3 @@
# TODO (SERVER-73632): Delete featureFlagRouterPort.
- featureFlagRouterPort
- featureFlagSessionsCollectionCoordinatorOnConfigServer
# TODO (SERVER-101672) : Delete featureFlagOplogSamplingAsyncEnabled.
- featureFlagOplogSamplingAsyncEnabled

View File

@ -6,9 +6,6 @@ filters:
- "README.md":
approvers:
- 10gen/devprod-correctness
- "/generated_suites/*viewless_timeseries.yml":
approvers:
- 10gen/server-catalog-and-routing
- "/generated_suites/*injected_catalog_metadata.yml":
approvers:
- 10gen/server-catalog-and-routing

View File

@ -3,7 +3,7 @@
# IF YOU WISH TO MODIFY THIS SUITE, MODIFY THE CORRESPONDING MATRIX SUITE MAPPING FILE
# AND REGENERATE THE MATRIX SUITES.
#
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/aggregation_multiversion_fuzzer_last_continuous.yml
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/aggregation_expression_multiversion_fuzzer_deterministic_last_lts.yml
# regenerate matrix suites: buildscripts/resmoke.py generate-matrix-suites && bazel run //:format
##########################################################
executor:
@ -16,9 +16,9 @@ executor:
clusterType: standalone
internalQueryAppendIdToSetWindowFieldsSort: true
internalQueryMaxAllowedDensifyDocs: 1000
mongosBinVersion: last-continuous
mongosBinVersion: last-lts
traceExceptions: false
useRandomBinVersionsWithinReplicaSet: last-continuous
useRandomBinVersionsWithinReplicaSet: last-lts
nodb: ""
matrix_suite: true
selector:

View File

@ -1,27 +0,0 @@
##########################################################
# THIS IS A GENERATED FILE -- DO NOT MODIFY.
# IF YOU WISH TO MODIFY THIS SUITE, MODIFY THE CORRESPONDING MATRIX SUITE MAPPING FILE
# AND REGENERATE THE MATRIX SUITES.
#
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/aggregation_expression_multiversion_fuzzer_last_continuous.yml
# regenerate matrix suites: buildscripts/resmoke.py generate-matrix-suites && bazel run //:format
##########################################################
executor:
archive:
tests: true
config:
shell_options:
global_vars:
TestData:
clusterType: standalone
internalQueryAppendIdToSetWindowFieldsSort: true
internalQueryMaxAllowedDensifyDocs: 1000
mongosBinVersion: last-continuous
traceExceptions: false
useRandomBinVersionsWithinReplicaSet: last-continuous
nodb: ""
matrix_suite: true
selector:
roots:
- jstestfuzz/out/*.js
test_kind: js_test

View File

@ -1,27 +0,0 @@
##########################################################
# THIS IS A GENERATED FILE -- DO NOT MODIFY.
# IF YOU WISH TO MODIFY THIS SUITE, MODIFY THE CORRESPONDING MATRIX SUITE MAPPING FILE
# AND REGENERATE THE MATRIX SUITES.
#
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/aggregation_multiversion_fuzzer_deterministic_last_continuous.yml
# regenerate matrix suites: buildscripts/resmoke.py generate-matrix-suites && bazel run //:format
##########################################################
executor:
archive:
tests: true
config:
shell_options:
global_vars:
TestData:
clusterType: standalone
internalQueryAppendIdToSetWindowFieldsSort: true
internalQueryMaxAllowedDensifyDocs: 1000
mongosBinVersion: last-continuous
traceExceptions: false
useRandomBinVersionsWithinReplicaSet: last-continuous
nodb: ""
matrix_suite: true
selector:
roots:
- jstestfuzz/out/*.js
test_kind: js_test

View File

@ -1,47 +0,0 @@
##########################################################
# THIS IS A GENERATED FILE -- DO NOT MODIFY.
# IF YOU WISH TO MODIFY THIS SUITE, MODIFY THE CORRESPONDING MATRIX SUITE MAPPING FILE
# AND REGENERATE THE MATRIX SUITES.
#
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/concurrency_replication_viewless_timeseries.yml
# regenerate matrix suites: buildscripts/resmoke.py generate-matrix-suites && bazel run //:format
##########################################################
executor:
archive:
hooks:
- RunDBCheckInBackground
- CheckReplDBHashInBackground
- ValidateCollectionsInBackground
- CheckReplDBHash
- ValidateCollections
tests: true
config: {}
fixture:
class: ReplicaSetFixture
mongod_options:
oplogSize: 1024
set_parameters:
enableTestCommands: 1
featureFlagCreateViewlessTimeseriesCollections: 1
queryAnalysisSamplerConfigurationRefreshSecs: 1
queryAnalysisWriterIntervalSecs: 1
roleGraphInvalidationIsFatal: 1
num_nodes: 3
hooks:
- class: RunDBCheckInBackground
- class: CheckReplDBHashInBackground
- class: ValidateCollectionsInBackground
- class: CheckReplDBHash
- class: ValidateCollections
- class: CleanupConcurrencyWorkloads
matrix_suite: true
selector:
exclude_with_any_tags:
- requires_standalone
- requires_sharding
- viewless_timeseries_incompatible
- does_not_support_viewless_timeseries_yet
roots:
- jstests/concurrency/fsm_workloads/**/*timeseries*/**/*.js
- jstests/concurrency/fsm_workloads/**/*timeseries*.js
test_kind: fsm_workload_test

View File

@ -1,68 +0,0 @@
##########################################################
# THIS IS A GENERATED FILE -- DO NOT MODIFY.
# IF YOU WISH TO MODIFY THIS SUITE, MODIFY THE CORRESPONDING MATRIX SUITE MAPPING FILE
# AND REGENERATE THE MATRIX SUITES.
#
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/concurrency_sharded_replication_viewless_timeseries.yml
# regenerate matrix suites: buildscripts/resmoke.py generate-matrix-suites && bazel run //:format
##########################################################
executor:
archive:
hooks:
- CheckReplDBHashInBackground
- CheckReplDBHash
- CheckMetadataConsistencyInBackground
- ValidateCollections
tests: true
config:
shell_options:
eval: await import("jstests/libs/override_methods/implicitly_shard_accessed_collections.js");
global_vars:
TestData:
implicitlyShardOnCreateCollectionOnly: true
runningWithBalancer: false
fixture:
class: ShardedClusterFixture
enable_balancer: false
mongod_options:
set_parameters:
enableTestCommands: 1
featureFlagCreateViewlessTimeseriesCollections: 1
queryAnalysisSamplerConfigurationRefreshSecs: 1
queryAnalysisWriterIntervalSecs: 1
roleGraphInvalidationIsFatal: 1
mongos_options:
set_parameters:
enableTestCommands: 1
featureFlagCreateViewlessTimeseriesCollections: 1
queryAnalysisSamplerConfigurationRefreshSecs: 1
num_mongos: 2
num_rs_nodes_per_shard: 3
num_shards: 2
shard_options:
mongod_options:
oplogSize: 1024
hooks:
- class: CheckShardFilteringMetadata
- class: CheckReplDBHashInBackground
- class: CheckReplDBHash
- class: CheckMetadataConsistencyInBackground
- class: CheckOrphansDeleted
- class: CheckRoutingTableConsistency
- class: ValidateCollections
- class: CleanupConcurrencyWorkloads
matrix_suite: true
selector:
exclude_with_any_tags:
- requires_standalone
- requires_replication
- assumes_against_mongod_not_mongos
- assumes_balancer_on
- requires_profiling
- assumes_unsharded_collection
- viewless_timeseries_incompatible
- does_not_support_viewless_timeseries_yet
roots:
- jstests/concurrency/fsm_workloads/**/*timeseries*/**/*.js
- jstests/concurrency/fsm_workloads/**/*timeseries*.js
test_kind: fsm_workload_test

View File

@ -1,129 +0,0 @@
##########################################################
# THIS IS A GENERATED FILE -- DO NOT MODIFY.
# IF YOU WISH TO MODIFY THIS SUITE, MODIFY THE CORRESPONDING MATRIX SUITE MAPPING FILE
# AND REGENERATE THE MATRIX SUITES.
#
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/no_passthrough_viewless_timeseries.yml
# regenerate matrix suites: buildscripts/resmoke.py generate-matrix-suites && bazel run //:format
##########################################################
description:
'"Passthrough" means running a test against different runtime Cluster
configurations, including topology, runtime flags, fault injections, and other
parameters. Most tests by default are able to run in "passthrough" suites.
NoPassthrough is an exception, where tests here only run in the exact
configuration predefined in the tests themselves.
'
executor:
archive:
tests:
- jstests/noPassthrough/backup*.js
- jstests/noPassthrough/oplog_writes_only_permitted_on_standalone.js
- jstests/noPassthrough/wt_unclean_shutdown.js
- src/mongo/db/modules/*/jstests/live_restore/*.js
- src/mongo/db/modules/enterprise/jstests/hot_backups/**/*.js
config:
shell_options:
global_vars:
TestData:
setParameters:
featureFlagCreateViewlessTimeseriesCollections: true
setParametersMongos:
featureFlagCreateViewlessTimeseriesCollections: true
nodb: ""
matrix_suite: true
selector:
exclude_with_any_tags:
- viewless_timeseries_incompatible
- does_not_support_viewless_timeseries_yet
roots:
- jstests/noPassthrough/catalog/server_status_catalog_stats.js
- jstests/noPassthrough/cluster_scalability_misc/catalog_shard.js
- jstests/noPassthrough/cluster_scalability_misc/move_collection_create_options.js
- jstests/noPassthrough/crud/vectored_insert_no_oplog_hole.js
- jstests/noPassthrough/global_catalog/shareded_data_distribution_orphan_docs.js
- jstests/noPassthrough/index_builds/create_indexes_return_on_start.js
- jstests/noPassthrough/move_collection_not_tracked.js
- jstests/noPassthrough/query/change_streams/change_streams_timeseries_shard_collection_event.js
- jstests/noPassthrough/query/timeseries/bucket_unpacking_with_sort_granularity_change.js
- jstests/noPassthrough/query/timeseries/timeseries_validate_mixed_schema_bucket.js
- jstests/noPassthrough/query/timeseries/timeseries_validate_mixed_schema_bucket_legacy_flag.js
- jstests/noPassthrough/timeseries/bucket_reopening/timeseries_direct_remove_reopen.js
- jstests/noPassthrough/timeseries/bucket_reopening/timeseries_reopen_uncompressed_bucket_for_compression.js
- jstests/noPassthrough/timeseries/create/initial_sync_recreate_timeseries_collection.js
- jstests/noPassthrough/timeseries/create/timeseries_block_compressor_options.js
- jstests/noPassthrough/timeseries/create/timeseries_server_parameters.js
- jstests/noPassthrough/timeseries/create/timeseries_startup.js
- jstests/noPassthrough/timeseries/data_integrity/intermediate_data_consistency_checks_skips.js
- jstests/noPassthrough/timeseries/data_integrity/timeseries_insert_compression_failure.js
- jstests/noPassthrough/timeseries/data_integrity/timeseries_insert_decompression_failure.js
- jstests/noPassthrough/timeseries/data_integrity/timeseries_partial_compressed_bucket.js
- jstests/noPassthrough/timeseries/extended_range/timeseries_bucket_spanning_epoch.js
- jstests/noPassthrough/timeseries/extended_range/timeseries_extended_range_rollback.js
- jstests/noPassthrough/timeseries/extended_range/timeseries_extended_range_startup.js
- jstests/noPassthrough/timeseries/index_builds/hybrid_index_timeseries.js
- jstests/noPassthrough/timeseries/index_builds/resumable_timeseries_index_build_collection_scan_phase.js
- jstests/noPassthrough/timeseries/index_builds/timeseries_create_index_option_defaults.js
- jstests/noPassthrough/timeseries/query/timeseries_concurrent.js
- jstests/noPassthrough/timeseries/query/timeseries_group_block_metrics.js
- jstests/noPassthrough/timeseries/query/timeseries_group_bson_types.js
- jstests/noPassthrough/timeseries/query/timeseries_group_count_min_max_sharded.js
- jstests/noPassthrough/timeseries/query/timeseries_group_spill_metrics.js
- jstests/noPassthrough/timeseries/query/timeseries_group_topN_bottomN.js
- jstests/noPassthrough/timeseries/query/timeseries_id_filtered_explicit_upack_bucket.js
- jstests/noPassthrough/timeseries/query/timeseries_internal_bounded_sort_spilling.js
- jstests/noPassthrough/timeseries/query/timeseries_nested_elem_match.js
- jstests/noPassthrough/timeseries/query/timeseries_query_knob_sbe.js
- jstests/noPassthrough/timeseries/query/timeseries_sample_on_buckets.js
- jstests/noPassthrough/timeseries/query/timeseries_sort.js
- jstests/noPassthrough/timeseries/query/timeseries_sparse_fields.js
- jstests/noPassthrough/timeseries/stats/timeseries_collStats.js
- jstests/noPassthrough/timeseries/stats/timeseries_latency_stats.js
- jstests/noPassthrough/timeseries/stats/timeseries_serverStatus.js
- jstests/noPassthrough/timeseries/stats/timeseries_server_status_state_management.js
- jstests/noPassthrough/timeseries/timeseries_buckets_oplog_update.js
- jstests/noPassthrough/timeseries/timeseries_resharding_disabled.js
- jstests/noPassthrough/timeseries/timeseries_write_concurrent_collection_replace.js
- jstests/noPassthrough/timeseries/ttl/timeseries_expire.js
- jstests/noPassthrough/timeseries/ttl/timeseries_expires_with_partial_index.js
- jstests/noPassthrough/timeseries/ttl/timeseries_ttl.js
- jstests/noPassthrough/timeseries/updates_deletes/timeseries_direct_update.js
- jstests/noPassthrough/timeseries/updates_deletes/timeseries_updates_create_compressed_buckets.js
- jstests/noPassthrough/timeseries/validate/validate_compressed_bucket_with_time_out_of_order.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_bucket_max_span.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_bucket_reopening.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_bucketing_parameters_change.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_count.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_data_indexes.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_id_timestamp.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_minmax.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_version.js
- jstests/noPassthrough/timeseries/validate/validate_v3_buckets_are_unsorted.js
- jstests/noPassthrough/timeseries/write/promoting_compressed_sorted_bucket_to_compressed_unsorted_bucket.js
- jstests/noPassthrough/timeseries/write/timeseries_bucket_limit_size.js
- jstests/noPassthrough/timeseries/write/timeseries_bucket_max_size_min_limit.js
- jstests/noPassthrough/timeseries/write/timeseries_bucket_oids.js
- jstests/noPassthrough/timeseries/write/timeseries_closed_due_to_schema_changes.js
- jstests/noPassthrough/timeseries/write/timeseries_compressed_bucket_size.js
- jstests/noPassthrough/timeseries/write/timeseries_dynamic_bucket_sizing.js
- jstests/noPassthrough/timeseries/write/timeseries_dynamic_bucket_sizing_large.js
- jstests/noPassthrough/timeseries/write/timeseries_find_and_modify_without_shard_key_raw_data.js
- jstests/noPassthrough/timeseries/write/timeseries_idle_buckets.js
- jstests/noPassthrough/timeseries/write/timeseries_insert_after_cycle_primary.js
- jstests/noPassthrough/timeseries/write/timeseries_insert_after_failed_insert.js
- jstests/noPassthrough/timeseries/write/timeseries_insert_invalid_timefield.js
- jstests/noPassthrough/timeseries/write/timeseries_insert_ordered_false.js
- jstests/noPassthrough/timeseries/write/timeseries_insert_ordered_true.js
- jstests/noPassthrough/timeseries/write/timeseries_insert_rollback.js
- jstests/noPassthrough/timeseries/write/timeseries_large_measurements.js
- jstests/noPassthrough/timeseries/write/timeseries_large_measurements_max_size.js
- jstests/noPassthrough/timeseries/write/timeseries_retry_writes.js
- jstests/noPassthrough/timeseries/write/timeseries_retryable_writes_restart.js
- jstests/noPassthrough/timeseries/write/timeseries_time_backward.js
- jstests/noPassthrough/timeseries/write/timeseries_write_concern.js
- jstests/noPassthrough/versioning_protocol/stale_shard_version_with_timeseries.js
test_kind: js_test

View File

@ -1,59 +0,0 @@
##########################################################
# THIS IS A GENERATED FILE -- DO NOT MODIFY.
# IF YOU WISH TO MODIFY THIS SUITE, MODIFY THE CORRESPONDING MATRIX SUITE MAPPING FILE
# AND REGENERATE THE MATRIX SUITES.
#
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/replica_sets_jscore_passthrough_viewless_timeseries.yml
# regenerate matrix suites: buildscripts/resmoke.py generate-matrix-suites && bazel run //:format
##########################################################
executor:
archive:
hooks:
- RunDBCheckInBackground
- CheckReplDBHashInBackground
- ValidateCollectionsInBackground
- CheckReplDBHash
- CheckReplOplogs
- ValidateCollections
config:
shell_options:
eval: globalThis.testingReplication = true;
fixture:
class: ReplicaSetFixture
mongod_options:
set_parameters:
enableTestCommands: 1
featureFlagCreateViewlessTimeseriesCollections: 1
num_nodes: 2
hooks:
- class: RunDBCheckInBackground
- class: CheckReplDBHashInBackground
- class: ValidateCollectionsInBackground
- class: CheckReplOplogs
- class: CheckReplDBHash
- class: ValidateCollections
- class: CleanEveryN
n: 20
matrix_suite: true
selector:
exclude_files:
- jstests/core/txns/abort_expired_transaction.js
- jstests/core/txns/abort_transaction_thread_does_not_block_on_locks.js
- jstests/core/txns/kill_op_on_txn_expiry.js
- jstests/core/**/set_param1.js
- jstests/core/query/awaitdata_getmore_cmd.js
- jstests/core/administrative/current_op/currentop.js
- jstests/core/administrative/fsync/fsync.js
- jstests/core/txns/prepare_conflict.js
- jstests/core/txns/prepare_conflict_aggregation_behavior.js
- jstests/core/timeseries/write/timeseries_update_multi.js
exclude_with_any_tags:
- assumes_standalone_mongod
- requires_profiling
- viewless_timeseries_incompatible
- does_not_support_viewless_timeseries_yet
roots:
- jstests/core/**/*.js
- jstests/fle2/**/*.js
- src/mongo/db/modules/*/jstests/fle2/**/*.js
test_kind: js_test

View File

@ -1,88 +0,0 @@
##########################################################
# THIS IS A GENERATED FILE -- DO NOT MODIFY.
# IF YOU WISH TO MODIFY THIS SUITE, MODIFY THE CORRESPONDING MATRIX SUITE MAPPING FILE
# AND REGENERATE THE MATRIX SUITES.
#
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/sharded_collections_jscore_passthrough_viewless_timeseries.yml
# regenerate matrix suites: buildscripts/resmoke.py generate-matrix-suites && bazel run //:format
##########################################################
executor:
archive:
hooks:
- CheckReplDBHash
- CheckMetadataConsistencyInBackground
- ValidateCollections
config:
shell_options:
eval: await import("jstests/libs/override_methods/implicitly_shard_accessed_collections.js");
fixture:
class: ShardedClusterFixture
enable_balancer: false
mongod_options:
set_parameters:
enableTestCommands: 1
featureFlagCreateViewlessTimeseriesCollections: 1
mongos_options:
set_parameters:
enableTestCommands: 1
featureFlagCreateViewlessTimeseriesCollections: 1
num_shards: 2
hooks:
- class: CheckReplDBHash
- class: CheckMetadataConsistencyInBackground
- class: ValidateCollections
- class: CheckOrphansDeleted
- class: CleanEveryN
n: 20
matrix_suite: true
selector:
exclude_files:
- jstests/core/txns/**/*.js
- jstests/core/**/apitest_db.js
- jstests/core/**/awaitdata_getmore_cmd.js
- jstests/core/**/bypass_doc_validation.js
- jstests/core/**/check_shard_index.js
- jstests/core/**/compact_keeps_indexes.js
- jstests/core/**/currentop.js
- jstests/core/**/dbhash.js
- jstests/core/**/fsync.js
- jstests/core/**/geo_s2cursorlimitskip.js
- jstests/core/**/geo_update_btree2.js
- jstests/core/**/queryoptimizera.js
- jstests/core/**/startup_log.js
- jstests/core/**/tailable_cursor_invalidation.js
- jstests/core/**/tailable_getmore_batch_size.js
- jstests/core/**/tailable_skip_limit.js
- jstests/core/**/query/top/top.js
- jstests/core/**/geo_2d_explain.js
- jstests/core/**/geo_s2explain.js
- jstests/core/**/geo_s2sparse.js
- jstests/core/**/operation_latency_histogram.js
- jstests/core/**/apitest_dbcollection.js
- jstests/core/**/bad_index_plugin.js
- jstests/core/**/create_indexes.js
- jstests/core/**/list_indexes_non_existent_ns.js
- jstests/core/**/mr_preserve_indexes.js
- jstests/core/**/distinct_index1.js
- jstests/core/**/expr_index_use.js
- jstests/core/**/index_multikey.js
- jstests/core/**/query/explain/optimized_match_explain.js
- jstests/core/**/sort_array.js
exclude_with_any_tags:
- assumes_standalone_mongod
- assumes_against_mongod_not_mongos
- assumes_no_implicit_collection_creation_on_get_collection
- assumes_no_implicit_collection_creation_after_drop
- assumes_no_implicit_index_creation
- assumes_unsharded_collection
- cannot_create_unique_index_when_using_hashed_shard_key
- requires_profiling
- requires_capped
- viewless_timeseries_incompatible
- does_not_support_viewless_timeseries_yet
roots:
- jstests/core/**/*.js
- jstests/core_sharding/**/*.js
- jstests/fle2/**/*.js
- src/mongo/db/modules/*/jstests/fle2/**/*.js
test_kind: js_test

View File

@ -1,66 +0,0 @@
##########################################################
# THIS IS A GENERATED FILE -- DO NOT MODIFY.
# IF YOU WISH TO MODIFY THIS SUITE, MODIFY THE CORRESPONDING MATRIX SUITE MAPPING FILE
# AND REGENERATE THE MATRIX SUITES.
#
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/sharding_jscore_passthrough_viewless_timeseries.yml
# regenerate matrix suites: buildscripts/resmoke.py generate-matrix-suites && bazel run //:format
##########################################################
executor:
archive:
hooks:
- CheckReplDBHash
- CheckMetadataConsistencyInBackground
- ValidateCollections
config:
shell_options: {}
fixture:
class: ShardedClusterFixture
mongod_options:
set_parameters:
enableTestCommands: 1
featureFlagCreateViewlessTimeseriesCollections: 1
mongos_options:
set_parameters:
enableTestCommands: 1
featureFlagCreateViewlessTimeseriesCollections: 1
num_rs_nodes_per_shard: 1
num_shards: 2
hooks:
- class: CheckReplDBHash
- class: CheckMetadataConsistencyInBackground
- class: ValidateCollections
- class: CheckOrphansDeleted
- class: CleanEveryN
n: 20
matrix_suite: true
selector:
exclude_files:
- jstests/core/txns/**/*.js
- jstests/core/**/apitest_db.js
- jstests/core/**/check_shard_index.js
- jstests/core/**/compact_keeps_indexes.js
- jstests/core/**/currentop.js
- jstests/core/**/dbhash.js
- jstests/core/**/fsync.js
- jstests/core/**/geo_s2cursorlimitskip.js
- jstests/core/**/geo_update_btree2.js
- jstests/core/**/queryoptimizera.js
- jstests/core/**/startup_log.js
- jstests/core/**/query/top/top.js
- jstests/core/**/geo_2d_explain.js
- jstests/core/**/geo_s2explain.js
- jstests/core/**/geo_s2sparse.js
- jstests/core/**/operation_latency_histogram.js
exclude_with_any_tags:
- assumes_standalone_mongod
- assumes_against_mongod_not_mongos
- requires_profiling
- viewless_timeseries_incompatible
- does_not_support_viewless_timeseries_yet
roots:
- jstests/core/**/*.js
- jstests/core_sharding/**/*.js
- jstests/fle2/**/*.js
- src/mongo/db/modules/*/jstests/fle2/**/*.js
test_kind: js_test

View File

@ -1,60 +0,0 @@
##########################################################
# THIS IS A GENERATED FILE -- DO NOT MODIFY.
# IF YOU WISH TO MODIFY THIS SUITE, MODIFY THE CORRESPONDING MATRIX SUITE MAPPING FILE
# AND REGENERATE THE MATRIX SUITES.
#
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/sharding_viewless_timeseries.yml
# regenerate matrix suites: buildscripts/resmoke.py generate-matrix-suites && bazel run //:format
##########################################################
executor:
archive:
tests:
- jstests/sharding/*reshard*.js
config:
shell_options:
global_vars:
TestData:
setParameters:
featureFlagCreateViewlessTimeseriesCollections: true
setParametersMongos:
featureFlagCreateViewlessTimeseriesCollections: true
nodb: ""
matrix_suite: true
selector:
exclude_with_any_tags:
- viewless_timeseries_incompatible
- does_not_support_viewless_timeseries_yet
roots:
- jstests/sharding/analyze_shard_key/timeseries.js
- jstests/sharding/libs/timeseries_update_multi_util.js
- jstests/sharding/resharding_timeseries/move_timeseries.js
- jstests/sharding/resharding_timeseries/reshard_timeseries.js
- jstests/sharding/resharding_timeseries/reshard_timeseries_disallow_writes.js
- jstests/sharding/resharding_timeseries/reshard_timeseries_nonempty_stash.js
- jstests/sharding/resharding_timeseries/reshard_timeseries_stash_resolution.js
- jstests/sharding/resharding_timeseries/reshard_timeseries_ttl_deletes.js
- jstests/sharding/resharding_timeseries/reshard_timeseries_validation.js
- jstests/sharding/resharding_timeseries/resharding_ts_resume_agg_token.js
- jstests/sharding/resharding_timeseries/unshard_timeseries.js
- jstests/sharding/timeseries/timeseries_balancer.js
- jstests/sharding/timeseries/timeseries_buckets_modification_with_id.js
- jstests/sharding/timeseries/timeseries_cluster_collstats.js
- jstests/sharding/timeseries/timeseries_cluster_indexstats.js
- jstests/sharding/timeseries/timeseries_coll_mod.js
- jstests/sharding/timeseries/timeseries_coll_mod_bucketing_parameters.js
- jstests/sharding/timeseries/timeseries_drop.js
- jstests/sharding/timeseries/timeseries_indexes.js
- jstests/sharding/timeseries/timeseries_insert.js
- jstests/sharding/timeseries/timeseries_insert_move_collection.js
- jstests/sharding/timeseries/timeseries_insert_targeting_normalize_metadata.js
- jstests/sharding/timeseries/timeseries_multiple_mongos.js
- jstests/sharding/timeseries/timeseries_orphan_buckets.js
- jstests/sharding/timeseries/timeseries_out_conflicting_namespace_on_shard.js
- jstests/sharding/timeseries/timeseries_query.js
- jstests/sharding/timeseries/timeseries_query_extended_range.js
- jstests/sharding/timeseries/timeseries_shard_collection.js
- jstests/sharding/timeseries/timeseries_shard_collection_with_deprecation_message.js
- jstests/sharding/timeseries/timeseries_sharding_admin_commands.js
- jstests/sharding/timeseries/timeseries_time_value_rounding.js
- jstests/sharding/timeseries/timeseries_user_system_buckets_sharding.js
test_kind: js_test

View File

@ -1,83 +0,0 @@
##########################################################
# THIS IS A GENERATED FILE -- DO NOT MODIFY.
# IF YOU WISH TO MODIFY THIS SUITE, MODIFY THE CORRESPONDING MATRIX SUITE MAPPING FILE
# AND REGENERATE THE MATRIX SUITES.
#
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/unsplittable_collections_created_on_any_shard_jscore_passthrough_viewless_timeseries.yml
# regenerate matrix suites: buildscripts/resmoke.py generate-matrix-suites && bazel run //:format
##########################################################
executor:
archive:
hooks:
- CheckReplDBHash
- CheckMetadataConsistencyInBackground
- ValidateCollections
config:
shell_options:
global_vars:
TestData:
createsUnsplittableCollectionsOnRandomShards: true
implicitlyTrackUnshardedCollectionOnCreation: true
fixture:
class: ShardedClusterFixture
enable_balancer: false
mongod_options:
set_parameters:
enableTestCommands: 1
failpoint.createUnshardedCollectionRandomizeDataShard:
mode: alwaysOn
featureFlagCreateViewlessTimeseriesCollections: 1
mongos_options:
set_parameters:
enableTestCommands: 1
failpoint.createUnshardedCollectionRandomizeDataShard:
mode: alwaysOn
featureFlagCreateViewlessTimeseriesCollections: 1
num_shards: 2
hooks:
- class: CheckReplDBHash
- class: CheckMetadataConsistencyInBackground
- class: ValidateCollections
- class: CheckOrphansDeleted
- class: CleanEveryN
n: 20
matrix_suite: true
selector:
exclude_files:
- jstests/core/txns/**/*.js
- jstests/core/**/apitest_db.js
- jstests/core/**/check_shard_index.js
- jstests/core/**/compact_keeps_indexes.js
- jstests/core/**/currentop.js
- jstests/core/**/dbhash.js
- jstests/core/**/fsync.js
- jstests/core/**/geo_s2cursorlimitskip.js
- jstests/core/**/geo_update_btree2.js
- jstests/core/**/queryoptimizera.js
- jstests/core/**/startup_log.js
- jstests/core/**/query/top/top.js
- jstests/core/**/geo_2d_explain.js
- jstests/core/**/geo_s2explain.js
- jstests/core/**/geo_s2sparse.js
- jstests/core/**/operation_latency_histogram.js
- jstests/core/**/or_to_in.js
- jstests/core/query/explain/explain_find_trivially_false_predicates_in_agg_pipelines.js
- jstests/core/query/query_settings/query_settings_index_application_aggregate.js
- jstests/core/write/insert/insert_id_undefined.js
- jstests/core/timeseries/ddl/timeseries_user_system_buckets.js
- jstests/core/ddl/clone_collection_as_capped.js
exclude_with_any_tags:
- assumes_standalone_mongod
- assumes_against_mongod_not_mongos
- requires_profiling
- requires_system_dot_js_stored_functions
- assumes_no_track_upon_creation
- viewless_timeseries_incompatible
- does_not_support_viewless_timeseries_yet
roots:
- jstests/core/**/*.js
- jstests/core_sharding/**/*.js
- jstests/fle2/**/*.js
- src/mongo/db/modules/*/jstests/fle2/**/*.js
- jstests/libs/random_data_shard_suite_selftest.js
test_kind: js_test

View File

@ -3,7 +3,7 @@
# IF YOU WISH TO MODIFY THIS SUITE, MODIFY THE CORRESPONDING MATRIX SUITE MAPPING FILE
# AND REGENERATE THE MATRIX SUITES.
#
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/update_fuzzer_last_continuous.yml
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/update_fuzzer_deterministic_last_lts.yml
# regenerate matrix suites: buildscripts/resmoke.py generate-matrix-suites && bazel run //:format
##########################################################
executor:

View File

@ -1,22 +0,0 @@
##########################################################
# THIS IS A GENERATED FILE -- DO NOT MODIFY.
# IF YOU WISH TO MODIFY THIS SUITE, MODIFY THE CORRESPONDING MATRIX SUITE MAPPING FILE
# AND REGENERATE THE MATRIX SUITES.
#
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/update_fuzzer_replication_deterministic_last_continuous.yml
# regenerate matrix suites: buildscripts/resmoke.py generate-matrix-suites && bazel run //:format
##########################################################
executor:
archive:
tests: true
config:
shell_options:
global_vars:
TestData:
clusterType: replset
nodb: ""
matrix_suite: true
selector:
roots:
- jstestfuzz/out/*.js
test_kind: js_test

View File

@ -1,22 +0,0 @@
##########################################################
# THIS IS A GENERATED FILE -- DO NOT MODIFY.
# IF YOU WISH TO MODIFY THIS SUITE, MODIFY THE CORRESPONDING MATRIX SUITE MAPPING FILE
# AND REGENERATE THE MATRIX SUITES.
#
# matrix suite mapping file: buildscripts/resmokeconfig/matrix_suites/mappings/update_fuzzer_replication_last_continuous.yml
# regenerate matrix suites: buildscripts/resmoke.py generate-matrix-suites && bazel run //:format
##########################################################
executor:
archive:
tests: true
config:
shell_options:
global_vars:
TestData:
clusterType: replset
nodb: ""
matrix_suite: true
selector:
roots:
- jstestfuzz/out/*.js
test_kind: js_test

View File

@ -47,9 +47,6 @@ filters:
approvers:
- 10gen/query-execution-query-settings
- 10gen/query-integration-observability
- "*viewless_timeseries*":
approvers:
- 10gen/server-catalog-and-routing
- "sharding_pqs*":
approvers:
- 10gen/query-execution-query-settings

View File

@ -0,0 +1,3 @@
base_suite: generational_fuzzer
overrides:
- "multiversion.replica_sets_multiversion_testdata_last_lts"

View File

@ -1,3 +0,0 @@
base_suite: generational_fuzzer
overrides:
- "multiversion.replica_sets_multiversion_testdata_last_continuous"

View File

@ -1,3 +0,0 @@
base_suite: generational_fuzzer
overrides:
- "multiversion.replica_sets_multiversion_testdata_last_continuous"

View File

@ -1,3 +0,0 @@
base_suite: generational_fuzzer
overrides:
- "multiversion.replica_sets_multiversion_testdata_last_continuous"

View File

@ -1,12 +0,0 @@
# Temorary suites to assists the development of new viewless timeseries collection.
# Run all jstests/concurrency/fsm_workloads/timeseries/* tests in concurrency_replication suite suite with
# CreateViewlessTimeseriesCollection feature flag enabled.
#
# TODO SERVER-101590: remove this temorary suite and all its usages.
base_suite: concurrency_replication
overrides:
- "viewless_timeseries.all_concurrency_timeseries_tests_selector"
- "viewless_timeseries.enable_viewless_timeseries_feature_flag_mongod"
excludes:
- "viewless_timeseries.exclude_viewless_timeseries_incompabile_tag"

View File

@ -1,13 +0,0 @@
# Temorary suites to assists the development of new viewless timeseries collection.
# Run all jstests/concurrency/fsm_workloads/timeseries/* tests in concurrency_sharded_replication suite suite with
# CreateViewlessTimeseriesCollection feature flag enabled.
#
# TODO SERVER-101590: remove this temorary suite and all its usages.
base_suite: concurrency_sharded_replication
overrides:
- "viewless_timeseries.all_concurrency_timeseries_tests_selector"
- "viewless_timeseries.enable_viewless_timeseries_feature_flag_mongod"
- "viewless_timeseries.enable_viewless_timeseries_feature_flag_mongos"
excludes:
- "viewless_timeseries.exclude_viewless_timeseries_incompabile_tag"

View File

@ -1,13 +0,0 @@
# Temporary suites to assists the development of new viewless timeseries collection.
# Run all timeseries tests in the noPassthrough suite with
# CreateViewlessTimeseriesCollection feature flag enabled.
#
# TODO SERVER-101590: remove this temporary suite and all its usages.
base_suite: no_passthrough
overrides:
- "viewless_timeseries.only_validated_no_passthrough_timeseries_tests_selector"
- "viewless_timeseries.enable_viewless_timeseries_feature_flag_testdata_parameter_mongod"
- "viewless_timeseries.enable_viewless_timeseries_feature_flag_testdata_parameter_mongos"
excludes:
- "viewless_timeseries.exclude_viewless_timeseries_incompabile_tag"

View File

@ -1,11 +0,0 @@
# Temorary suites to assists the development of new viewless timeseries collection.
# Run all jstests/core/timeseries/* tests in replicaset jscore pasthrough suite with
# CreateViewlessTimeseriesCollection feature flag enabled.
#
# TODO SERVER-101590: remove this temorary suite and all its usages.
base_suite: replica_sets_jscore_passthrough
overrides:
- "viewless_timeseries.enable_viewless_timeseries_feature_flag_mongod"
excludes:
- "viewless_timeseries.exclude_viewless_timeseries_incompabile_tag"

View File

@ -1,12 +0,0 @@
# Temorary suites to assists the development of new viewless timeseries collection.
# Run all jstests/core/timeseries/* tests in sharded jscore pasthrough suite with
# CreateViewlessTimeseriesCollection feature flag enabled.
#
# TODO SERVER-101590: remove this temorary suite and all its usages.
base_suite: sharded_collections_jscore_passthrough
overrides:
- "viewless_timeseries.enable_viewless_timeseries_feature_flag_mongod"
- "viewless_timeseries.enable_viewless_timeseries_feature_flag_mongos"
excludes:
- "viewless_timeseries.exclude_viewless_timeseries_incompabile_tag"

View File

@ -1,12 +0,0 @@
# Temorary suites to assists the development of new viewless timeseries collection.
# Run all jstests/core/timeseries/* tests in sharding jscore pasthrough suite with
# CreateViewlessTimeseriesCollection feature flag enabled.
#
# TODO SERVER-101590: remove this temorary suite and all its usages.
base_suite: sharding_jscore_passthrough
overrides:
- "viewless_timeseries.enable_viewless_timeseries_feature_flag_mongod"
- "viewless_timeseries.enable_viewless_timeseries_feature_flag_mongos"
excludes:
- "viewless_timeseries.exclude_viewless_timeseries_incompabile_tag"

View File

@ -1,13 +0,0 @@
# Temporary suites to assists the development of new viewless timeseries collection.
# Run all timeseries tests in the sharding suite with
# CreateViewlessTimeseriesCollection feature flag enabled.
#
# TODO SERVER-101590: remove this temporary suite and all its usages.
base_suite: sharding
overrides:
- "viewless_timeseries.only_validated_sharding_timeseries_tests_selector"
- "viewless_timeseries.enable_viewless_timeseries_feature_flag_testdata_parameter_mongod"
- "viewless_timeseries.enable_viewless_timeseries_feature_flag_testdata_parameter_mongos"
excludes:
- "viewless_timeseries.exclude_viewless_timeseries_incompabile_tag"

View File

@ -1,12 +0,0 @@
# Temorary suites to assists the development of new viewless timeseries collection.
# Run all jstests/core/timeseries/* tests in sharding jscore pasthrough suite with
# CreateViewlessTimeseriesCollection feature flag enabled.
#
# TODO SERVER-101590: remove this temorary suite and all its usages.
base_suite: unsplittable_collections_created_on_any_shard_jscore_passthrough
overrides:
- "viewless_timeseries.enable_viewless_timeseries_feature_flag_mongod"
- "viewless_timeseries.enable_viewless_timeseries_feature_flag_mongos"
excludes:
- "viewless_timeseries.exclude_viewless_timeseries_incompabile_tag"

View File

@ -1 +0,0 @@
base_suite: update_fuzzer_replication

View File

@ -24,9 +24,6 @@ filters:
- "repeat_queries.yml":
approvers:
- 10gen/query-optimization
- "*viewless_timeseries.yml":
approvers:
- 10gen/server-catalog-and-routing
- "stepdown_primary_change_streams.yml":
approvers:
- 10gen/query-execution-change-streams

View File

@ -1,194 +0,0 @@
- name: enable_viewless_timeseries_feature_flag_mongod
value:
executor:
fixture:
mongod_options:
set_parameters:
featureFlagCreateViewlessTimeseriesCollections: 1
- name: enable_viewless_timeseries_feature_flag_mongos
value:
executor:
fixture:
mongos_options:
set_parameters:
featureFlagCreateViewlessTimeseriesCollections: 1
- name: enable_viewless_timeseries_feature_flag_testdata_parameter_mongod
value:
executor:
config:
shell_options:
global_vars:
TestData:
setParameters:
featureFlagCreateViewlessTimeseriesCollections: true
- name: enable_viewless_timeseries_feature_flag_testdata_parameter_mongos
value:
executor:
config:
shell_options:
global_vars:
TestData:
setParametersMongos:
featureFlagCreateViewlessTimeseriesCollections: true
- name: all_concurrency_timeseries_tests_selector
value:
selector:
exclude_files: null
roots:
- jstests/concurrency/fsm_workloads/**/*timeseries*/**/*.js
- jstests/concurrency/fsm_workloads/**/*timeseries*.js
- name: all_sharding_timeseries_tests_selector
value:
selector:
exclude_files: null
roots:
- jstests/sharding/*timeseries*/*.js
- jstests/sharding/**/*timeseries*.js
- name: only_validated_sharding_timeseries_tests_selector
value:
selector:
exclude_files: null
roots:
- jstests/sharding/analyze_shard_key/timeseries.js
- jstests/sharding/libs/timeseries_update_multi_util.js
- jstests/sharding/resharding_timeseries/move_timeseries.js
- jstests/sharding/resharding_timeseries/reshard_timeseries.js
- jstests/sharding/resharding_timeseries/reshard_timeseries_disallow_writes.js
- jstests/sharding/resharding_timeseries/reshard_timeseries_nonempty_stash.js
- jstests/sharding/resharding_timeseries/reshard_timeseries_stash_resolution.js
- jstests/sharding/resharding_timeseries/reshard_timeseries_ttl_deletes.js
- jstests/sharding/resharding_timeseries/reshard_timeseries_validation.js
- jstests/sharding/resharding_timeseries/resharding_ts_resume_agg_token.js
- jstests/sharding/resharding_timeseries/unshard_timeseries.js
- jstests/sharding/timeseries/timeseries_balancer.js
- jstests/sharding/timeseries/timeseries_buckets_modification_with_id.js
- jstests/sharding/timeseries/timeseries_cluster_collstats.js
- jstests/sharding/timeseries/timeseries_cluster_indexstats.js
- jstests/sharding/timeseries/timeseries_coll_mod.js
- jstests/sharding/timeseries/timeseries_coll_mod_bucketing_parameters.js
- jstests/sharding/timeseries/timeseries_drop.js
- jstests/sharding/timeseries/timeseries_indexes.js
- jstests/sharding/timeseries/timeseries_insert.js
- jstests/sharding/timeseries/timeseries_insert_move_collection.js
- jstests/sharding/timeseries/timeseries_insert_targeting_normalize_metadata.js
- jstests/sharding/timeseries/timeseries_multiple_mongos.js
- jstests/sharding/timeseries/timeseries_orphan_buckets.js
- jstests/sharding/timeseries/timeseries_out_conflicting_namespace_on_shard.js
- jstests/sharding/timeseries/timeseries_query.js
- jstests/sharding/timeseries/timeseries_query_extended_range.js
- jstests/sharding/timeseries/timeseries_shard_collection.js
- jstests/sharding/timeseries/timeseries_shard_collection_with_deprecation_message.js
- jstests/sharding/timeseries/timeseries_sharding_admin_commands.js
- jstests/sharding/timeseries/timeseries_time_value_rounding.js
- jstests/sharding/timeseries/timeseries_user_system_buckets_sharding.js
- name: all_no_passthrough_timeseries_tests_selector
value:
selector:
exclude_files: null
roots:
- jstests/noPassthrough/catalog/server_status_catalog_stats.js
- jstests/noPassthrough/cluster_scalability_misc/catalog_shard.js
- jstests/noPassthrough/cluster_scalability_misc/move_collection_create_options.js
- jstests/noPassthrough/crud/bulk_write_metrics.js
- jstests/noPassthrough/crud/vectored_insert_no_oplog_hole.js
- jstests/noPassthrough/global_catalog/shareded_data_distribution_orphan_docs.js
- jstests/noPassthrough/index_builds/create_indexes_return_on_start.js
- jstests/noPassthrough/logging/log_collectionType.js
- jstests/noPassthrough/move_collection_not_tracked.js
- jstests/noPassthrough/**/timeseries/**/*.js
- jstests/noPassthrough/**/*timeseries*.js
- name: only_validated_no_passthrough_timeseries_tests_selector
value:
selector:
exclude_files: null
roots:
- jstests/noPassthrough/catalog/server_status_catalog_stats.js
- jstests/noPassthrough/cluster_scalability_misc/catalog_shard.js
- jstests/noPassthrough/cluster_scalability_misc/move_collection_create_options.js
- jstests/noPassthrough/crud/vectored_insert_no_oplog_hole.js
- jstests/noPassthrough/global_catalog/shareded_data_distribution_orphan_docs.js
- jstests/noPassthrough/index_builds/create_indexes_return_on_start.js
- jstests/noPassthrough/move_collection_not_tracked.js
- jstests/noPassthrough/query/change_streams/change_streams_timeseries_shard_collection_event.js
- jstests/noPassthrough/query/timeseries/bucket_unpacking_with_sort_granularity_change.js
- jstests/noPassthrough/query/timeseries/timeseries_validate_mixed_schema_bucket.js
- jstests/noPassthrough/query/timeseries/timeseries_validate_mixed_schema_bucket_legacy_flag.js
- jstests/noPassthrough/timeseries/bucket_reopening/timeseries_direct_remove_reopen.js
- jstests/noPassthrough/timeseries/bucket_reopening/timeseries_reopen_uncompressed_bucket_for_compression.js
- jstests/noPassthrough/timeseries/create/initial_sync_recreate_timeseries_collection.js
- jstests/noPassthrough/timeseries/create/timeseries_block_compressor_options.js
- jstests/noPassthrough/timeseries/create/timeseries_server_parameters.js
- jstests/noPassthrough/timeseries/create/timeseries_startup.js
- jstests/noPassthrough/timeseries/data_integrity/intermediate_data_consistency_checks_skips.js
- jstests/noPassthrough/timeseries/data_integrity/timeseries_insert_compression_failure.js
- jstests/noPassthrough/timeseries/data_integrity/timeseries_insert_decompression_failure.js
- jstests/noPassthrough/timeseries/data_integrity/timeseries_partial_compressed_bucket.js
- jstests/noPassthrough/timeseries/extended_range/timeseries_bucket_spanning_epoch.js
- jstests/noPassthrough/timeseries/extended_range/timeseries_extended_range_rollback.js
- jstests/noPassthrough/timeseries/extended_range/timeseries_extended_range_startup.js
- jstests/noPassthrough/timeseries/index_builds/hybrid_index_timeseries.js
- jstests/noPassthrough/timeseries/index_builds/resumable_timeseries_index_build_collection_scan_phase.js
- jstests/noPassthrough/timeseries/index_builds/timeseries_create_index_option_defaults.js
- jstests/noPassthrough/timeseries/query/timeseries_concurrent.js
- jstests/noPassthrough/timeseries/query/timeseries_group_block_metrics.js
- jstests/noPassthrough/timeseries/query/timeseries_group_bson_types.js
- jstests/noPassthrough/timeseries/query/timeseries_group_count_min_max_sharded.js
- jstests/noPassthrough/timeseries/query/timeseries_group_spill_metrics.js
- jstests/noPassthrough/timeseries/query/timeseries_group_topN_bottomN.js
- jstests/noPassthrough/timeseries/query/timeseries_id_filtered_explicit_upack_bucket.js
- jstests/noPassthrough/timeseries/query/timeseries_internal_bounded_sort_spilling.js
- jstests/noPassthrough/timeseries/query/timeseries_nested_elem_match.js
- jstests/noPassthrough/timeseries/query/timeseries_query_knob_sbe.js
- jstests/noPassthrough/timeseries/query/timeseries_sample_on_buckets.js
- jstests/noPassthrough/timeseries/query/timeseries_sort.js
- jstests/noPassthrough/timeseries/query/timeseries_sparse_fields.js
- jstests/noPassthrough/timeseries/stats/timeseries_collStats.js
- jstests/noPassthrough/timeseries/stats/timeseries_latency_stats.js
- jstests/noPassthrough/timeseries/stats/timeseries_serverStatus.js
- jstests/noPassthrough/timeseries/stats/timeseries_server_status_state_management.js
- jstests/noPassthrough/timeseries/timeseries_buckets_oplog_update.js
- jstests/noPassthrough/timeseries/timeseries_resharding_disabled.js
- jstests/noPassthrough/timeseries/timeseries_write_concurrent_collection_replace.js
- jstests/noPassthrough/timeseries/ttl/timeseries_expire.js
- jstests/noPassthrough/timeseries/ttl/timeseries_expires_with_partial_index.js
- jstests/noPassthrough/timeseries/ttl/timeseries_ttl.js
- jstests/noPassthrough/timeseries/updates_deletes/timeseries_direct_update.js
- jstests/noPassthrough/timeseries/updates_deletes/timeseries_updates_create_compressed_buckets.js
- jstests/noPassthrough/timeseries/validate/validate_compressed_bucket_with_time_out_of_order.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_bucket_max_span.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_bucket_reopening.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_bucketing_parameters_change.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_count.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_data_indexes.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_id_timestamp.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_minmax.js
- jstests/noPassthrough/timeseries/validate/validate_timeseries_version.js
- jstests/noPassthrough/timeseries/validate/validate_v3_buckets_are_unsorted.js
- jstests/noPassthrough/timeseries/write/promoting_compressed_sorted_bucket_to_compressed_unsorted_bucket.js
- jstests/noPassthrough/timeseries/write/timeseries_bucket_limit_size.js
- jstests/noPassthrough/timeseries/write/timeseries_bucket_max_size_min_limit.js
- jstests/noPassthrough/timeseries/write/timeseries_bucket_oids.js
- jstests/noPassthrough/timeseries/write/timeseries_closed_due_to_schema_changes.js
- jstests/noPassthrough/timeseries/write/timeseries_compressed_bucket_size.js
- jstests/noPassthrough/timeseries/write/timeseries_dynamic_bucket_sizing.js
- jstests/noPassthrough/timeseries/write/timeseries_dynamic_bucket_sizing_large.js
- jstests/noPassthrough/timeseries/write/timeseries_find_and_modify_without_shard_key_raw_data.js
- jstests/noPassthrough/timeseries/write/timeseries_idle_buckets.js
- jstests/noPassthrough/timeseries/write/timeseries_insert_after_cycle_primary.js
- jstests/noPassthrough/timeseries/write/timeseries_insert_after_failed_insert.js
- jstests/noPassthrough/timeseries/write/timeseries_insert_invalid_timefield.js
- jstests/noPassthrough/timeseries/write/timeseries_insert_ordered_false.js
- jstests/noPassthrough/timeseries/write/timeseries_insert_ordered_true.js
- jstests/noPassthrough/timeseries/write/timeseries_insert_rollback.js
- jstests/noPassthrough/timeseries/write/timeseries_large_measurements.js
- jstests/noPassthrough/timeseries/write/timeseries_large_measurements_max_size.js
- jstests/noPassthrough/timeseries/write/timeseries_retry_writes.js
- jstests/noPassthrough/timeseries/write/timeseries_retryable_writes_restart.js
- jstests/noPassthrough/timeseries/write/timeseries_time_backward.js
- jstests/noPassthrough/timeseries/write/timeseries_write_concern.js
- jstests/noPassthrough/versioning_protocol/stale_shard_version_with_timeseries.js
- name: exclude_viewless_timeseries_incompabile_tag
value:
exclude_with_any_tags:
- viewless_timeseries_incompatible
- does_not_support_viewless_timeseries_yet

View File

@ -22,6 +22,9 @@ selector:
# These workloads use a verbose log level.
- jstests/concurrency/fsm_workloads/ddl/rename_collection/collection_uuid.js
# TODO BACKPORT-25689: reenable this workload after backport to 8.0.
- jstests/concurrency/fsm_workloads/query/remove/update_and_batched_delete.js
exclude_with_any_tags:
- requires_standalone
# The ability to shut down a node while its in the middle of applying ops is required for

View File

@ -1,6 +1,9 @@
test_kind: js_test
selector:
exclude_files:
# TODO: SERVER-109999 Remove this line.
- src/mongo/db/modules/enterprise/jstests/external_auth_oidc_azure/oidc_e2e_azure_machine.js
roots:
- src/mongo/db/modules/enterprise/jstests/external_auth_oidc_azure/*.js

View File

@ -53,6 +53,7 @@ selector:
- jstests/core/query/resume_query_from_non_existent_record.js
- jstests/core/query/resume_query.js
- jstests/core/timeseries/query/timeseries_resume_after.js
- jstests/core/query/start_at_blocked_in_sbe.js
# This test creates collections directly using applyOps commands. So those collections are not
# tracked by sharding and therefore not discoverable by listCollections commands that go through
# the router. TODO (SERVER-84342): listCollections command doesn't return collections created

View File

@ -48,6 +48,7 @@ selector:
- jstests/core/query/resume_query_from_non_existent_record.js
- jstests/core/query/resume_query.js
- jstests/core/timeseries/query/timeseries_resume_after.js
- jstests/core/query/start_at_blocked_in_sbe.js
# This test creates collections directly using applyOps commands. So those collections are not
# tracked by sharding and therefore not discoverable by listCollections commands that go through
# the router. TODO (SERVER-84342): listCollections command doesn't return collections created

View File

@ -278,6 +278,7 @@ selector:
- jstests/sharding/query/change_streams/change_stream_no_shards.js
- jstests/sharding/query/change_streams/change_stream_on_system_collection.js
- jstests/sharding/query/change_streams/change_stream_transaction_sharded.js
- jstests/sharding/query/change_streams/change_stream_transaction_sharded_commit_timestamp.js
- jstests/sharding/query/change_streams/change_stream_update_lookup_collation.js
- jstests/sharding/query/change_streams/change_stream_update_lookup_read_concern.js
- jstests/sharding/query/change_streams/change_streams.js

View File

@ -268,6 +268,7 @@ selector:
# TODO(SERVER-85322): Try to include this test(s).
- jstests/core/query/resume_query.js
- jstests/core/query/resume_query_from_non_existent_record.js
- jstests/core/query/start_at_blocked_in_sbe.js
# Inserting into a time-series collection does not replace top-level empty timestamps.
- jstests/core/write/ts1.js
# Cursor manipulation incompatible with count 'applySkipLimit' option.

View File

@ -3,7 +3,7 @@
import collections
import datetime
import itertools
import os.path
import os
import time
import buildscripts.resmokelib.setup_multiversion.config as multiversion_config
@ -817,3 +817,6 @@ VALIDATE_SELECTOR_PATHS = True
# If set, resmoke.py will set Testdata.pauseAfterPopulate to allow tests that check this
# flag to pause after populating their initial datasets.
PAUSE_AFTER_POPULATE = None
# Whether ASAN (AddressSanitizer) is enabled, determined by the presence of ASAN_OPTIONS.
IS_ASAN = bool(os.environ.get("ASAN_OPTIONS"))

View File

@ -56,6 +56,12 @@ If the min/max is not inclusive, this is added as a note above the parameter.
config_fuzzer_params = {
"mongod": {
"analyzeShardKeyNumRanges": {
"min": 2,
"max": 100,
"period": 5,
"fuzz_at": ["startup", "runtime"],
},
"analyzeShardKeySplitPointExpirationSecs": {"min": 1, "max": 300, "fuzz_at": ["startup"]},
"collectionSamplingLogIntervalSeconds": {
"min": 5,
@ -495,6 +501,17 @@ config_fuzzer_params = {
"period": 5,
"fuzz_at": ["startup", "runtime"],
},
"preAuthMaximumMessageSizeBytes": {
"min": 65536, # 64 KiB
"max": 16777216, # 16 MiB
"period": 10,
"fuzz_at": ["startup", "runtime"],
},
"capMemoryConsumptionForPreAuthBuffers": {
"min": 80,
"max": 100,
"fuzz_at": ["startup"],
},
},
"mongos": {
# We need a higher timeout to account for test slowness
@ -539,6 +556,17 @@ config_fuzzer_params = {
"choices": [{"mode": "off"}],
"fuzz_at": ["startup"],
},
"preAuthMaximumMessageSizeBytes": {
"min": 65536, # 64 KiB
"max": 16777216, # 16 MiB
"period": 10,
"fuzz_at": ["startup", "runtime"],
},
"capMemoryConsumptionForPreAuthBuffers": {
"min": 80,
"max": 100,
"fuzz_at": ["startup"],
},
},
}

View File

@ -823,7 +823,7 @@ class GDBDumper(Dumper):
text=True,
)
regex = re.search("Core was generated by `(.*)'.", process.stdout)
regex = re.search("Core was generated by `(.*)'.", process.stdout, re.DOTALL)
if not regex:
raise RuntimeError("gdb output did not match pattern, could not find binary name")

View File

@ -22,20 +22,18 @@ BACKPORTS_REQUIRED_FILE = "backports_required_for_multiversion_tests.yml"
BACKPORTS_REQUIRED_BASE_URL = "https://raw.githubusercontent.com/10gen/mongo"
def get_backports_required_hash_for_shell_version(mongo_shell_path: str | None = None):
"""Parse the old shell binary to get the commit hash."""
def get_backports_required_hash(mongod_path: str | None = None):
"""Parse the old binary to get the commit hash."""
env_vars = os.environ.copy()
paths = get_path_env_var(env_vars=env_vars)
env_vars["PATH"] = os.pathsep.join(paths)
mongo_shell = mongo_shell_path
mongod = mongod_path
if is_windows():
mongo_shell = mongo_shell_path + ".exe"
mongod = mongod_path + ".exe"
shell_version = check_output(f"{mongo_shell} --version", shell=True, env=env_vars).decode(
"utf-8"
)
for line in shell_version.splitlines():
version = check_output(f"{mongod} --version", shell=True, env=env_vars).decode("utf-8")
for line in version.splitlines():
if "gitVersion" in line:
version_line = line.split(":")[1]
# We identify the commit hash as the string enclosed by double quotation marks.
@ -50,9 +48,7 @@ def get_backports_required_hash_for_shell_version(mongo_shell_path: str | None =
return commit_hash
else:
break
raise ValueError(
f"Could not find a valid commit hash from the {mongo_shell_path} mongo binary."
)
raise ValueError(f"Could not find a valid commit hash from the {mongod_path} mongo binary.")
def get_git_file_content(commit_hash: str) -> str:
@ -112,14 +108,12 @@ def generate_exclude_yaml(old_bin_version: str, output: str, logger: logging.Log
# mongo shell executable.
from buildscripts.resmokelib import multiversionconstants
shell_version = {
MultiversionOptions.LAST_LTS: multiversionconstants.LAST_LTS_MONGO_BINARY,
MultiversionOptions.LAST_CONTINUOUS: multiversionconstants.LAST_CONTINUOUS_MONGO_BINARY,
old_mongod = {
MultiversionOptions.LAST_LTS: multiversionconstants.LAST_LTS_MONGOD_BINARY,
MultiversionOptions.LAST_CONTINUOUS: multiversionconstants.LAST_CONTINUOUS_MONGOD_BINARY,
}[old_bin_version]
old_version_commit_hash = get_backports_required_hash_for_shell_version(
mongo_shell_path=shell_version
)
old_version_commit_hash = get_backports_required_hash(old_mongod)
# Get the yaml contents from the old commit.
logger.info(f"Downloading file from commit hash of old branch {old_version_commit_hash}")

View File

@ -217,7 +217,7 @@ class Fixture(object, metaclass=registry.make_registry_metaclass(_FIXTURES)):
def mongo_client(
self, read_preference=pymongo.ReadPreference.PRIMARY, timeout_millis=30000, **kwargs
):
) -> pymongo.MongoClient:
"""Return a pymongo.MongoClient connecting to this fixture with specified 'read_preference'.
The PyMongo driver will wait up to 'timeout_millis' milliseconds
@ -524,7 +524,9 @@ def create_fixture_table(fixture):
return "Fixture status:\n" + table
def build_client(node, auth_options=None, read_preference=pymongo.ReadPreference.PRIMARY):
def build_client(
node, auth_options=None, read_preference=pymongo.ReadPreference.PRIMARY, **kwargs
) -> pymongo.MongoClient:
"""Authenticate client for the 'authenticationDatabase' and return the client."""
if auth_options is not None:
return node.mongo_client(
@ -533,9 +535,10 @@ def build_client(node, auth_options=None, read_preference=pymongo.ReadPreference
authSource=auth_options["authenticationDatabase"],
authMechanism=auth_options["authenticationMechanism"],
read_preference=read_preference,
**kwargs,
)
else:
return node.mongo_client(read_preference=read_preference)
return node.mongo_client(read_preference=read_preference, **kwargs)
# Represents a row in a node info table.

View File

@ -587,3 +587,4 @@ def _add_testing_set_parameters(suite_set_parameters):
# The placeholder is needed so older versions don't have this option won't have this value set.
suite_set_parameters.setdefault("backtraceLogFile", True)
suite_set_parameters.setdefault("disableTransitionFromLatestToLastContinuous", False)
suite_set_parameters.setdefault("oplogApplicationEnforcesSteadyStateConstraints", True)

View File

@ -341,7 +341,7 @@ class _AddRemoveShardThread(threading.Thread):
if err.code == self._ILLEGAL_OPERATION:
if "Can't move an internal resharding collection" in str(err):
return True
if "Can't reshard a timeseries collection" in str(err):
if "Can't register a temporary collection" in str(err):
return True
for regex in self._UNMOVABLE_NAMESPACE_REGEXES:
if re.search(regex, namespace):

View File

@ -7,7 +7,7 @@ import sys
import threading
import time
from buildscripts.resmokelib import errors
from buildscripts.resmokelib import config, errors
from buildscripts.resmokelib.mongo_fuzzer_configs import generate_normal_mongo_parameters
from buildscripts.resmokelib.testing.fixtures import interface as fixture_interface
from buildscripts.resmokelib.testing.fixtures import replicaset, shardedcluster, standalone
@ -23,6 +23,13 @@ def validate_runtime_parameter_spec(spec):
)
def build_client(node, auth_options):
"""Build a pymongo MongoClient for the given node with the given auth options."""
if config.IS_ASAN:
return fixture_interface.build_client(node, auth_options, timeout_millis=120000)
return fixture_interface.build_client(node, auth_options)
class RuntimeParametersState:
"""Encapsulates the runtime-state of a set of parameters we are fuzzing. Tracks the last time we set a parameter value and holds
the logic for generating new values."""
@ -222,7 +229,7 @@ class FuzzRuntimeParameters(interface.Hook):
def _invoke_get_parameter_and_log(self, node):
"""Helper to print the current state of a node's runtime-fuzzable parameters. Only usable once before_suite has initialized the runtime state of the parameters."""
client = fixture_interface.build_client(node, self._auth_options)
client = build_client(node, self._auth_options)
params_to_get = (
self._mongos_param_state.get_spec()
if client.is_mongos
@ -366,9 +373,7 @@ class _SetParameterThread(threading.Thread):
mongod_params_to_set,
)
for node in repl_set.nodes:
invoke_set_parameter(
fixture_interface.build_client(node, self._auth_options), mongod_params_to_set
)
invoke_set_parameter(build_client(node, self._auth_options), mongod_params_to_set)
for standalone in self._standalone_fixtures:
self.logger.info(
@ -376,9 +381,7 @@ class _SetParameterThread(threading.Thread):
standalone.port,
mongod_params_to_set,
)
invoke_set_parameter(
fixture_interface.build_client(standalone, self._auth_options), mongod_params_to_set
)
invoke_set_parameter(build_client(standalone, self._auth_options), mongod_params_to_set)
for mongos in self._mongos_fixtures:
self.logger.info(
@ -386,6 +389,4 @@ class _SetParameterThread(threading.Thread):
mongos.port,
mongos_params_to_set,
)
invoke_set_parameter(
fixture_interface.build_client(mongos, self._auth_options), mongos_params_to_set
)
invoke_set_parameter(build_client(mongos, self._auth_options), mongos_params_to_set)

View File

@ -11,6 +11,7 @@ import pymongo.mongo_client
from pymongo.collection import Collection
from pymongo.database import Database
from buildscripts.resmokelib import config
from buildscripts.resmokelib.testing.fixtures.external import ExternalFixture
from buildscripts.resmokelib.testing.fixtures.interface import build_client
from buildscripts.resmokelib.testing.fixtures.standalone import MongoDFixture
@ -131,7 +132,12 @@ def validate_node(
auth_options = None
if shell_options and "authenticationMechanism" in shell_options:
auth_options = shell_options
client = build_client(node, auth_options, pymongo.ReadPreference.PRIMARY_PREFERRED)
if config.IS_ASAN:
client = build_client(
node, auth_options, pymongo.ReadPreference.PRIMARY_PREFERRED, timeout_millis=120000
)
else:
client = build_client(node, auth_options, pymongo.ReadPreference.PRIMARY_PREFERRED)
# Skip validating collections for arbiters.
admin_db = client.get_database("admin")

View File

@ -3,6 +3,7 @@
import argparse
import hashlib
import os
import shutil
import sys
import tempfile
import time
@ -22,6 +23,7 @@ def read_sha_file(filename):
content = f.read()
return content.strip().split()[0]
def _fetch_remote_sha256_hash(s3_path: str):
downloaded = False
result = None
@ -40,7 +42,7 @@ def _fetch_remote_sha256_hash(s3_path: str):
if downloaded:
result = read_sha_file(tempfile_name)
if tempfile_name and os.path.exists(tempfile_name):
os.unlink(tempfile_name)
@ -61,7 +63,7 @@ def _verify_s3_hash(s3_path: str, local_path: str, expected_hash: str) -> None:
raise ValueError(
f"Hash mismatch for {s3_path}, expected {expected_hash} but got {hash_string}"
)
print(f"File is valid: {local_path} (sha256: {expected_hash})")
def validate_file(s3_path, output_path, remote_sha_allowed):
hexdigest = S3_SHA256_HASHES.get(s3_path)
@ -69,7 +71,7 @@ def validate_file(s3_path, output_path, remote_sha_allowed):
print(f"Validating against hard coded sha256: {hexdigest}")
_verify_s3_hash(s3_path, output_path, hexdigest)
return True
if not remote_sha_allowed:
raise ValueError(f"No SHA256 hash available for {s3_path}")
@ -82,13 +84,13 @@ def validate_file(s3_path, output_path, remote_sha_allowed):
print(f"Validating against remote sha256 {hexdigest}\n({s3_path}.sha256)")
else:
print(f"Failed to download remote sha256 at {s3_path}.sha256)")
if hexdigest:
_verify_s3_hash(s3_path, output_path, hexdigest)
return True
else:
raise ValueError(f"No SHA256 hash available for {s3_path}")
def _download_and_verify(s3_path, output_path, remote_sha_allowed):
for i in range(5):
@ -98,8 +100,9 @@ def _download_and_verify(s3_path, output_path, remote_sha_allowed):
download_from_s3_with_boto(s3_path, output_path)
except Exception:
download_from_s3_with_requests(s3_path, output_path)
validate_file(s3_path, output_path, remote_sha_allowed)
break
except Exception:
print("Download failed:")
@ -133,7 +136,15 @@ def download_s3_binary(
with tempfile.NamedTemporaryFile(delete=False) as temp_file:
tempfile_name = temp_file.name
_download_and_verify(s3_path, tempfile_name, remote_sha_allowed)
try:
os.replace(tempfile_name, local_path)
except OSError as e:
if e.errno == 18: # EXDEV cross filesystem error, need to use a mv
shutil.move(tempfile_name, local_path)
else:
raise
print(f"Downloaded and verified {s3_path} -> {local_path}")
return True
except Exception as e:
@ -146,8 +157,6 @@ def download_s3_binary(
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Download and verify S3 binary.")
parser.add_argument("s3_path", help="S3 URL to download from")
parser.add_argument("local_path", nargs="?", help="Optional output file path")

View File

@ -12,6 +12,7 @@ def compute_sha256(file_path: str) -> str:
sha256.update(block)
return sha256.hexdigest()
def write_sha256_file(file_path: str, hash_value: str):
sha256_path = file_path + ".sha256"
file_name = os.path.basename(file_path)
@ -19,6 +20,7 @@ def write_sha256_file(file_path: str, hash_value: str):
f.write(f"{hash_value} {file_name}\n")
print(f"Wrote SHA-256 to {sha256_path}")
def main():
if len(sys.argv) != 2:
print("Usage: sha256sum.py <file>")
@ -32,5 +34,6 @@ def main():
hash_value = compute_sha256(file_path)
write_sha256_file(file_path, hash_value)
if __name__ == "__main__":
main()

View File

@ -0,0 +1,29 @@
load("@rules_python//python:defs.bzl", "py_binary", "py_library")
py_library(
name = "config",
srcs = ["config.py"],
visibility = ["//visibility:public"],
)
py_library(
name = "endorctl_utils",
srcs = ["endorctl_utils.py"],
visibility = ["//visibility:public"],
)
py_binary(
name = "generate_sbom",
srcs = ["generate_sbom.py"],
visibility = ["//visibility:public"],
deps = [
"config",
"endorctl_utils",
],
)
py_binary(
name = "sbom_files_pr",
srcs = ["sbom_files_pr.py"],
visibility = ["//visibility:public"],
)

View File

@ -0,0 +1,5 @@
version: 2.0.0
filters:
- "*":
approvers:
- 10gen/code-review-team-ssdlc

184
buildscripts/sbom/config.py Normal file
View File

@ -0,0 +1,184 @@
#!/usr/bin/env python3
"""generate_sbom.py config. Operational configuration values stored separately from the core code."""
import logging
import re
logger = logging.getLogger("generate_sbom")
logger.setLevel(logging.NOTSET)
# ################ Component Filters ################
# List of Endor Labs SBOM components that must be removed before processing
endor_components_remove = []
# bom-ref prefixes (Endor Labs has been changing them, so add all that we have seen)
prefixes = [
"pkg:c/github.com/",
"pkg:generic/github.com/",
"pkg:github/",
]
components_remove = [
# Endor Labs includes the main component in 'components'. This is not standard, so we remove it.
"10gen/mongo",
# should be pkg:github/antirez/linenoise - waiting on Endor Labs fix
"amokhuginnsson/replxx",
# a transitive dependency of s2 that is not necessary to include
"sparsehash/sparsehash",
]
for component in components_remove:
for prefix in prefixes:
endor_components_remove.append(prefix + component)
# ################ Component Renaming ################
# Endor does not have syntactically valid PURLs for C/C++ packages.
# e.g.,
# Invalid: pkg:c/github.com/abseil/abseil-cpp@20250512.1
# Valid: pkg:github/abseil/abseil-cpp@20250512.1
# Run string replacements to correct for this:
endor_components_rename = [
["pkg:c/sourceware.org/git/valgrind", "pkg:generic/valgrind/valgrind"],
["pkg:generic/sourceware.org/git/valgrind", "pkg:generic/valgrind/valgrind"],
["pkg:generic/zlib.net/zlib", "pkg:github/madler/zlib"],
["pkg:generic/tartarus.org/libstemmer", "pkg:github/snowballstem/snowball"],
["pkg:generic/intel.com/intel-dfp-math", "pkg:generic/intel/IntelRDFPMathLib"],
["pkg:c/git.openldap.org/openldap/openldap", "pkg:generic/openldap/openldap"],
["pkg:generic/github.com/", "pkg:github/"],
["pkg:c/github.com/", "pkg:github/"],
]
# ################ Version Transformation ################
# In some cases we need to transform the version string to strip out tag-related text
# It is unknown what patterns may appear in the future, so we have targeted (not broad) regex
# This a list of 'pattern' and 'repl' inputs to re.sub()
RE_VER_NUM = r"(0|[1-9]\d*)"
RE_VER_LBL = r"(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?"
RE_SEMVER = rf"{RE_VER_NUM}\.{RE_VER_NUM}\.{RE_VER_NUM}{RE_VER_LBL}"
regex_semver = re.compile(RE_SEMVER)
VERSION_PATTERN_REPL = [
# 'debian/1.28.1-1' pkg:github/mongodb/mongo-c-driver (temporary workaround)
[re.compile(rf"^debian/({RE_SEMVER})-\d$"), r"\1"],
# 'gperftools-2.9.1' pkg:github/gperftools/gperftools
# 'mongo/v1.5.2' pkg:github/google/benchmark
# 'mongodb-8.2.0-alpha2' pkg:github/wiredtiger/wiredtiger
# 'release-1.12.0' pkg:github/apache/avro
# 'yaml-cpp-0.6.3' pkg:github/jbeder/yaml-cpp
[re.compile(rf"^[-a-z]+[-/][vr]?({RE_SEMVER})$"), r"\1"],
# 'asio-1-34-2' pkg:github/chriskohlhoff/asio
# 'cares-1_27_0' pkg:github/c-ares/c-ares
[
re.compile(rf"^[a-z]+-{RE_VER_NUM}[_-]{RE_VER_NUM}[_-]{RE_VER_NUM}{RE_VER_LBL}$"),
r"\1.\2.\3",
],
# 'pcre2-10.40' pkg:github/pcre2project/pcre2
[re.compile(rf"^[a-z0-9]+-({RE_VER_NUM}\.{RE_VER_NUM})$"), r"\1"],
# 'icu-release-57-1' pkg:github/unicode-org/icu
[re.compile(rf"^[a-z]+-?[a-z]+-{RE_VER_NUM}-{RE_VER_NUM}$"), r"\1.\2"],
# 'v2.6.0' pkg:github/confluentinc/librdkafka
# 'r2.5.1'
[re.compile(rf"^[rv]({RE_SEMVER})$"), r"\1"],
# 'v2025.04.21.00' pkg:github/facebook/folly
[re.compile(r"^v(\d+\.\d+\.\d+\.\d+)$"), r"\1"],
]
def get_semver_from_release_version(release_ver: str) -> str:
"""Extract the version number from string with tags or other annotations"""
if release_ver:
for re_obj, repl in VERSION_PATTERN_REPL:
if re_obj.match(release_ver):
return re_obj.sub(repl, release_ver)
return release_ver
# region special component use-case functions
def get_version_from_wiredtiger_release_info(wt_dir: str) -> str:
"""Get version from 'RELEASE_INFO' file in the wiredtiger folder"""
import os
v = {}
try:
for l in open(os.path.join(wt_dir, "RELEASE_INFO"), "r", encoding="utf-8"):
if re.match(r"WIREDTIGER_VERSION_(?:MAJOR|MINOR|PATCH)=", l):
exec(l, v)
wt_ver = "%d.%d.%d" % (
v["WIREDTIGER_VERSION_MAJOR"],
v["WIREDTIGER_VERSION_MINOR"],
v["WIREDTIGER_VERSION_PATCH"],
)
return wt_ver
except Exception as e:
logger.error(f"Error loading file from {wt_dir}")
logger.error(e)
return None
def get_version_sasl_from_workspace(file_path: str) -> str:
"""Determine the version that is pulled for Windows Cyrus SASL by searching WORKSPACE.bazel"""
# e.g.,
# "https://s3.amazonaws.com/boxes.10gen.com/build/windows_cyrus_sasl-2.1.28.zip",
try:
with open(file_path, "r") as file:
for line in file:
if line.strip().startswith(
'"https://s3.amazonaws.com/boxes.10gen.com/build/windows_cyrus_sasl-'
):
return line.strip().split("windows_cyrus_sasl-")[1].split(".zip")[0]
except Exception as e:
logger.warning(f"Unable to load {file_path}")
logger.warning(e)
else:
return None
def process_component_special_cases(
component_key: str, component: dict, versions: dict, repo_root: str
) -> None:
"""Handle special cases for specific components."""
## Special case for Cyrus SASL ##
if component_key == "pkg:github/cyrusimap/cyrus-sasl":
# Cycrus SASL is optionally loaded as a Windows library, when needed. There is no source code for Endor Labs to scan.
# The version of Cyrus SASL that is used is defined in the WORKSPACE.bazel file:
# "https://s3.amazonaws.com/boxes.10gen.com/build/windows_cyrus_sasl-2.1.28.zip",
# Rather than add the complexity of Bazel queries to this script, we just search the text.
versions["import_script"] = get_version_sasl_from_workspace(repo_root + "/WORKSPACE.bazel")
logger.info(
f"VERSION SPECIAL CASE: {component_key}: Found version '{versions['import_script']}' in 'WORKSPACE.bazel' file"
)
## Special case for wiredtiger ##
elif component_key == "pkg:github/wiredtiger/wiredtiger":
# MongoDB release branches import wiredtiger commits via a bot. These commits will likely not line up with a release or tag.
# Endor labs will try to pull the nearest release/tag, but we want the more precise commit hash, which is stored in:
# src/third_party/wiredtiget/import.data
occurrences = component.get("evidence", {}).get("occurrences", [])
if occurrences:
location = occurrences[0].get("location")
versions["import_script"] = get_version_from_wiredtiger_release_info(
f"{repo_root}/{location}"
)
logger.info(
f"VERSION SPECIAL CASE: {component_key}: Found version '{versions['import_script']}' in 'RELEASE_INFO' file"
)
## Special case for opentelemetry-cpp ##
elif component_key == "pkg:github/open-telemetry/opentelemetry-cpp":
# The opentelementry import script has the mongodb-forks version ref in Major.Minor format (e.g., 1.17), which deviates from
# what the open-telemetry/opentelemetry-cpp project uses (v{SEMVER}). This corrects the version string by adding a '.0', if needed
if re.match(rf"^{RE_VER_NUM}\.{RE_VER_NUM}$", versions["import_script"]):
versions["import_script"] += ".0"
logger.info(
f"VERSION SPECIAL CASE: {component_key}: Adjusted import script version string to semver format: '{versions['import_script']}'"
)
# endregion special component use-case functions

View File

@ -0,0 +1,486 @@
#!/usr/bin/env python3
"""
Utility functions for the Endor Labs API via endorctl
"""
import json
import logging
import subprocess
import time
from datetime import datetime
from enum import Enum
logger = logging.getLogger("generate_sbom")
logger.setLevel(logging.NOTSET)
default_field_masks = {
"PackageVersion": [
"context",
"meta",
"processing_status",
"spec.package_name",
"spec.resolved_dependencies.dependencies",
"spec.source_code_reference",
],
"ScanResult": [
"context",
"meta",
"spec.end_time",
"spec.logs",
"spec.refs",
"spec.start_time",
"spec.status",
"spec.versions",
],
}
def _get_default_field_mask(kind):
default_field_mask = default_field_masks.get(kind, [])
return ",".join(default_field_mask)
class EndorResourceKind(Enum):
"""Enumeration for Endor Labs API resource kinds"""
PROJECT = "Project"
REPOSITORY_VERSION = "RepositoryVersion"
SCAN_RESULT = "ScanResult"
PACKAGE_VERSION = "PackageVersion"
class EndorContextType(Enum):
"""Most objects include a common nested object called Context. Contexts keep objects from different scans separated.
https://docs.endorlabs.com/rest-api/using-the-rest-api/data-model/common-fields/#context"""
# Objects from a scan of the default branch. All objects in the OSS namespace are in the main context. The context ID is always default.
MAIN = "CONTEXT_TYPE_MAIN"
# Objects from a scan of a specific branch. The context ID is the branch reference name.
REF = "CONTEXT_TYPE_REF"
# Objects from a PR scan. The context ID is the PR UUID. Objects in this context are deleted after 30 days.
CI_RUN = "CONTEXT_TYPE_CI_RUN"
class EndorFilter:
"""Provide standard filters for Endor Labs API resource kinds"""
def __init__(self, context_id=None, context_type=None):
self.context_id = context_id
self.context_type = context_type
def _base_filters(self):
base_filters = []
if self.context_id:
base_filters.append(f"context.id=={self.context_id}")
if self.context_type:
base_filters.append(f"context.type=={self.context_type}")
return base_filters
def repository_version(self, project_uuid=None, sha=None, ref=None):
filters = self._base_filters()
if project_uuid:
filters.append(f"meta.parent_uuid=={project_uuid}")
if sha:
filters.append(f"spec.version.sha=={sha}")
if ref:
filters.append(f"spec.version.ref=={ref}")
return " and ".join(filters)
def package_version(
self,
context_type: EndorContextType = None,
context_id=None,
project_uuid=None,
name=None,
package_name=None,
):
filters = self._base_filters()
if context_type:
filters.append(f"context.type=={context_type.value}")
if context_type:
filters.append(f"context.id=={context_id}")
if project_uuid:
filters.append(f"spec.project_uuid=={project_uuid}")
if name:
filters.append(f"spec.package_name=={name}")
if package_name:
filters.append(f"meta.name=={package_name}")
return " and ".join(filters)
def scan_result(
self,
context_type: EndorContextType = None,
project_uuid=None,
ref=None,
sha=None,
status=None,
):
filters = self._base_filters()
if context_type:
filters.append(f"context.type=={context_type.value}")
if project_uuid:
filters.append(f"meta.parent_uuid=={project_uuid}")
if ref:
filters.append(f"spec.versions.ref contains '{ref}'")
if sha:
filters.append(f"spec.versions.sha contains '{sha}'")
if status:
filters.append(f"spec.status=={status}")
return " and ".join(filters)
class EndorCtl:
"""Interact with endorctl (Endor Labs CLI)"""
# region internal functions
def __init__(
self,
namespace,
retry_limit=5,
sleep_duration=30,
endorctl_path="endorctl",
config_path=None,
):
self.namespace = namespace
self.retry_limit = retry_limit
self.sleep_duration = sleep_duration
self.endorctl_path = endorctl_path
self.config_path = config_path
def _call_endorctl(self, command, subcommand, **kwargs):
"""https://docs.endorlabs.com/endorctl/"""
try:
command = [self.endorctl_path, command, subcommand, f"--namespace={self.namespace}"]
if self.config_path:
command.append(f"--config-path={self.config_path}")
# parse args into flags
for key, value in kwargs.items():
# Handle endorctl flags with hyphens that are defined in the script with underscores
flag = key.replace("_", "-")
if value:
command.append(f"--{flag}={value}")
logger.info("Running: %s", " ".join(command))
result = subprocess.run(command, capture_output=True, text=True, check=True)
resource = json.loads(result.stdout)
except subprocess.CalledProcessError as e:
logger.error(f"Error executing command: {e}")
logger.error(e.stderr)
except json.JSONDecodeError as e:
logger.error(f"Error decoding JSON: {e}")
logger.error(f"Stdout: {result.stdout}")
except FileNotFoundError as e:
logger.error(f"FileNotFoundError: {e}")
logger.error(
f"'endorctl' not found in path '{self.endorctl_path}'. Supply the correct path, run 'buildscripts/install_endorctl.sh' or visit https://docs.endorlabs.com/endorctl/install-and-configure/"
)
except Exception as e:
logger.error(f"An unexpected error occurred: {e}")
else:
return resource
def _api_get(self, resource, **kwargs):
"""https://docs.endorlabs.com/endorctl/commands/api/"""
return self._call_endorctl("api", "get", resource=resource, **kwargs)
def _api_list(self, resource, filter=None, retry=True, **kwargs):
"""https://docs.endorlabs.com/endorctl/commands/api/"""
# If this script is run immediately after making a commit, Endor Labs will likely not yet have created the assocaited ScanResult object. The wait/retry logic below handles this scenario.
tries = 0
while True:
tries += 1
result = self._call_endorctl("api", "list", resource=resource, filter=filter, **kwargs)
# The expected output of 'endorctl api list' is: { "list": { "objects": [...] } }
# We want to just return the objects. In case we get an empty list, return a list
# with a single None to avoid having to handle index errors downstream.
if result and result["list"].get("objects") and len(result["list"]["objects"]) > 0:
return result["list"]["objects"]
elif retry:
logger.info(
f"API LIST: Resource not found: {resource} with filter '{filter}' in namespace '{self.namespace}'"
)
if tries <= self.retry_limit:
logger.info(
f"API LIST: Waiting for {self.sleep_duration} seconds before retry attempt {tries} of {self.retry_limit}"
)
time.sleep(self.sleep_duration)
else:
logger.warning(
f"API LIST: Maximum number of allowed retries {self.retry_limit} attempted with no {resource} found using filter '{filter}'"
)
return [None]
else:
return [None]
def _check_resource(self, resource, resource_description) -> None:
if not resource:
raise LookupError(f"Resource not found: {resource_description}")
logger.info(f"Retrieved: {resource_description}")
# endregion internal functions
# region resource functions
def get_resource(self, resource, uuid=None, name=None, field_mask=None, **kwargs):
"""https://docs.endorlabs.com/rest-api/using-the-rest-api/data-model/resource-kinds/"""
if not field_mask:
field_mask = _get_default_field_mask(resource)
return self._api_get(
resource=resource, uuid=uuid, name=name, field_mask=field_mask, **kwargs
)
def get_resources(
self,
resource,
filter=None,
field_mask=None,
sort_path="meta.create_time",
sort_order="descending",
retry=True,
**kwargs,
):
"""https://docs.endorlabs.com/rest-api/using-the-rest-api/data-model/resource-kinds/"""
if not field_mask:
field_mask = _get_default_field_mask(resource)
return self._api_list(
resource=resource,
filter=filter,
field_mask=field_mask,
sort_path=sort_path,
sort_order=sort_order,
retry=retry,
**kwargs,
)
def get_project(self, git_url):
resource_kind = EndorResourceKind.PROJECT.value
resource_description = (
f"{resource_kind} with name '{git_url}' in namespace '{self.namespace}'"
)
project = self.get_resource(resource_kind, name=git_url)
self._check_resource(project, resource_description)
return project
def get_repository_version(self, filter=None, retry=True):
resource_kind = EndorResourceKind.REPOSITORY_VERSION.value
resource_description = (
f"{resource_kind} with filter '{filter}' in namespace '{self.namespace}'"
)
repository_version = self.get_resources(
resource_kind, filter=filter, retry=retry, page_size=1
)[0]
self._check_resource(repository_version, resource_description)
return repository_version
def get_scan_result(self, filter=None, retry=True):
resource_kind = EndorResourceKind.SCAN_RESULT.value
resource_description = (
f"{resource_kind} with filter '{filter}' in namespace '{self.namespace}'"
)
scan_result = self.get_resources(resource_kind, filter=filter, retry=retry, page_size=1)[0]
self._check_resource(scan_result, resource_description)
uuid = scan_result.get("uuid")
start_time = scan_result["spec"].get("start_time")
refs = scan_result["spec"].get("refs")
polling_start_time = datetime.now()
while True:
status = scan_result["spec"].get("status")
end_time = scan_result["spec"].get("end_time")
if status == "STATUS_SUCCESS":
logger.info(
f" Scan completed successfully. ScanResult uuid {uuid} for refs {refs} started at {start_time}, ended at {end_time}."
)
return scan_result
elif status == "STATUS_RUNNING":
logger.info(
f" Scan is running. ScanResult uuid {uuid} for refs {refs} started at {start_time}."
)
logger.info(
f" Waiting {self.sleep_duration} seconds before checking status. Total wait time: {(datetime.now() - polling_start_time).total_seconds()/60:.2f} minutes"
)
time.sleep(self.sleep_duration)
scan_result = self.get_resources(
resource_kind, filter=filter, retry=retry, page_size=1
)[0]
elif status == "STATUS_PARTIAL_SUCCESS":
scan_logs = scan_result["spec"].get("logs")
raise RuntimeError(
f" Scan completed, but with critical warnings or errors. ScanResult uuid {uuid} for refs {refs} started at {start_time}, ended at {end_time}. Scan logs: {scan_logs}"
)
elif status == "STATUS_FAILURE":
scan_logs = scan_result["spec"].get("logs")
raise RuntimeError(
f" Scan failed. ScanResult uuid {uuid} for refs {refs} started at {start_time}, ended at {end_time}. Scan logs: {scan_logs}"
)
def get_package_versions(self, filter):
resource_kind = EndorResourceKind.PACKAGE_VERSION.value
resource_description = (
f"{resource_kind} with filter '{filter}' in namespace '{self.namespace}'"
)
package_versions = self.get_resources(resource_kind, filter=filter)
self._check_resource(package_versions, resource_description)
return package_versions
def export_sbom(
self,
package_version_uuid=None,
package_version_uuids=None,
package_version_name=None,
app_name=None,
project_name=None,
project_uuid=None,
):
"""Export an SBOM from Endor Labs
Valid parameter sets (other combinations result in an error from 'endorctl'):
Single-Package SBOM:
package_version_uuid
package_version_name
Multi-Package SBOM:
package_version_uuids,app_name
project_uuid,app_name,app_name
project_name,app_name,app_name
https://docs.endorlabs.com/endorctl/commands/sbom/export/
"""
if package_version_uuids:
package_version_uuids = ",".join(package_version_uuids)
return self._call_endorctl(
"sbom",
"export",
package_version_uuid=package_version_uuid,
package_version_uuids=package_version_uuids,
package_version_name=package_version_name,
app_name=app_name,
project_name=project_name,
project_uuid=project_uuid,
)
# endregion resource functions
# region workflow functions
def get_sbom_for_commit(self, git_url: str, commit_sha: str) -> dict:
"""Export SBOM for the PR commit (sha)"""
endor_filter = EndorFilter()
try:
# Project: get uuid
project = self.get_project(git_url)
project_uuid = project["uuid"]
app_name = project["spec"]["git"]["full_name"]
# RepositoryVersion: get the context for the PR scan
endor_filter.context_type = EndorContextType.CI_RUN.value
filter_str = endor_filter.repository_version(project_uuid, commit_sha)
repository_version = self.get_repository_version(filter_str)
context_id = repository_version["context"]["id"]
# ScanResult: wait for a completed scan
endor_filter.context_id = context_id
filter_str = endor_filter.scan_result(project_uuid)
self.get_scan_result(filter_str)
# PackageVersions: get package versions for SBOM
filter_str = endor_filter.package_version(project_uuid)
package_versions = self.get_package_versions(filter_str)
package_version_uuids = [
package_version["uuid"] for package_version in package_versions
]
package_version_names = [
package_version["meta"]["name"] for package_version in package_versions
]
# Export SBOM
sbom = self.export_sbom(package_version_uuids=package_version_uuids, app_name=app_name)
print(
f"Retrieved: CycloneDX SBOM for PackageVersion(s), name: {package_version_names}, uuid: {package_version_uuids}"
)
return sbom
except Exception as e:
print(f"Exception: {e}")
return
def get_sbom_for_branch(self, git_url: str, branch: str) -> dict:
"""Export lastest SBOM for a monitored branch/ref"""
endor_filter = EndorFilter()
try:
# Project: get uuid
project = self.get_project(git_url)
project_uuid = project["uuid"]
app_name = project["spec"]["git"]["full_name"]
# RepositoryVersion: get the context for the latest branch scan
filter_str = endor_filter.repository_version(project_uuid, ref=branch)
repository_version = self.get_repository_version(filter_str)
repository_version_uuid = repository_version["uuid"]
repository_version_ref = repository_version["spec"]["version"]["ref"]
repository_version_sha = repository_version["spec"]["version"]["sha"]
repository_version_scan_object_status = repository_version["scan_object"]["status"]
if repository_version_scan_object_status != "STATUS_SCANNED":
logger.warning(
f"RepositoryVersion (uuid: {repository_version_uuid}, ref: {repository_version_ref}, sha: {repository_version_sha}) scan status is '{repository_version_scan_object_status}' (expected 'STATUS_SCANNED')"
)
# ScanResult: search for a completed scan
filter_str = endor_filter.scan_result(
EndorContextType.MAIN, project_uuid, repository_version_ref, repository_version_sha
)
scan_result = self.get_scan_result(filter_str, retry=False)
project_uuid = scan_result["meta"]["parent_uuid"]
# PackageVersions: get package versions for SBOM
if branch == "master":
context_type = EndorContextType.MAIN
context_id = "default"
else:
context_type = EndorContextType.REF
context_id = branch
filter_str = endor_filter.package_version(context_type, context_id, project_uuid)
package_version = self.get_package_versions(filter_str)[0]
package_version_name = package_version["meta"]["name"]
package_version_uuid = package_version["uuid"]
# Export SBOM
sbom = self.export_sbom(package_version_uuid=package_version_uuid, app_name=app_name)
logger.info(
f"SBOM: Retrieved CycloneDX SBOM for PackageVersion, name: {package_version_name}, uuid {package_version_uuid}"
)
return sbom
except Exception as e:
print(f"Exception: {e}")
return
def get_sbom_for_project(self, git_url: str) -> dict:
"""Export latest SBOM for EndorCtl project default branch"""
try:
# Project: get uuid
project = self.get_project(git_url)
project_uuid = project["uuid"]
app_name = project["spec"]["git"]["full_name"]
# Export SBOM
sbom = self.export_sbom(project_uuid=project_uuid, app_name=app_name)
logger.info(f"Retrieved: CycloneDX SBOM for Project {app_name}")
return sbom
except Exception as e:
print(f"Exception: {e}")
return
# endregion workflow functions

1016
buildscripts/sbom/generate_sbom.py Executable file

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,199 @@
#!/bin/bash
set -o errexit
echo "+----------------------------------------------------------------------------+"
echo "| Script to install the Endor Labs CLI and verify authentication |"
echo "| endorctl (https://docs.endorlabs.com/endorctl/) |"
echo "| Environment Variables (optional): |"
echo "| ENDOR_INSTALL_PATH - only if in CI or not installed with homebrew or npm |"
echo "| ENDOR_CONFIG_PATH - endor config directory (default: ~/.endorctl) |"
echo "+----------------------------------------------------------------------------+"
echo
function endorctl_check_install() {
# Check if installed
ENDOR_INSTALLED_PATH=$(command -v endorctl)
if [[ -n "$ENDOR_INSTALLED_PATH" ]]; then
# Is Installed
echo "Binary 'endorctl' is installed in '${ENDOR_INSTALLED_PATH}'."
chmod +x $ENDOR_INSTALLED_PATH
if [[ -x "$ENDOR_INSTALLED_PATH" ]]; then
echo "Binary 'endorctl' is executable."
return 0 # True (success)
else
echo "Binary 'endorctl' is NOT executable after attempting to make it executable."
return 1 # False (failure)
fi
else
echo "Binary 'endorctl' is NOT installed or not in PATH."
return 1 # False (failure)
fi
}
function endorctl_install() {
# Skip trying homebrew and npm if runing in CI
if [[ "$CI" == "true" ]]; then
echo "---------------------------------"
echo "Detected that script is running in CI. Skipping Homebrew and NPM."
else
# Try brew
echo "---------------------------------"
echo "Checking if Homebrew is available"
if command -v brew --version &>/dev/null; then
echo "Attempting to install with Homebrew"
brew tap endorlabs/tap
brew install endorctl
if [ $? -ne 0 ]; then
echo "Warning: Homebrew installation failed."
else
echo "Installed with Homebrew"
return 0 # True (success)
fi
else
echo "Homebrew is not available"
fi
# Try NPM
echo "---------------------------------"
echo "Checking if npm is available"
if command -v npm --version &>/dev/null; then
# Install binary for linux or macos
echo "Attempting to install with npm"
npm install --global endorctl
if [ $? -ne 0 ]; then
echo "Warning: npm installation failed."
else
echo "Installed with npm"
return 0 # True (success)
fi
else
echo "npm is not available"
fi
fi
# Try binary installation
echo "---------------------------------"
echo "Attempting binary install"
if [[ -z "$ENDOR_INSTALL_PATH" ]]; then
ENDOR_INSTALL_PATH="${HOME}/.local/bin"
fi
echo "Installation path set to $ENDOR_INSTALL_PATH"
mkdir -p "$ENDOR_INSTALL_PATH"
export PATH="${ENDOR_INSTALL_PATH}:$PATH"
ENDOR_BIN_PATH="${ENDOR_INSTALL_PATH}/endorctl"
case $(uname -m) in
"x86_64" | "amd64")
ARCH="amd64"
;;
"aarch64" | "arm64")
ARCH="arm64"
;;
*)
echo "Error: Unexpected architecture: $(uname -m). Expected x86_64, amd64, or arm64."
return 1 # False (failure)
;;
esac
case "$OSTYPE" in
linux*)
PLATFORM="linux"
;;
darwin*)
PLATFORM="macos"
;;
msys* | cygwin* | "Windows_NT")
echo "Error: Automated installation on Windows without npm is not implemented in this script."
echo "For manual Windows installation, follow instructions at:"
echo " https://docs.endorlabs.com/endorctl/install-and-configure/#download-and-install-the-endorctl-binary-directly"
echo ""
echo_auth_instructions
return 1 # False (failure)
;;
*)
echo "Error: Unexpected OS type: $OSTYPE"
return 1 # False (failure)
;;
esac
## Download the latest CLI for supported platform and architecture
URL="https://api.endorlabs.com/download/latest/endorctl_${PLATFORM}_${ARCH}"
echo "Downloading latest CLI for $PLATFORM $ARCH to $BIN_PATH from $URL"
curl --silent $URL --output "$ENDOR_BIN_PATH"
## Verify the checksum of the binary
echo "Verifying checksum of binary"
case "$PLATFORM" in
linux)
echo "$(curl -s https://api.endorlabs.com/sha/latest/endorctl_${PLATFORM}_${ARCH})" $ENDOR_BIN_PATH | sha256sum -c
;;
macos)
echo "$(curl -s https://api.endorlabs.com/sha/latest/endorctl_${PLATFORM}_${ARCH})" $ENDOR_BIN_PATH | shasum -a 256 -c
;;
esac
## Modify the permissions of the binary to ensure it is executable
echo " Modifying binary permissions to executable"
chmod +x $ENDOR_BIN_PATH
## Create an alias endorctl of the binary to ensure it is available in other directory
alias endorctl=$ENDOR_BIN_PATH
echo "endorctl installed in $ENDOR_BIN_PATH"
return 0 # True (success)
}
function endorctl_check_auth() {
# Check authentication
echo "Checking authentication with command: endorctl api get --resource Project --namespace mongodb.10gen --name https://github.com/10gen/mongo.git --config-path $ENDOR_CONFIG_PATH"
endorctl api get --resource Project --namespace mongodb.10gen --name "https://github.com/10gen/mongo.git" --config-path $ENDOR_CONFIG_PATH >/dev/null
if [ $? -eq 0 ]; then
echo "Authentication confirmed."
return 0 # True (success)
else
echo "Authentication failure. Command exit code: $?"
echo_auth_instructions
return 1 # False (failure)
fi
}
function echo_auth_instructions() {
echo ""
echo "------------------------------------------------ AUTOMATED AUTH ------------------------------------------------"
echo "Set the following environment variables:"
echo " export ENDOR_API_CREDENTIALS_KEY=<api-key>"
echo " export ENDOR_API_CREDENTIALS_SECRET=<api-key-secret>"
echo " export ENDOR_NAMESPACE=mongodb.{github_org}"
echo ""
echo "--------------------------------------------------- USER AUTH ---------------------------------------------------"
echo "To authenticate endorctl, visit the following URL, authenticate via Okta SSO, and copy the authentication token."
echo " https://api.endorlabs.com/v1/auth/sso?tenant=mongodb.10gen&redirect=headless"
echo "Then run:"
echo " endorctl auth --token [AUTH_TOKEN]"
echo ""
echo "Alternatively, run the init command. Must use headless mode when no GUI is available:"
echo " endorctl init --auth-mode=sso --auth-tenant=mongodb.10gen --headless-mode"
echo ""
echo "Enter 'y' if prompted to overwrite existing configuration and/or delete account keys."
echo ""
echo "If authentication fails, confirm in MANA that you are a member of a '10gen-endor-labs-*' Okta group."
echo ""
}
# Set/Create config folder
if [[ -z "$ENDOR_CONFIG_PATH" ]]; then
ENDOR_CONFIG_PATH="${HOME}/.endorctl"
fi
echo "Config path set to ${ENDOR_CONFIG_PATH}"
if ! endorctl_check_install; then
if ! endorctl_install; then
exit 1
fi
fi
if ! endorctl_check_auth; then
exit 1
fi
exit 0

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,199 @@
#!/usr/bin/env python3
"""
Script that opens a PR using a bot to update SBOM-related files.
"""
import argparse
import os
import re
import time
from github import (
Commit,
GithubException,
GithubIntegration,
GitRef,
PullRequest,
Repository,
)
SBOM_FILES = ["sbom.json", "README.third_party.md"]
def get_repository(github_owner, github_repo, app_id, _private_key) -> Repository.Repository:
"""
Gets the mongo github repository
"""
app = GithubIntegration(int(app_id), _private_key)
installation = app.get_repo_installation(github_owner, github_repo)
g = installation.get_github_for_installation()
return g.get_repo(f"{github_owner}/{github_repo}")
def get_pull_request(branch_gitref: GitRef.GitRef) -> PullRequest.PullRequest | None:
"""
Gets the pull request for the branch ref, if it exists
"""
pulls = branch_gitref
print("get_pull_request:")
for pull in pulls:
print(" pull: ", pull)
if pulls.totalCount > 0:
pull = pulls[0]
print(f"Found open PR #{pull.number} '{pull.title}'")
return pull
else:
return None
def create_branch(base_branch, new_branch) -> None:
"""
Create a new branch or get existing branch.
"""
try:
print(f"Attempting to create branch '{new_branch}' with base branch '{base_branch}'.")
ref = f"refs/heads/{new_branch}"
base_repo_branch = repo.get_branch(base_branch)
sha = base_repo_branch.commit.sha
repo.create_git_ref(ref=ref, sha=sha)
print(f"Created branch '{new_branch}', ref: {ref}, sha: {sha}")
except GithubException as e:
if e.status == 422:
print(f"Branch {new_branch} already exists, ref: {ref}")
else:
raise
def read_text_file(file_path: str) -> str:
"""Read a text file and return as string"""
try:
with open(file_path, "r", encoding="utf-8") as file:
content = file.read()
return content
except FileNotFoundError:
print(f"ERROR: The file '{file_path}' was not found.")
return f"ERROR: The file '{file_path}' was not found."
except Exception as e:
print(f"An error occurred: {e}")
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="This script checks for changes to SBOM and related files and creats a PR if files have been updated.",
)
parser.add_argument("--github-owner", help="GitHub org/owner (e.g., 10gen).", type=str)
parser.add_argument("--github-repo", help="GitHub repository name (e.g., mongo).", type=str)
parser.add_argument("--base-branch", help="base branch to merge into.", type=str)
parser.add_argument("--new-branch", help="New branch for the PR.", type=str)
parser.add_argument("--pr-title", help="Title for the PR.", type=str)
parser.add_argument(
"--saved-warnings", help="Path to file to include as text in PR message.", type=str
)
parser.add_argument(
"--app-id",
help="GitHub App ID used for authentication.",
type=str,
default=os.getenv("MONGO_PR_BOT_APP_ID"),
)
parser.add_argument(
"--private-key",
help="Key to use for GitHub App authentication.",
type=str,
default=os.getenv("MONGO_PR_BOT_PRIVATE_KEY"),
)
args = parser.parse_args()
if not args.app_id or not args.private_key:
parser.error(
"Must define --app-id or env MONGO_PR_BOT_APP_ID and --private-key or env MONGO_PR_BOT_PRIVATE_KEY."
)
# Replace spaces with newline, if applicable
private_key = (
args.private_key[:31] + args.private_key[31:-29].replace(" ", "\n") + args.private_key[-29:]
)
repo = get_repository(args.github_owner, args.github_repo, args.app_id, private_key)
print("repo: ", repo)
HAS_UPDATE = False
for file_path in SBOM_FILES:
original_file = repo.get_contents(file_path, ref=f"refs/heads/{args.base_branch}")
print("original_file: ", original_file)
original_content = original_file.decoded_content.decode()
try:
with open(file_path, "r", encoding="utf-8") as file:
new_content = file.read()
except FileNotFoundError:
print("Error: file '%s' not found.", file_path)
# Compare content with removed Endor Labs version to avoid triggering a new SBOM on only that change
PATTERN = r'{"name":"EndorLabsInc","version":".*"}'
REPL = r'{"name":"EndorLabsInc","version":""}'
original_content_compare = re.sub(PATTERN, REPL, "".join(original_content.split()))
new_content_compare = re.sub(PATTERN, REPL, "".join(new_content.split()))
if original_content_compare != new_content_compare:
create_branch(args.base_branch, args.new_branch)
original_file_new_branch = repo.get_contents(
file_path, ref=f"refs/heads/{args.new_branch}"
)
print("original_file_new_branch: ", original_file_new_branch)
print("New file is different from original file.")
print("repo.update_file:")
print(f" message: Updating '{file_path}'")
print(" path: ", file_path)
print(" sha: ", original_file_new_branch.sha)
print(" content:")
print(new_content[:128])
print("...[truncated]...")
print(new_content[-128:])
print(" branch: ", args.new_branch)
time.sleep(10) # Wait to reduce chance of 409 errors
update_file_result = repo.update_file(
message=f"Updating '{file_path}'",
path=file_path,
sha=original_file_new_branch.sha,
content=new_content,
branch=args.new_branch,
)
print("update_file_result: ", update_file_result)
commit: Commit = update_file_result.get("commit")
print("commit: ", commit)
HAS_UPDATE = True
if HAS_UPDATE:
# Get open PR or create new PR
pull_requests = repo.get_pulls(
state="open", head=f"{args.github_owner}:{args.new_branch}", base=args.base_branch
)
if pull_requests.totalCount:
pull_request = pull_requests[0]
print("pull_request: ", pull_request)
else:
pr_body = "Automated PR updating SBOM and related files."
print("Creating PR:")
print(f" title={args.pr_title}")
print(f" head={args.new_branch}")
print(f" base={args.base_branch}")
print(f" body={pr_body}")
pull_request = repo.create_pull(
title=args.pr_title,
head=args.new_branch,
base=args.base_branch,
body=pr_body,
)
print("pull_request: ", pull_request)
if args.saved_warnings:
pr_comment = "The following warnings were output by the SBOM generation script:\n"
if os.path.isfile(args.saved_warnings):
pr_comment += read_text_file(args.saved_warnings)
comment = pull_request.create_issue_comment(pr_comment)
print("Added PR comment: ", comment)
else:
print(f"Files '{SBOM_FILES}' have not changed. Skipping PR.")

View File

@ -4,9 +4,15 @@ import os
import sys
from typing import List
import jsonschema
from license_expression import get_spdx_licensing
from referencing import Registry, Resource
try:
import jsonschema
except ImportError:
print("'jsonschema' not found. Continuing without it.")
jsonschema = None
BOM_SCHEMA_LOCATION = os.path.join("buildscripts", "tests", "sbom_linter", "bom-1.5.schema.json")
SPDX_SCHEMA_LOCATION = os.path.join("buildscripts", "tests", "sbom_linter", "spdx.schema.json")
SPDX_SCHEMA_REF = "spdx.schema.json"
@ -32,8 +38,9 @@ MISSING_TEAM_ERROR = "Component must include a 'internal:team_responsible' prope
SCHEMA_MATCH_FAILURE = "File did not match the CycloneDX schema"
MISSING_VERSION_IN_SBOM_COMPONENT_ERROR = "Component must include a version."
MISSING_VERSION_IN_IMPORT_FILE_ERROR = "Missing version in the import file: "
MISSING_LICENSE_IN_SBOM_COMPONENT_ERROR = "Component must include a license."
COULD_NOT_FIND_OR_READ_SCRIPT_FILE_ERROR = "Could not find or read the import script file"
VERSION_MISMATCH_ERROR = "Version mismatch: "
VERSION_MISMATCH_ERROR = "Version mismatch (may simply be an artifact of SBOM automation): "
# A class for managing error messages for components
@ -115,30 +122,49 @@ def strip_extra_prefixes(string_with_prefix: str) -> str:
return string_with_prefix.removeprefix("mongo/").removeprefix("v")
def validate_evidence(component: dict, third_party_libs: set, error_manager: ErrorManager) -> None:
if "evidence" not in component or "occurrences" not in component["evidence"]:
error_manager.append_full_error_message(MISSING_EVIDENCE_ERROR)
def validate_license(component: dict, error_manager: ErrorManager) -> None:
if "licenses" not in component:
error_manager.append_full_error_message(MISSING_LICENSE_IN_SBOM_COMPONENT_ERROR)
return
occurrences = component["evidence"]["occurrences"]
if not occurrences:
error_manager.append_full_error_message(
"'evidence.occurrences' field must include at least one location."
)
for occurrence in occurrences:
location = occurrence["location"]
valid_license = False
expression = None
for component_license in component["licenses"]:
if "expression" in component_license:
expression = component_license.get("expression")
elif "license" in component_license:
if "id" in component_license["license"]:
# Should be a valid SPDX license ID
expression = component_license["license"].get("id")
elif "name" in component_license["license"]:
# If SPDX does not define the license used, the name field may be used to provide the license name
valid_license = True
if not os.path.exists(location) and not SKIP_FILE_CHECKING:
error_manager.append_full_error_message("location does not exist in repo.")
if not valid_license:
licensing_validate = get_spdx_licensing().validate(expression, validate=True)
# ExpressionInfo(
# original_expression='',
# normalized_expression='',
# errors=[],
# invalid_symbols=[]
# )
valid_license = not licensing_validate.errors or not licensing_validate.invalid_symbols
if not valid_license:
error_manager.append_full_error_message(licensing_validate)
return
if location.startswith(THIRD_PARTY_LOCATION_PREFIX):
lib = location.removeprefix(THIRD_PARTY_LOCATION_PREFIX)
if lib in third_party_libs:
third_party_libs.remove(lib)
def validate_evidence(component: dict, third_party_libs: set, error_manager: ErrorManager) -> None:
if component.get("scope") == "required":
if "evidence" not in component or "occurrences" not in component["evidence"]:
error_manager.append_full_error_message(MISSING_EVIDENCE_ERROR)
return
validate_location(component, third_party_libs, error_manager)
def validate_properties(component: dict, error_manager: ErrorManager) -> None:
has_team_responsible_property = False
has_team_responsible_property = False or component.get("scope") == "excluded"
script_path = ""
if "properties" in component:
for prop in component["properties"]:
@ -146,43 +172,62 @@ def validate_properties(component: dict, error_manager: ErrorManager) -> None:
has_team_responsible_property = True
elif prop["name"] == "import_script_path":
script_path = prop["value"]
if not has_team_responsible_property:
error_manager.append_full_error_message(MISSING_TEAM_ERROR)
if script_path:
script_path_is_file = os.path.isfile(script_path)
if not script_path_is_file:
error_manager.append_full_error_message(COULD_NOT_FIND_OR_READ_SCRIPT_FILE_ERROR)
# Only look for VERSION if the import script is a shell script file
elif script_path.endswith(".sh"):
script_version = get_script_version(script_path, "VERSION", error_manager)
if script_version == "":
error_manager.append_full_error_message(
MISSING_VERSION_IN_IMPORT_FILE_ERROR + script_path
)
if not component.get("version"):
error_manager.append_full_error_message(MISSING_VERSION_IN_SBOM_COMPONENT_ERROR)
return
comp_version = component["version"]
# If the version is unknown or the script path property is absent, the version
# check is not possible (these are valid options and no error is generated).
if comp_version == "Unknown" or script_path == "":
return
# At this point a version is attempted to be read from the import script file
script_version = get_script_version(script_path, "VERSION", error_manager)
if script_version == "":
error_manager.append_full_error_message(MISSING_VERSION_IN_IMPORT_FILE_ERROR + script_path)
elif strip_extra_prefixes(script_version) != strip_extra_prefixes(comp_version):
error_manager.append_full_error_message(
VERSION_MISMATCH_ERROR
+ f"\nscript version:{script_version}\nsbom version:{comp_version}"
)
def validate_component(component: dict, third_party_libs: set, error_manager: ErrorManager) -> None:
error_manager.update_component_attribute(component["name"])
if "scope" not in component:
error_manager.append_full_error_message("component must include a scope.")
elif component["scope"] != "optional":
else:
validate_evidence(component, third_party_libs, error_manager)
validate_properties(component, error_manager)
validate_license(component, error_manager)
if "purl" not in component and "cpe" not in component:
error_manager.append_full_error_message(MISSING_PURL_CPE_ERROR)
error_manager.update_component_attribute("")
def validate_location(component: dict, third_party_libs: set, error_manager: ErrorManager) -> None:
if "evidence" in component:
if "occurrences" not in component["evidence"]:
error_manager.append_full_error_message(
"'evidence.occurrences' field must include at least one location."
)
occurrences = component["evidence"]["occurrences"]
for occurrence in occurrences:
if "location" in occurrence:
location = occurrence["location"]
if not os.path.exists(location) and not SKIP_FILE_CHECKING:
error_manager.append_full_error_message("location does not exist in repo.")
if location.startswith(THIRD_PARTY_LOCATION_PREFIX):
lib = location.removeprefix(THIRD_PARTY_LOCATION_PREFIX)
if lib in third_party_libs:
third_party_libs.remove(lib)
def lint_sbom(
input_file: str, output_file: str, third_party_libs: set, should_format: bool
) -> ErrorManager:
@ -197,15 +242,16 @@ def lint_sbom(
error_manager.append(f"Failed to parse {input_file}: {str(ex)}")
return error_manager
try:
schema = get_schema()
jsonschema.validators.validator_for(schema)(
schema, registry=local_schema_registry()
).validate(sbom)
except jsonschema.ValidationError as error:
error_manager.append(f"{SCHEMA_MATCH_FAILURE} {input_file}")
error_manager.append(error.message)
return error_manager
if jsonschema:
try:
schema = get_schema()
jsonschema.validators.validator_for(schema)(
schema, registry=local_schema_registry()
).validate(sbom)
except jsonschema.ValidationError as error:
error_manager.append(f"{SCHEMA_MATCH_FAILURE} {input_file}")
error_manager.append(error.message)
return error_manager
components = sbom["components"]
for component in components:
@ -257,10 +303,8 @@ def main() -> int:
)
# the only files in this dir that are not third party libs
third_party_libs.remove("scripts")
# wiredtiger will not be included in the sbom since it is considered part of the server
third_party_libs.remove("wiredtiger")
# the only files in the sasl dir are BUILD files to setup the sasl library in Windows
third_party_libs.remove("sasl")
# Nothing in this directory is included in Community/EA
third_party_libs.remove("private")
error_manager = lint_sbom(input_file, output_file, third_party_libs, should_format)
error_manager.print_errors()

View File

@ -53,7 +53,6 @@ suites:
- src/mongo/db/modules/enterprise/jstests/live_restore/live_restore_server.js
- src/mongo/db/modules/enterprise/jstests/live_restore/live_restore_no_server.js
- jstests/noPassthrough/validate/validate_empty_collection.js
- jstests/noPassthrough/validate/skip_geo_hash_checks.js
- jstests/noPassthrough/validate/validate_memory_limit.js
- jstests/noPassthrough/validate/validate_with_long_index_name.js
- src/mongo/db/modules/enterprise/jstests/live_restore/live_restore_block_fcv_change.js

Some files were not shown because too many files have changed in this diff Show More