mongo/docs/branching/README.md

Branching

This document describes branching task regarding file updates in 10gen/mongo repository that should be done on a new branch immediately after a branch cut.

Table of contents

1. Project settings
2. Create working branch
3. Update files
4. Test changes
5. Merge changes

1. Project settings

GitHub App credentials

Add GitHub app credentials (app id and key) in the new project settings, eg. https://spruce.corp.mongodb.com/project/mongodb-mongo-v8.3/settings/github-app-settings (additional MANA permissions may be required, else coordinate with Release team contacts).

2. Create working branch

To save time during the branch cut these branching changes could be done beforehand, but not too early to avoid extra file conflicts, and then rebased on a new vX.Y branch.

Create a working branch from master or from a new vX.Y branch if it already exists:

git checkout master
git pull
git checkout -b vX.Y-branching-task

2. Update files

IMPORTANT! All of these changes should be a separate commit, but they should be pushed together in the same commit-queue task.

The reason they should be pushed as separate commits is in the case of needing to revert one aspect of this entire task.

See 8.2 branching PR for reference.

Some have some automated steps you can run, but please double-check their edits. Initialize the version here, used throughout:

VERSION=8.3

Copybara configuration

Run the following automation in the private repo and verify results:

sed -i "s/master/v$VERSION/g" buildscripts/copybara/copy.bara.sky buildscripts/copybara/sync_repo_with_copybara.py

In the private repo, buildscripts/copybara/copy.bara.sky and buildscripts/copybara/sync_repo_with_copybara.py should have their "master" branch references replaced with the new branch name.

Evergreen YAML configurations

1. Version expansions

Run the following automation and verify results:

sed -i "s/suffix\"] = \"latest\"/suffix\"] = \"v$VERSION-latest\"/g" buildscripts/generate_version_expansions.py

In the file buildscripts/generate_version_expansions.py, the "latest" suffixes should be replaced with the new branch name.

2. Nightly YAML

etc/evergreen_nightly.yml will be used as YAML configuration in the new mongodb-mongo-vX.Y evergreen project.

This will move some build variants from etc/evergreen.yml to continue running on a new branch project. More information about build variants after branching is here.

• Copy over commit-queue aliases and patch aliases from etc/evergreen.yml
• Update "include" section: comment out or uncomment file includes as instructions in the comments suggest.

3. Burn-in tasks

Run the following automation and verify results:

sed -i '/burn_in_tag_include_build_variants/{N;N;N;d;}' etc/evergreen_yml_components/variants/misc/misc.yml

In the file etc/evergreen_yml_components/variants/misc/misc.yml, build variant names in the "burn_in_tag_include_build_variants" expansion that are not included in etc/evergreen_nightly.yml are removed.

4. Suggested to Required

Run the following automation and verify results:

sed -i 's@display_name: "\* Amazon Linux 2023 arm64 Enterprise"@display_name: "! Amazon Linux 2023 arm64 Enterprise"@g' etc/evergreen_yml_components/variants/amazon/test_dev.yml

sed -i 's/tags: \["suggested", "forbid_tasks_tagged_with_experimental"\]/tags: ["required", "forbid_tasks_tagged_with_experimental"]/g' etc/evergreen_yml_components/variants/amazon/test_dev.yml

For the variant enterprise-amazon-linux2023-arm64 in etc/evergreen_yml_components/variants/amazon/test_dev.yml, replace:

• "*" with "!" in their display names
• "suggested" variant tag with "required"

5. Feature flags

Remove all-feature-flags configuration from build variants

Run the following automation and verify results:

FILES='etc/evergreen_yml_components/variants/sanitizer/test_dev.yml etc/evergreen_yml_components/variants/windows/test_dev.yml'

sed -i 's/ (all feature flags)//g' $FILES
sed -i 's/-all-feature-flags//g' $FILES
sed -i '/--runAllFeatureFlagTests/d' $FILES
sed -i 's/!.incompatible_all_feature_flags/!.requires_all_feature_flags/g' $FILES

For the build variant names:

• in etc/evergreen_yml_components/variants/windows/test_dev.yml:

  • enterprise-windows-all-feature-flags-required
  • enterprise-windows-all-feature-flags-non-essential

• in etc/evergreen_yml_components/variants/sanitizer/test_dev.yml:

  • linux-debug-aubsan-lite-all-feature-flags-required

• It should:

  • Remove all-feature-flags from their names and display names
  • Remove --runAllFeatureFlagTests from test_flags expansion
  • Replace !.incompatible_all_feature_flags tag selectors with !.requires_all_feature_flags

6. Sys-perf YAML

etc/system_perf.yml will be used as YAML configuration for a new sys-perf-X.Y evergreen project

Ensure that DSI has been updated with new branches

Run the following automation and verify results:

sed -i '/evergreen\/system_perf\/master\/master_variants.yml/{N;d;}' etc/system_perf.yml

sed -i "s@evergreen/system_perf/master/compiles.yml@evergreen/system_perf/$VERSION/compiles.yml@g" etc/system_perf.yml
sed -i "s@evergreen/system_perf/master/variants.yml@evergreen/system_perf/$VERSION/variants.yml@g" etc/system_perf.yml

In the file etc/system_perf.yml, the following should be reflected:

• Remove evergreen/system_perf/master/master_variants.yml from "include" section
• With the exception of base.yml, update all other entries that contain master in the path to contain X.Y in the path instead. (e.g. evergreen/system_perf/master/variants.yml should become evergreen/system_perf/X.Y/variants.yml).
• Update the evergreen project variable compile_project in the new sys-perf-X.Y evergreen project to point to the new mongodb-mongo-vX.Y branch

7. Evergreen project validation

Run the following automation and verify results:

sed -i 's/RELEASE_BRANCH = False/RELEASE_BRANCH = True/g' buildscripts/validate_evg_project_config.py

In file buildscripts/validate_evg_project_config.py, the RELEASE_BRANCH variable should be set to True to leverage a specialized shortcut conditional to evaluate the project, not validate.

8. Coverity

Run the following automation and verify results:

sed -i "s/stream: mongo.master/stream: mongo.v$VERSION/g" etc/coverity.yml

In the file etc/coverity.yml, the "stream" should be updated to the new branch.

Finally: format and lint

Run the following automation and verify results:

sed -i "s/DEFAULT_ORIGIN_BRANCH = \"master\"/DEFAULT_ORIGIN_BRANCH = \"v$VERSION\"/g" buildscripts/bazel_rules_mongo/utils/evergreen_git.py

In the file buildscripts/bazel_rules_mongo/utils/evergreen_git.py, DEFAULT_ORIGIN_BRANCH should be updated to the new branch. This makes bazel run format and bazel run lint --fix compare changed files against the new release branch instead of master.

bazel run format && bazel run lint --fix

Run linters and formatters and fix anything that couldn't be autofixed.

3. Test changes

In case working branch was created from master branch, rebase it on a new vX.Y branch and fix file conflicts if any.

Schedule required patch on a new mongodb-mongo-vX.Y project:

evergreen patch -p mongodb-mongo-vX.Y -a required

If patch results reveal that some steps are missing or outdated in this file, make sure to update the branching documentation on a "master" branch accordingly.

4. Merge changes

Open a Github PR to merge to branch vX.Y-staging.