Rendered from docs/how-to/cut-a-release.md in the Headwater corpus. Every document on this half of the site is typed by the taxonomy the descriptor names: corpus.json.

Cut a release

Audience: a contributor to this repository who did not cut the last release. An adopter of Headwater needs no step of this guide, and the consumer surface in .headwater/overlay.yml does not list it. An adopter gets the result: each release has a changelog entry, a binary, the crates on crates.io and an install line that works.

This repository cuts two kinds of release. An engine release ships the headwater binary and the crates. A taxonomy release ships the headwater/standard package. Each kind has its own tag namespace and its own workflows, and you can cut one without the other. The steps below come from the v0.3.0 release of 2026-09-26 and from the workflow files that they name. Where this page and a workflow file disagree, the workflow file is correct and this page is stale.

Before you start

Four workflows under .github/workflows/ take part in a release. A pushed tag starts three of them, and a person starts the fourth by hand.

workflow what starts it what it does
.github/workflows/release.yml v* Builds three archives and a Debian package, runs three smoke jobs on hosts that did not build them, and creates the GitHub release. With the APT_SIGNING_KEY secret set, it also signs the APT metadata.
.github/workflows/publish-crates.yml v* Publishes every workspace crate to crates.io, leaves first, in the order that its order variable states.
.github/workflows/release-taxonomy.yml taxonomy/headwater-standard/v* Publishes taxonomy-source/headwater-standard at the tagged commit and attaches the zip to the release.
.github/workflows/yank-crates.yml workflow_dispatch Yanks the crate versions that a person names. No push and no tag starts it.

sh tools/repo/release-guide-fixtures.sh holds this table against the on: block of each workflow, in both directions. A new release workflow without a row here fails that suite.

A tag matches one of the two patterns at most. So an engine tag never starts the taxonomy workflow, and a taxonomy tag never starts an engine workflow. Both engine workflows take workflow_dispatch with a tag input too. release-taxonomy.yml does the same. Use that input to run a workflow again for a tag that exists already. HW-DR-0088 records why the two namespaces are apart, and HW-PD-0012 records why the workflows are files apart from CI.

Before an engine release, make sure that these conditions are true:

  • The version milestone for the release is closed, and the owner said yes to the release. The RELEASE READY line in .claude/agents/headwater-product-owner.md is where that question starts.
  • CI on main is green.
  • You have push access to tags on headwater-ai/headwater. The crates.io token is the CARGO_REGISTRY_TOKEN secret, and only the workflows use it.

Steps

The engine release

  1. Bump the version on a branch. Change version under [workspace.package] in engine/Cargo.toml. Change the version of each internal crate under [workspace.dependencies] in the same file to the same value. Cargo does not derive these values from [workspace.package] version. A test in engine/crates/cli/tests/publish_order.rs holds each entry at that version, and it names each entry that you did not change.
  2. Update the lock. Run a build with the engine manifest, so that engine/Cargo.lock records the new versions. Commit the lock.
  3. Re-record the fixtures and the records that print the version. Build the engine at the new version. Run the workspace suite with HEADWATER_BLESS=1, as DEVELOPING.md says. Then run headwater generate --root .. The records under docs/probe-results/ state "Graded by grader ", and only headwater generate writes them again. Read the diff. In v0.3.0 this moved six recorded fixtures, and each change was the version string only. In v0.5.0 it also moved two records under docs/probe-results/.
  4. Write the changelog entry. Add an <h2> section for the new version at the top of site/changelog/index.html. Link the release page for the tag, give the date, and name what an adopter meets. The v0.3.0 entry is the model.
  5. Merge the release pull request. It does not change the install text. It changes these classes of file: engine/Cargo.toml, engine/Cargo.lock, the recorded fixtures under engine/crates/*/fixtures/, the records under docs/probe-results/, and site/changelog/index.html. The number of files is different for each release, so this page does not give one. Pull requests #1124 (v0.2.1) and #1169 (v0.3.0) show the shape, and none of their files is an install page.
  6. Choose the commit to tag. Use a commit on main that carries the version bump and that has a green CI run. For v0.3.0 the owner tagged the head of main, and not the merge commit of the release pull request. Both are correct if both conditions are true.
  7. Tag and push. Use an annotated tag:

    git tag -a v<version> -m "Release v<version>"
    git push origin v<version>
    

    The tag starts release.yml and publish-crates.yml. Each one refuses a tag that does not agree with [workspace.package] version. release.yml compares the tag with what the binary prints. publish-crates.yml compares the tag with what cargo metadata reads. HW-PD-0010 records the guard and why it refuses an empty reading. 8. Wait until both workflows are complete, and confirm what they shipped. Do the first three checks under How to know it worked. They confirm that the release has every archive, that crates.io has every crate at the new version, and that the site serves the new package. If a check fails, go to When a step fails and do not continue. For v0.3.0 the first publish attempt stopped 15 minutes after the tag, and the crates were complete 6 minutes after that. 9. Move the install text last. Change each install line from the previous tag to the new tag. To find each line, run this search with the version of the last release, and put a backslash before each dot:

    git grep -nE '(^|[^0-9.])v?<previous>([^0-9]|$)'
    

    For v0.5.0 the pattern is (^|[^0-9.])v?0\.5\.0([^0-9]|$). The search finds the version with a v and without one, in a command and in prose. For the move from v0.4.1 to v0.5.0 in #1395, it found all 19 lines that the move changed, in 35 lines of 13 files. So the search also finds lines that you must not change. Change only a line that installs the engine or names the release that the page installs. Do not change a line that records history, such as an older entry in site/changelog/index.html or a record under docs/probe-results/ or docs/probe-runs/. Do not change a test fixture under tools/, or a lock file of another project. Pull request #1395 is the model. The last group of tools/repo/release-guide-fixtures.sh takes this search from this page. It runs the search on README.md, on the tutorial and its rendered page, on site/index.html, on site/install/index.html and on site/install.sh. The group fails when the search does not find a line that names the release that README.md checks out. Then walk the tutorial at the new binary with sh .claude/tutorial/fixtures.sh. If the walk passes, set last_verified in docs/tutorials/your-first-governed-corpus.md to the date of the walk, and run python3 tools/site/render-tutorial.py. If the walk fails, do not change the date.

    Do not move FIXTURE_TAG in .github/workflows/integrations-headwater-check.yml or in .github/workflows/integrations-headwater-upkeep.yml. integrations/headwater-check/fixtures/build-decisive-fixture.sh runs the binary of that tag. Each FIXTURE_TAG is a test pin, and a release does not move it (#1348).

    Group 4 of tools/repo/readme-fixtures.sh requires that the tag which README.md pins resolves on the remote, or that the page says the tag is not cut. DEVELOPING.md states that exclusive or. That group reads the tag only. It does not read the archives or the crates, so a green CI run on this change is not the confirmation of step 8. Group 11 requires that the install panels of site/install/index.html start with the download lines of README.md, line for line. It also requires that they download the release that README.md checks out. It requires that the default release of site/install.sh is that release too. It requires that site/index.html and the top of site/install/index.html run the line that pipes that script to sh. It also requires that site/install/index.html names the APT sources line, the keyring URL, the keyring path and the package that README.md names. So a change that moves the tag in one file and not in the other fails. If you merge this change before step 8, the download line in README.md can give a 404, and cargo install headwater-cli can install an older version.

    Group 13 of tools/repo/readme-fixtures.sh reads the tutorial and its rendered page. Each engine tag in them must be the tag that README.md downloads and checks out. The "This installs version" line must name that version. Group 13 also requires that the tag in README.md is the newest release tag in the clone, or the tag before it. So CI stays green between the tag push of step 7 and this step. A release that skips this step fails when somebody pushes the next tag.

The asset names follow the pattern headwater-<tag>-<target>.tar.gz, and each archive has a .sha256 file beside it. The matrix in release.yml is the list of targets. Group 7 of tools/repo/readme-fixtures.sh holds that list against README.md, so this page does not copy it.

The release also carries the Debian package headwater_<version>_amd64.deb, where a pre-release hyphen in the version becomes ~. When the APT_SIGNING_KEY secret is set, the release also carries Packages, Release, InRelease and Release.gpg. After the publish job creates the release, the last job of release.yml deploys the site from main. That deploy copies these files into https://headwater.tools/apt/, so no other push to main is necessary. The deploy of the merge in step 5 runs before the release exists, so it serves the release before this one. When the secret is not set, the publish job prints a warning that names it. HW-DR-0094 is the decision, and Rotate or revoke the APT signing subkey is the procedure for the key.

The taxonomy release

The four-step process in the header of .headwater/packages/headwater-standard/package.yml is the maintenance loop. A taxonomy release adds a tag to its result.

  1. Change the version in the source. Set version in taxonomy-source/headwater-standard/package.yml. Set the same version in the from.package line of taxonomy-source/headwater-standard/assemblies/starter/assembly.yml, because the resolver refuses a recipe that pins a different version. Then set it in the recipe and the diagram under An assembly has two consumption forms in spec 7. The test spec_seven_recipe.rs in engine/crates/resolve/tests/ fails until spec 7 prints the recipe that the file declares.
  2. Publish, pin, vendor and resolve. Do the four steps in the order that the header of .headwater/packages/headwater-standard/package.yml gives. Commit the result on a branch and merge it.
  3. Tag the merged commit and push. Use the tag taxonomy/headwater-standard/v<version>. The workflow publishes again at the tagged commit, for the reason in HW-DR-0089. Its release notes state the digest (HW-DR-0090). It refuses a tag whose version is not the version that publish wrote. It creates the release with --latest=false, so the newest engine release stays the latest release, which the site copies its APT repository from (#1449).
  4. Confirm the artifact. Do the last check under How to know it worked: the release has the zip, and its notes state the digest.
  5. Move the pinned taxonomy tag and digest last, and do it at once. Four files name the taxonomy version: README.md, docs/tutorials/your-first-governed-corpus.md, site/tutorial/index.html and .github/assets/headwater-demo.tape. A fifth file, .claude/tutorial/drive.py, reads the version from the tutorial page. standard_pin_files in tools/repo/readme-fixtures.sh is the list of all five. Change the version and the digest only after step 4. Then run python3 tools/site/render-tutorial.py, and record the demo again with the command in the header of the tape. Group 6 of that script reads the digest out of the tree of the pinned tag, so a digest from a different tag fails. Group 8 fails when one of the five files names a version that is not the newest release tag in the clone. It also fails when one of the four files names no version. A pre-release tag such as v4.13.0-rc.1 is not a release tag, so group 8 does not read it. So from the push in step 3, the headwater job of CI is red on every branch until this step merges. This red window is accepted: it lasts for one merge, and the failure of group 8 names the files to change. Do not write version history in these files, such as the first release that had a feature. Group 8 reads each version in them, so a line of history fails it at the next release.

How the two releases depend on each other

The workflows do not depend on each other, and a taxonomy release needs no engine version bump. The one link is requires_engine in taxonomy-source/headwater-standard/package.yml. When a package version raises that floor, cut the engine release that satisfies it first. An engine release depends on no taxonomy release. HW-DR-0088 is the record of this rule.

When a step fails

A crates.io publish stops partway. Run the failed job again. Do not yank. The loop in publish-crates.yml skips each crate that crates.io already has at this version. So a second run starts where the first run stopped. For v0.3.0, the first attempt stopped at headwater-compat, because the crates.io index did not yet list headwater-scaffold after a fixed 30-second wait. Four crates were not published. For v0.4.0, the first attempt stopped at headwater-import for the same reason. In both releases, a second run of the failed job published the rest. The loop now waits until the index lists each dependency. When the index is too slow, the loop stops with an error that names the dependency. After the second run, go back to step 8 and do both checks again before you do step 9. HW-PD-0011 records the fixed order, the skip and both waits. The loop waits out a 429 from crates.io and tries again, and HW-OBL-0182 records that this wait has no ceiling.

A release has no archive, or an archive is wrong. Run release.yml by hand with the tag input. It attaches the archives to a tag that exists, and it cuts no new tag. With publish set to false, it builds and runs the smoke jobs and creates no release. The header of release.yml gives the command. A hand run cannot repair a tag that already has a release, because GitHub makes a release immutable when it is created. To run again, delete the release, keep the tag, and run the workflow. HW-PD-0009 records why.

A crate version can never become complete. Only then use yank-crates.yml. Version 0.1.0 is the example: it was published before headwater-compat could be, and 0.1.1 replaced it. A yank hides a version from a new cargo add. It deletes nothing, and a lockfile that names the version continues to work. A crate that is complete but wrong gets a new patch version and no yank.

How to know it worked

  • gh release view v<version> --json url,assets lists three archives and three .sha256 files, one pair for each row of the matrix in release.yml. It also lists headwater_<version>_amd64.deb, and, when the APT_SIGNING_KEY secret is set, Packages, Release, InRelease and Release.gpg.
  • For each crate in the order variable of publish-crates.yml, https://crates.io/api/v1/crates/<name> reports the new version as max_version. The crates.io API refuses a request that has no User-Agent header, so send one.
  • When the APT_SIGNING_KEY secret is set, https://headwater.tools/apt/dists/stable/main/binary-amd64/Packages has the line Version: <version>, with ~ for a pre-release hyphen. The Release file beside it does not state a package version, but its Date is the time of the publish job. The deploy-site job of release.yml puts these files on the site after the release exists. If that job failed, run it again from the run page of release.yml.
  • site/changelog/index.html has an entry for the new version.
  • CI is green on the pull request that moves the install text, which includes tools/repo/readme-fixtures.sh.
  • For a taxonomy release, gh release view taxonomy/headwater-standard/v<version> lists headwater-standard-<version>.zip, and the release notes state the digest.

Step 8 of the engine release needs the first three checks. Step 4 of the taxonomy release needs the last check. Do them before you move any install text.