Check before a release
Run one RunStory check over every commit since the last release tag: by hand, from a job you start before you tag, or every night.
Where the check fits
You ship in versions: a tag, a release branch, a train that leaves on Tuesday. Lots of changes go out together, and coding agents wrote most of them, fast. One check before the tag covers everything that's about to ship, running as one app.
Six commits sit on main after the tag v1.4.0. One check covers them all and builds the newest. If it finds no problems, you tag the release.
Check a release by hand
Name the last release tag, then two dots. RunStory checks every commit after it.
git fetch --tags
runstory run v1.4.0..
| You type | RunStory checks |
|---|---|
runstory run v1.4.0.. | Every commit since the tag v1.4.0, on the branch you're on |
runstory run v1.4.0..release/1.5 | Every commit on release/1.5 since the tag, without checking the branch out |
runstory run v1.4.0.. --dry-run | Nothing. It shows what a check would cover and what it costs |
$ runstory run v1.4.0.. --dry-run
Checking v1.4.0.., 6 commits from v1.4.0 (b61f0c2) to main (0d9e7a4), with 9 files changed
Start the check from a job
Store an API key as the secret RUNSTORY_API_TOKEN, the way Gate a merge into main shows. Then add a workflow you can start from the Actions tab before you tag.
.github/workflows/release-check.yml
name: release check
on:
workflow_dispatch:
jobs:
check:
runs-on: ubuntu-latest
timeout-minutes: 80
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- run: curl -fsSL https://runstory.specstory.com/install.sh | sh
- run: echo "$HOME/.local/bin" >> "$GITHUB_PATH"
- run: runstory run "$(git describe --tags --abbrev=0).." --wait --timeout 75m
env:
RUNSTORY_API_TOKEN: ${{ secrets.RUNSTORY_API_TOKEN }}
git describe --tags --abbrev=0 prints the newest tag the commit can reach, so the job needs no input once the repository has a tag. The job fails when the check confirms a problem, and its summary lists each finding's id and title, with a link to the check.
Check main every night
If your team deploys from main without tags, a nightly job can check the last day of commits as one change.
.github/workflows/nightly-check.yml
name: nightly check
on:
schedule:
- cron: "17 3 * * *"
jobs:
check:
runs-on: ubuntu-latest
timeout-minutes: 80
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- run: curl -fsSL https://runstory.specstory.com/install.sh | sh
- run: echo "$HOME/.local/bin" >> "$GITHUB_PATH"
- run: |
base=$(git rev-list -1 --first-parent --before="24 hours ago" HEAD)
if [ -n "$base" ] && [ "$base" != "$(git rev-parse HEAD)" ]; then
runstory run "$base.." --wait --timeout 75m
fi
env:
RUNSTORY_API_TOKEN: ${{ secrets.RUNSTORY_API_TOKEN }}
Example: the Tuesday release
A team at Acme, an online furniture store, ships every Tuesday. Version 1.5.0 holds a week of commits since v1.4.0, and coding agents wrote most of them.
- On Monday afternoon, a developer starts the release check from the Actions tab.
- RunStory builds the store from the newest commit and starts it. Its agents place an order with a gift card, then open the order history. The order's total is wrong there.
- The failure's script repeats it 3 more times, and a second agent confirms it. The job fails, and its summary lists finding
f-be28dce9cba3. - A coding agent fixes the total and names the finding's id in its commit message.
- The developer starts the job again. Because the commit message names the id, the check runs the finding's script 3 times. It passes, and the team tags
v1.5.0on Tuesday morning.
What it costs
A release check is one check, however many commits it covers. A default check uses 2 frontier models and costs 6 credits.
| To | Do this |
|---|---|
| Give a large release more time | On Pro and Team, run runstory settings --time-limit 120. Then wait with --timeout 125m, and raise the job's timeout-minutes to 130. A longer limit doesn't change the price |
| Check a very large release in parts | Split the range at a commit, like v1.4.0..7a2c9e1, then 7a2c9e1... One check studies up to 200 KB of diff |
| Cap the project's checks | runstory settings --daily-limit 30 |