Skip to content
#

Check before a release

Run one RunStory check over every commit since the last release tag: by hand, from a job you start before you tag, or every night.

#

Where the check fits

You ship in versions: a tag, a release branch, a train that leaves on Tuesday. Lots of changes go out together, and coding agents wrote most of them, fast. One check before the tag covers everything that's about to ship, running as one app.

One check covers every commit since the last release main v1.4.0 v1.5.0 check 88 runstory run v1.4.0.. no problems: tag the release

Six commits sit on main after the tag v1.4.0. One check covers them all and builds the newest. If it finds no problems, you tag the release.

#

Check a release by hand

Name the last release tag, then two dots. RunStory checks every commit after it.

git fetch --tags
runstory run v1.4.0..
You typeRunStory checks
runstory run v1.4.0..Every commit since the tag v1.4.0, on the branch you're on
runstory run v1.4.0..release/1.5Every commit on release/1.5 since the tag, without checking the branch out
runstory run v1.4.0.. --dry-runNothing. It shows what a check would cover and what it costs
$ runstory run v1.4.0.. --dry-run
Checking v1.4.0.., 6 commits from v1.4.0 (b61f0c2) to main (0d9e7a4), with 9 files changed
#

Start the check from a job

Store an API key as the secret RUNSTORY_API_TOKEN, the way Gate a merge into main shows. Then add a workflow you can start from the Actions tab before you tag.

.github/workflows/release-check.yml

name: release check
on:
  workflow_dispatch:
jobs:
  check:
    runs-on: ubuntu-latest
    timeout-minutes: 80
    steps:
      - uses: actions/checkout@v6
        with:
          fetch-depth: 0
      - run: curl -fsSL https://runstory.specstory.com/install.sh | sh
      - run: echo "$HOME/.local/bin" >> "$GITHUB_PATH"
      - run: runstory run "$(git describe --tags --abbrev=0).." --wait --timeout 75m
        env:
          RUNSTORY_API_TOKEN: ${{ secrets.RUNSTORY_API_TOKEN }}

git describe --tags --abbrev=0 prints the newest tag the commit can reach, so the job needs no input once the repository has a tag. The job fails when the check confirms a problem, and its summary lists each finding's id and title, with a link to the check.

#

Check main every night

If your team deploys from main without tags, a nightly job can check the last day of commits as one change.

.github/workflows/nightly-check.yml

name: nightly check
on:
  schedule:
    - cron: "17 3 * * *"
jobs:
  check:
    runs-on: ubuntu-latest
    timeout-minutes: 80
    steps:
      - uses: actions/checkout@v6
        with:
          fetch-depth: 0
      - run: curl -fsSL https://runstory.specstory.com/install.sh | sh
      - run: echo "$HOME/.local/bin" >> "$GITHUB_PATH"
      - run: |
          base=$(git rev-list -1 --first-parent --before="24 hours ago" HEAD)
          if [ -n "$base" ] && [ "$base" != "$(git rev-parse HEAD)" ]; then
            runstory run "$base.." --wait --timeout 75m
          fi
        env:
          RUNSTORY_API_TOKEN: ${{ secrets.RUNSTORY_API_TOKEN }}
#

Example: the Tuesday release

A team at Acme, an online furniture store, ships every Tuesday. Version 1.5.0 holds a week of commits since v1.4.0, and coding agents wrote most of them.

  1. On Monday afternoon, a developer starts the release check from the Actions tab.
  2. RunStory builds the store from the newest commit and starts it. Its agents place an order with a gift card, then open the order history. The order's total is wrong there.
  3. The failure's script repeats it 3 more times, and a second agent confirms it. The job fails, and its summary lists finding f-be28dce9cba3.
  4. A coding agent fixes the total and names the finding's id in its commit message.
  5. The developer starts the job again. Because the commit message names the id, the check runs the finding's script 3 times. It passes, and the team tags v1.5.0 on Tuesday morning.
#

What it costs

A release check is one check, however many commits it covers. A default check uses 2 frontier models and costs 6 credits.

ToDo this
Give a large release more timeOn Pro and Team, run runstory settings --time-limit 120. Then wait with --timeout 125m, and raise the job's timeout-minutes to 130. A longer limit doesn't change the price
Check a very large release in partsSplit the range at a commit, like v1.4.0..7a2c9e1, then 7a2c9e1... One check studies up to 200 KB of diff
Cap the project's checksrunstory settings --daily-limit 30