Build reference

This page is the lookup for what a runner does before and after a build’s command. What happens to one build gives the order of the stages. The commands that start and inspect a runner are in Runner commands. To fix a build that failed, see Troubleshoot builds.

The branch a claim names

A claim names the branch that the build works on in each repository the factory connected. The item’s own repository is the primary. The build gets the primary’s branch as OUTERLAYER_BRANCH, and all of them as OUTERLAYER_REPOSITORIES. See The repositories in a build. In each repository, the branch depends on the item’s open pull requests: OuterLayer writes the factory name in lower case. It replaces each run of characters other than letters, digits, ., _ and - with -. A name that this changes also gets 8 characters of a hash, so two factories never share a branch. Item 7 of acme builds on outerlayer/acme/7. Item 7 of Acme Prod builds on outerlayer/acme-prod-<hash>/7. The runner checks the branch out before the command starts:

• An outerlayer/ branch that exists and is not merged is continued.
• An outerlayer/ branch that is missing or merged starts from the default branch.
• Any other branch must exist on the remote, or the build fails at provision.

A container or microVM build may push only that branch. See The git remote and the build’s tokens. OuterLayer leaves a repository other than the primary out of the claim when its GitHub App installation is suspended. The build then runs in the other repositories. A hook of your own must not make a git worktree on the claim’s branch while another worktree has that branch checked out. Git refuses, and the build fails. Make a detached worktree.

When a claim is refused

A refused claim records a failed build with the code as its reason. The runner logs #<item> skipped: <code>. The refusal uses up the build request, except where the table says otherwise. So fix the cause, then ask for a build again.

Opening the pull request

outerlayer work open-pr opens the pull request from the claim’s branch into the default branch. If one is open, it edits that one. A runner build opens its pull request as the factory’s GitHub App, not with a GitHub login on the host. OuterLayer uses a GitHub token that covers only the item’s repository, and the build never receives it. In your own session, outerlayer work open-pr runs gh under your login. Its refusals in a runner build:

Item key

The runner gives a build an item key (olitem_…) as OUTERLAYER_API_KEY, not the runner key. The item key acts only on its own item:

• It can read the item, its threads, checks and changes.
• It can link and sync sessions, and post comments.
• It can declare, open or edit the pull request.
• It can emit artifacts, results and findings.

The key stops working when the claim ends, and at the latest 24 hours after the claim began. So runner.timeLimitMinutes is at most 1320 minutes, which is 22 hours. A container or microVM build holds only a placeholder. The runner adds the real item key to the build’s calls to OuterLayer. A build under hooks with no isolation runs as the runner’s account. So it can read the runner key and the item key from the runner’s files. See What the keys limit. The build’s sessions upload under the item key while the build runs. The item shows each session and its cost within a minute of each turn. The runner uploads the sessions in full before it ends the claim. When a build is killed, the turns after the last upload are lost.

Repository tokens

Through the factory’s GitHub App, OuterLayer issues short-lived GitHub tokens to a build. The permissions the App holds are in GitHub App permissions and build tokens.

Who can get a token

A build gets repository tokens only when a member of your organization asked for it:

• For a build, that is the person who ran outerlayer work build or chose Build on the Work page.
• For an amend, that is the person whose newest comment or fail on the item started it.

A key counts as a member’s only when it belongs to an active member and is not a runner key. OuterLayer checks this on every request, so a member removed from the organization stops at once. Only the key that holds the item’s live claim can ask. A claim that ended gets nothing. A session that you start yourself gets no token. outerlayer work build with a runner key still records the build request, and prints a warning. A container or microVM build of that request fails, because it gets no token. A build that an automation starts needs no member. It gets tokens only while the automation’s setting is on: For a build that a member asked for, the audit event records kind member. See What OuterLayer records about a token.

The two kinds of token

Each token covers one repository that the claim names. It lasts about an hour. The runner also asks for a contents-read token on the control plane that governs the item’s repository. It uses that token to fetch the build’s governed context.

When a token is refused

The proofs stage

When a build’s command exits 0, the runner records the item’s proofs itself, before the judge stage. An amend gets the same stage. A proof counts only at the pull request’s head, so the runner records at the build’s final commit. The stage runs outerlayer work proofs --record --comment in the build’s own container, microVM or process. It runs again the tests that name each proof that is not current, and emits what they write. See Record proofs by rerunning tests. The stage records nothing when every declared proof is current, or when the item declares no proof. A proof that the stage cannot record stays out of date or missing:

• The item’s verdict names the proof.
• The item’s general thread gets one comment that names each proof and the reason.
• The build’s reason names the checks that do not pass, then the cause, such as the host's recording left proofs out of date (setup failed: "yarn db:setup" exited with status 1). A reason has at most 200 characters, so a long cause is cut. A cause for one proof starts with the criterion’s id.

The stage’s output is in proofs.log in the build’s folder.

When the proofs are recorded again

The head of a pull request can move after a build ends. For example, a person pushes, or an amend fixes a failed check. The proofs are then out of date at the new head. The built-in automation Record proofs again starts an amend for this. It is on by default. Its switch is in Settings → Automations, under Pull requests and reviews. The amend starts when all of these are true:

• OuterLayer evaluated the pull request’s current head and found a proof from an earlier commit.
• A runner built the item.
• The head has no merge conflict, and no failed check that the runner still tries to fix. The amend for either moves the head again.

When nothing else waits on an agent, this amend runs no agent. The runner records the proofs at the head, then runs the judge and the host checks. The automation starts once for each head and each set of out-of-date proofs. A recording that fails starts no second amend. The next push, or a change to which proofs are out of date, starts one again.

The judge stage

A where: host check that declares a judge runs in the judge stage. The stage runs when the build’s command exits 0, after the proofs stage and before the host checks.

What the judge reads

The judge is a fresh Claude Code agent with the check’s brief and no shell. It gets these inputs:

• each criterion’s text and declared proof
• the branch’s diff
• the files of the pushed commit

Nothing tells the judge which tests prove a criterion. It searches the commit’s files, and starts from the tests that the diff changes. A test needs no comment or title that names the criterion. The judge reads the pushed commit, never the build’s checkout. It does not judge a file that the build changed and did not push. A claim can name several repositories. The judge then reads one of them. It reads the primary repository when the build pushed there. Otherwise, it reads the first repository of the claim that the build pushed in. The judge fails for a missing branch only when the build pushed in none of the repositories. For each criterion, the judge gives a result, a reason and the tests that the result rests on. A pass must name at least one test file of the pushed commit. The runner records one check item for each criterion, and the item’s Criteria tab lists the tests. The runner asks the judge about at most 12 criteria in one run. For more criteria, it judges them in groups. Each group has its own limit of 60 turns, and its own time limit of runner.checks.timeLimitMinutes. A judge with subjects: none has no criteria. It judges the whole change every time. The judge needs these things:

• A container or microVM build. A build under your own hooks has no judge.
• A build image that holds Claude Code, because the judge runs inside the image.
• The runner’s Claude credential.

The stage also runs the built-in code review when the factory’s Code review automation is on. The code review never changes the build’s outcome. A repository can keep its own copy of the starter brief at the path the check names. If that copy finds tests by // proves comments or reads citations.json, the runner uses the brief from its CLI and logs that it did.

Which files count as tests

A file counts as a test in three cases:

• Its name is a common test name.
• It is a source file in a test folder.
• A runner glob in the proofs file matches it.

The common test names are: Class names match the capital letter, so Contest.java is not a test. A test folder is named test, tests, spec, __tests__, Test or Tests, or has a name that ends in Test or Tests, such as UnitTests. The names are case-sensitive. Each source file in a test folder counts, whatever its name. A source file has the extension of a programming language. Documentation, data and image files do not count. The test-folder rule skips a file whose path holds one of these folders, in any case:

• Test data: fixtures, fixture, __fixtures__, testdata, test-data, snapshots, __snapshots__ and resources.
• Other code or build output: node_modules, vendor, third_party, target, build, dist, out, obj and .git.

A common test name wins over this rule, so test/fixtures/wallet_test.rb counts. The runner reads the globs from .outerlayer-proofs.yaml at the pushed commit. A glob only adds files.

The result for one criterion

The runner asks the judge again about a criterion that failed before, in every run. A test that proves it can be in any file. A reviewer can accept a criterion that the judge failed. See Accept a criterion without proof.

Why a judge check fails

The runner records the check as a fail, with one of these reasons. The next build or amend of the item runs the judge again. The judge’s output is in judge.log in the build’s folder. When a subjects: none judge cannot run, its result has a failing check item with the id judge-not-run. The check item’s reason is one of the reasons above.

Host checks

A repository’s where: host validators declare commands that the runner runs itself. Checks that run on your host gives the file format. The runner runs the commands only when the build’s command exits 0, after the judge stage. Only builtin:container and builtin:vm builds run host checks. The checks of one build run in a new container or microVM:

• It is made from the build’s image, with the build’s limits, tunnel destinations and allowHosts.
• It holds no OuterLayer key, no address of OuterLayer and no model credential.
• Its git remote serves fetches only.

The runner reads the check definitions from the repository that governs the item, never from the build’s checkout. So a build cannot change its own checks. Only the runner can record a result for a host check. In the new environment, the runner does these steps:

1. Checks out the claim’s branch.
2. Installs the runtimes that the branch’s version files name, on the universal image or an image built from it. Each check command then gets the variables that mise env prints. See Variables from mise.
3. Runs the recipe’s lifecycle commands.
4. Runs each repository’s setup script.
5. Runs each check command, one at a time.

Two variables name paths that a command can use:

Why a host check fails

The runner records a check as a fail, with one of these reasons: If the runner cannot read the definitions, it records nothing and logs why. The check then reads as waiting while the claim is live, and as failed after the claim ends.

Logs and time limits

Each command’s output is in checks/<name>.log in the build’s folder. The setup of the environment is in checks/environment.log. runner.checks.timeLimitMinutes limits two things: the setup of the environment, and each check command. It is 10 by default. runner.build.setupMinutes limits each setup script, as it does in the build. See the config file. Host checks run the recipe’s setup commands once more for each build, and a builtin:vm build starts a second microVM. Allow for that time.

Files the runner keeps

The runner’s files are in <config dir>/runner/, which is ~/.outerlayer/runner/ by default. Only the runner’s account can read them. Housekeeping says what the runner removes and when. Each build has one folder in runner/jobs/, named for the queue, the item and the claim. These are the files to read when a build fails: outerlayer work logs --item <n> prints the runner’s log for a build without a login to the host. See Find out why a build ended.

Runner status

The factory’s Settings → Runners page lists each runner that has checked in, with a status. GET /v1/hosts returns the same runners with the status as a code. The call needs the hosts.read permission. See List the factory’s runner hosts for every field. A runner has the first status in this table that applies: A runner that is blocked or low on disk still reads Ready, but takes no work. The API shows this in state, blocked and lowDisk. See When the runner is blocked and When the runner holds off for disk.

Runner protocol

A runner reports a protocol number to OuterLayer. This number is not the CLI version. When OuterLayer needs a higher number, it refuses the runner’s claims, and the runner’s status is Outdated. Update the runner.