> ## Documentation Index > Fetch the complete documentation index at: https://docs.outerlayer.ai/llms.txt > Use this file to discover all available pages before exploring further. # Prepare builds with a setup script > Commit .outerlayer/setup.sh to prepare each build's checkout, and learn what a later build, a rerun and an agent job start from. A repository commits one file, `.outerlayer/setup.sh`. It prepares the repository's checkout, for example by installing dependencies. The runner runs it in every container, microVM and sandbox build, before the build's command. After the scripts pass, the runner saves the build's disk. A later build starts from that [saved state](#the-saved-state), so it does not install everything again. ## Write the script The script is a shell script. Put the commands that prepare a checkout in it, such as `npm ci` or `pip install -r requirements.txt`. ```sh theme={"system"} #!/bin/sh set -eu npm ci ``` Follow these rules: * Commit the file with its executable bit set. The runner then runs the file itself, and its first line chooses the shell. Without the bit, the runner runs the file with `/bin/sh`, which is `dash` on Ubuntu. A script that needs bash fails there. * Use `set -e`. The script's exit status is the status of its last command. A script without `set -e` can pass after a failed command, and the runner then saves a broken state. * Write commands for an unprivileged user. The script cannot install system packages. Put those in the Dockerfile of the recipe, the repository's devcontainer file. See [Where setup that needs root belongs](https://docs.outerlayer.ai/build-image-from-devcontainer.md#where-setup-that-needs-root-belongs). * Write the script to run twice without harm. It runs in every build, even a build that starts from a saved state. There, the run takes seconds, because the work is already done. * In a container build, write files into the build user's home or a checkout. The saved state holds only those, so a file in `/tmp` is not in the next build. A microVM build's saved state holds its whole build disk. * In a checkout, write only files that git ignores, such as `node_modules`. A build that starts from a saved state removes the other untracked files. The runner runs each repository's script in that repository's checkout, as the build user. It runs the primary repository's script first. The scripts run after the clone, the [runtime install](https://docs.outerlayer.ai/build-image-from-devcontainer.md#the-runtimes-in-the-universal-image) and the recipe's [lifecycle commands](https://docs.outerlayer.ai/build-image-from-devcontainer.md#lifecycle-commands). The script reaches only the hosts that `runner.build.allowHosts` allows, as every other command of the build does. ## When the script fails A script that exits with a code other than 0 ends the build at stage `provision`. A script that runs past `runner.build.setupMinutes` does the same. The default limit is 30 minutes for each script. The build's command does not start, and the runner saves no state. The item shows the reason. It starts with `setup_failed` or `setup_timed_out`, and names the repository, then the exit code or the limit, then the last lines the script printed. To read more of the output, run `outerlayer work logs --item `. It prints the last 200 lines of the provision log. ## In host checks [Host checks](https://docs.outerlayer.ai/reference/build-reference.md#host-checks) run in a new container or microVM, so nothing the build installed is there. The runner runs each script again in that environment, before the first check. A check can then use what the script installed, such as `node_modules`. The script runs from the branch the build pushed, with the same time limit. The check environment never starts from a saved state, so the script does all its work there. A script that fails there fails every host check of the build. ## Repositories with no script When a repository has no `.outerlayer/setup.sh`, the build runs without it. The runner sets `OUTERLAYER_MISSING_SETUP_SCRIPTS` for the build's command. Its value is a JSON list of the repositories that have no script, such as `["acme/app"]`. The [`outerlayer-build` skill](https://docs.outerlayer.ai/starter-skills) reads it. The agent writes the script from the commands it ran to prepare the checkout, and the build's pull request adds the file. A build in which no setup script ran saves no state. ## Git hooks in a build A build credits the member who asked for it as co-author of each commit. To add that line, the runner puts git hooks of its own in front of the repository's hooks while the build's command runs. Each one runs the repository's hook of the same name, so your hooks still run. Two cases need a change in the repository: * **Hooks that are not installed.** A repository that uses husky or lefthook must install its hooks in the build. Otherwise each commit and push prints a line that names the hooks folder. Install them in the setup script. * **`lefthook install` in the build's command.** It fails with `permission denied`, because it writes into the runner's hooks folder. This happens when a `prepare` script of `package.json` runs it. Use this command there: ```sh theme={"system"} env -u GIT_CONFIG_COUNT lefthook install ``` `env -u GIT_CONFIG_COUNT` clears the runner's hooks setting for this one command. It also clears every other git setting that the build's variables pass in the same way. The command also works in the setup script. husky needs no change. ## The saved state A saved state is a copy of what the setup scripts prepared. The runner keeps saved states on its own disk. A container build's state holds the build user's home and the checkouts. A microVM build's state holds the whole build disk. The runner saves a state after a build whose setup scripts all pass, when the build did not start from one. A later build uses the state when all of these match: * The factory and its repositories. * The size of the build disk, `runner.build.disk`. * The content of each repository's setup script and [version files](https://docs.outerlayer.ai/build-image-from-devcontainer.md#the-runtimes-in-the-universal-image), on the default branch. The runner reads the script and the version files from each repository's default branch, never from the branch a build works on. So a build cannot change the state that later builds use. A build never changes the state it starts from. It works on a copy. When a build starts from a saved state, its log says so and gives the state's age. ### When a build starts from a fresh disk A build starts from a fresh disk, runs the whole script, and saves a new state in these cases: * No saved state matches. * The saved state is seven days old or older. * A setup script or a version file changed on the default branch. The runner's log names the file. To discard a saved state sooner, change the setup script on the default branch. A change to a comment line is enough. ### Builds that save no state These builds run the script and save no state: * A build whose image comes from a recipe. Only a build on the universal image saves a state. * A build on a [managed runner](https://docs.outerlayer.ai/build-security.md#managed-runners). * A build on a runner with `builtin:sandbox` hooks. * A build on a runner whose `runner.housekeeping.savedStateMax` is `off`. ## What an agent job starts from A rerun of a build, and an agent job, start from the build disk that the item's previous job left. An agent job answers a comment or a failed check on the pull request of a build. Neither starts from a saved state. The runner keeps one disk for each work item, on its own host. It keeps the disk when a container or microVM job ends, whatever the outcome of the job's command. A newer disk replaces the older one. Two kinds of job keep no disk, and the older disk stays: * A job that fails at provision. * A build that the runner ends for passing its disk limit. The kept disk holds what the previous job left: * Untracked and ignored files in each checkout, such as installed dependencies and build outputs. * Files outside the checkouts, such as caches and the home folder. * For a microVM job, the Docker images that the job pulled or built. It does not hold memory or running processes. A service or a database starts again. When a job starts from a kept disk, the runner does this first: 1. It discards the changes to tracked files in each checkout. 2. It checks out the head of the item's branch. Each tracked file then matches the branch. A rerun checks out the default branch when the remote has no such branch. 3. It runs the setup scripts again. Commits that the earlier job made and did not push are not on the branch. The runner keeps them in the checkout at the ref `refs/outerlayer/kept/`, and its log names that ref. ### With more than one runner A runner keeps an item's agent session and its unfinished work on its own host. What it keeps of the build disk depends on its hooks: | Hooks | Where the kept disk is | | - | - | | `builtin:container`, `builtin:vm` | On the runner's host. | | `builtin:sandbox`, on your own provider account | At the provider, as a paused sandbox. Only the runner that paused it resumes it. | | `builtin:sandbox`, in a managed sandbox | No disk is kept. | Another runner has none of this. So an item's next job goes first to the runner that ran its last job. When another runner of your own claims the item, the gateway refuses the claim with `work_item_held_for_runner`. That runner logs the refusal and takes other work. The first runner takes the item at its next poll. The wait is twice the poll interval of the first runner. It is at least 60 seconds and at most 10 minutes. After the wait, any runner takes the item, and the job starts with a new session and without what the first runner keeps. The gateway does not wait when the first runner cannot take the job: * It sent no heartbeat for three of its poll intervals. * All its slots are in use. * It does not take the item's queue or repository. * It reports that it takes no work, for example while it drains or updates. A build that you start from a fresh disk does not wait. ### When a job starts without the kept disk A job starts as a first build does, from a saved state or a fresh disk, in these cases: * The job is the first build of the item. * The job runs on another host than the one that holds the disk. * The build image changed. A runner update can change it. * `runner.build.disk` changed. * The kept disk cannot be used. For example, a file that the earlier job left untracked is tracked on the branch. * A member asked for a fresh disk. The job never fails for one of these reasons. The job's card in the **Activity** section of the item page gives the reason. See [The Overview tab](https://docs.outerlayer.ai/review-the-work.md#the-overview-tab). Two more rules apply on a managed runner: * It keeps no disk for a build that held a build variable. A command can write the value to the disk. * The disk is on one machine. A job that another machine takes starts without it. An agent job waits up to 1 minute for the machine that holds the disk. ### Start a rerun from a fresh disk A damaged `node_modules` or a full disk can break each rerun. To start a build without the kept disk, tick **Start from a fresh disk** in the rerun dialog. Or run [`outerlayer work build --fresh`](https://docs.outerlayer.ai/reference/cli-work#outerlayer-work-build). The build then starts from a saved state or a fresh disk. When it ends, its disk replaces the kept one. The choice applies to a build only. An agent job on an open pull request has no fresh-disk choice. A request for one is refused as `fresh_disk_unavailable` when the item's open pull request waits on an agent. ### Continue the session of the previous build A rerun of a build starts a new agent session. The agent reads the issue and the code again, on the kept disk. After a build that you stopped, or that ended at its time limit, the item page offers **Resume** at its foot. **Resume** builds again in the session of that build. After any other build, open the rerun dialog and tick **Continue the session of the last build**. That box and **Start from a fresh disk** cannot both be ticked. To have the agent go on in the session of the item's previous build, run [`outerlayer work build --item --continue`](https://docs.outerlayer.ai/reference/cli-work#outerlayer-work-build). Use it after you stop a build to redirect it, or after a build ends at its time limit. Add `--instructions` to say what to change. The continued session starts with the files as the earlier job left them. A job that a member stops, or that ends at its time limit, leaves work it did not push. So does a job whose agent ends before it opens a pull request. The runner keeps that work with the session: the edits the job did not commit, and the commits it did not push. A job that continues the session gets both back. Each checkout is at the commit the earlier job was at, and the edits are not committed. This also applies to an agent job on an open pull request. The runner keeps this work on its own host, with the session. It does not push it, and it does not send it to OuterLayer. Ignored files, such as installed dependencies, are not part of it: the setup scripts and the kept disk supply those. The work is not put back in two cases, and the runner's log says which: * The remote branch has commits that the work was not made on. The checkout stays at the head of the branch. The earlier files are in the checkout at the ref `refs/outerlayer/kept/`. * The work of one checkout is larger than 64 MiB. The runner keeps none for that checkout. Three things are not put back as they were: * Which edits were staged. All edits come back unstaged. * A merge or a rebase that the job did not finish. The checkout is at the commit the job was at, and the merge or rebase is not in progress. Its commits are parents of the commit at `refs/outerlayer/kept/`. So are a stash, and the commits of another local branch. * A git repository inside a checkout. Its files are not part of the work. A job that starts a new session never gets this work. It starts from the branch as pushed. These limits apply: * The rerun must run on the host that ran the previous build. A runner keeps the session of a container or microVM build on its own host. * When the runner has no session to continue, the build starts a new session. The runner's log says why. * `--continue` and `--fresh` cannot go together. * An agent job on an open pull request continues the session already. The option changes nothing there. A session that went wrong stays wrong when you continue it. After a build that failed, a new session is often the better choice. ## Disk Saved states and kept disks use the disk of the runner's host. These settings under `runner.housekeeping` limit them: | Setting | Default | What it limits | | - | - | - | | `savedStateMax` | `20g` | The total size of the saved states. Past it, the runner removes the oldest states first. `off` saves no states. | | `workspaceMax` | `40g` | The total size of the kept disks. Past it, the runner removes the oldest disk first. | | `workspaceKeepDays` | `7`, or `1` with `builtin:sandbox` hooks | The days a kept disk stays when no job of its item replaces it. | The runner also removes these: * A saved state that is seven days old. * The kept disk of an item whose pull request merged or closed. * Kept disks, then its oldest saved states, when free disk is low. It does this before it stops claiming work. A microVM build saves no state when its build disk uses more than `savedStateMax`. It also saves none when the copy would leave less than `runner.housekeeping.minFreeDisk` free. A build with a large build directory needs a higher `savedStateMax`. On a Mac, a container build's saved states and kept disks use the disk of the VM that runs Docker. `outerlayer runner init --vm` adds room for `savedStateMax` to the disk of the Lima VM it makes, and none for kept disks. On Docker Desktop, raise the disk limit in its settings, or set the limits lower. ## Keep secrets out of a saved state > [!WARNING] > > A file that a build writes stays in the saved state and on the kept disk. For > example, an `~/.npmrc` with a token from a build variable stays. Do not write > a secret into a file when you can avoid it. Both stay on the runner's host, and only builds of the same factory use them. A kept disk goes only to jobs of the same work item. To keep a secret out of the build, use a [destination](https://docs.outerlayer.ai/destinations-and-secrets.md) on a runner of your own. The build then holds no secret, so no file can. ### On a managed runner On a managed runner, every step before the build's command runs without the factory's build variables and without the build's `gh` login. That covers the runtime install, the recipe's lifecycle commands and the setup scripts. The build's command gets both. So a setup script there cannot read a build variable, such as a registry token for private packages. It also cannot call `gh`. Such a script fails, and the runner's log names the limit. A [proxied build variable](https://docs.outerlayer.ai/build-security.md#proxied-build-variables) works in a setup script. The script holds a stand-in, and the runner adds the real value to each request to the service. In the host checks, the setup script does get the build variables. Do not write a script that does different work when a variable is set. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.