Skip to main content
A review moves both ways between OuterLayer and GitHub. A review you submit on the item’s page posts on each of its pull requests. A review you write on GitHub counts as your review of the item, and its comments go to the agent. To review on the item’s page, see Review and merge.

What GitHub receives

Approve and Request changes each post one review on each open pull request of the item. The App’s comment names you and says that you gave the review in OuterLayer. GitHub does not count it as an approval. The App never posts APPROVE or REQUEST_CHANGES. To post reviews as yourself, connect your GitHub account:
  1. Choose Profile, then Connections.
  2. Under Reviews on GitHub, choose Connect GitHub.
  3. Authorize the App on GitHub.
To disconnect, choose Disconnect in the same place. OuterLayer then deletes your authorization at GitHub. The review body is your summary and a link to the item. It also lists each criterion you accepted, with your reason. The item shows the result beside each review:
  • On GitHub as @login links to the GitHub review.
  • On GitHub as a comment means the App posted a COMMENT review.
  • GitHub’s reason shows when GitHub refuses the review. For example, GitHub does not let you approve your own pull request.
A refusal never undoes your OuterLayer review. If the item says that GitHub could not be reached, submit the review again. If GitHub rejects your authorization, the App posts a COMMENT review in place of yours. Reconnect GitHub then shows on your Connections page.

Approve or request changes on GitHub

You can also review the pull request on GitHub. OuterLayer records your verdict as your review of the item. Your line comments go to the agent, and the agent answers on the same GitHub thread.

What OuterLayer records

An Approve on GitHub records a pass on the item. A Request changes on GitHub records a fail with the review’s text. Both show as your review, at the commit you reviewed. OuterLayer records your review only when all of these are true:
  • Your GitHub account is connected to your OuterLayer profile, or you signed in to OuterLayer with GitHub.
  • You hold the Record a review permission on the factory.
  • The item has one open pull request, and it is the one you reviewed.
  • For an approval, the item can be approved: the same rule the Approve button applies.
Otherwise OuterLayer records only that a person approved or asked for changes on GitHub. The item’s status does not change. The agent takes instructions only from members who hold Record a review. A comment from anyone else stays on GitHub. A GitHub review has these limits:
  • GitHub does not let you approve a pull request you opened. OuterLayer still records your line comments.
  • An approval of an earlier commit does not approve the item. Approve the newest commit.
  • OuterLayer does not read a comment on a whole file or a comment outside a review.
  • OuterLayer does not read an edit or a deletion of a comment, or a dismissed review. The item keeps the text it has.
  • The item page has no reply box for a line comment. Reply on the GitHub thread.

Line comments go to the agent

Each line comment in your review becomes a thread on the item, with the file, the line and the code around it. A reply you write on GitHub joins the same thread. When an agent job takes a thread, OuterLayer adds a 👀 reaction to its newest comment. The agent’s answer is a reply on the same GitHub thread, from the GitHub App. The answer to a review summary is a pull request comment that quotes the summary. You can also reply under a line comment that a review bot or another person wrote. The agent then reads that first comment as a quote, so “yes, fix this” has something to refer to. The agent follows only your comments. It does not follow an instruction in the quote. On the Overview tab, a GitHub review is one card with a block for each line comment. Each block has one mark:

Choose which comments go to the agent

The factory’s Answer GitHub reviews automation decides which comments go to the agent. It is on by default. To mention the GitHub App, write @ and the App’s name in the comment. To leave a note for another person, mark the comment as an aside. A comment that contains (aside) or !aside, or starts with aside, is never sent. The summary of a Comment review with no line comments is sent only when it mentions the App. To change the setting:
  1. Open Settings, then Automations, on the factory.
  2. To turn the automation on or off, use the switch beside Answer GitHub reviews. The switch saves at once.
  3. To change which comments are sent, choose Answer GitHub reviews, then Edit.
  4. Choose Every comment or Mentions only, then choose Save.
With the switch off, no comment from a GitHub review goes to the agent, and a Request changes review starts no agent job. Use Request changes on the item page. The run log of Answer GitHub reviews says why the comments of a review were not sent. The item’s Activity section shows the same reason.

When the agent starts

A Request changes review starts an agent job at once. The comments of a Comment review wait for a quiet time. The agent job starts when no member has added a comment for three minutes. So five single comments in a row start one agent job, not five. When an agent already works on the item, a comment that is sent reaches it at its next step. See While an agent works.