Skip to content

ci: CI, PR-Agent review, the auto-merge gate and an SDK version check - #51

Merged
positonic merged 3 commits into
mainfrom
chore/ci-pr-agent-automerge
Nov 18, 2026
Merged

positonic merged 3 commits into
mainfrom
chore/ci-pr-agent-automerge

Conversation

@positonic

@positonic positonic commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

Adds CI, PR-Agent review and the auto-merge gate, copied from the app repo (exponential-im/exponential), plus a check that the exponential-sdk version this repo asks for is published. This repo had no .github folder.

What's added

  • .github/workflows/ci.yml (workflow CI, one job Build, typecheck & test): the SDK check below, then npm ci, npm run build, npm run typecheck, npm test, on every PR and every push to main. Node 22. Only scripts that are already in package.json.
  • .github/workflows/pr_agent.yml and .pr_agent.toml: review-only PR-Agent on GLM 5.3 through OpenRouter (Z.AI's endpoint only), one call per push, every finding tagged [blocker] or [minor]. The [config] and [openrouter] sections are the app's, unchanged. The house rules are rewritten for a CLI: JSON output, flags and exit codes are a contract (breaking changes need a major bump), nothing but the JSON document on stdout in JSON mode, errors go through handleError, API calls go through the SDK, the token is printed only by auth show --token, .js on relative imports, mocked-client tests.
  • .github/workflows/pr-agent-automerge.yml: squash-merges a PR into main once PR-Agent left nothing blocking on the latest commit and CI passed on it. [minor] findings don't block; untagged or [blocker] findings and security concerns do. Honours no-automerge, and pr-agent-ack only when positonic applied it.
  • CLAUDE.md: a "Who merges" section, matching the app's CLAUDE.md, and a note on the SDK check.

The exponential-sdk check

main currently can't run npm ci: package.json asks for exponential-sdk ^1.32.0 and package-lock.json still pins 1.31.0. The first commit here syncs the lockfile (only the exponential-sdk entry and the root version fields change), so CI has something that installs.

To stop that merging again:

  • .github/scripts/check-sdk-pin.mjs runs in CI before npm ci and fails with the cause and the fix when

    • package.json's exponential-sdk range has no published version on npm,
    • package-lock.json pins a version that isn't on npm, or takes the SDK from a local link, folder or tarball,
    • the lockfile's version doesn't satisfy package.json's range, or
    • the lockfile's integrity hash isn't the one npm serves (a lockfile made against a local build).

    It is a step in the required job, so the PR can't merge.

  • .github/workflows/sdk-bump.yml (optional extra): asks npm once an hour for a newer exponential-sdk and opens chore: bump exponential-sdk to <version>, setting the range to ^<version> and updating the lockfile. It opens the PR as the exponential-automerge App, so CI runs on it; it uses the same pr-agent-automerge environment and does nothing until that is set up. PR-Agent and the gate both skip bot-opened PRs, so you merge the bump once CI is green. Closing a bump declines that version. Turn it off with gh workflow disable sdk-bump.yml.

Where the gate differs from the app's

  • No database-schema rule (no Prisma here).
  • It checks CI itself, by name (REQUIRED_CHECKS = ['Build, typecheck & test']). The app's gate leaves "did the required checks pass?" to branch protection, which this repo doesn't have yet; without this, the gate would merge on a red or still-running build. Branch protection should still require the same check.
  • Without the App's credentials it does nothing, so this PR is safe to merge before the setup below.
  • Two fixes, from Codex's review of these PRs. Both holes are in the app repo's gate too, which I haven't touched:
    • pr-agent-ack is checked against the review it answers. The app's gate relies on the synchronize run to remove the label on a new push, and that run can be dropped (runs share one concurrency slot per branch, and a queued run is replaced by the next one queued), so an old acknowledgement could cover a newer commit. Here a label older than PR-Agent's review of the latest commit is ignored and removed.
    • A standalone review (the one PR-Agent posts when it can't update its comment) is read for findings in all three forms PR-Agent writes. The app's gate reads only the <details> form, so a [blocker] PR-Agent couldn't quote lines for is missed and the PR merges.

Setup for James (I can't do these)

Nothing merges on its own until all of these exist. Names are exactly as the workflows read them.

What Exact name Where
Repo secret: the OpenRouter key PR-Agent calls GLM with OPENROUTER_API_KEY Settings → Secrets and variables → Actions → Repository secrets
Environment, with deployment branches limited to main pr-agent-automerge Settings → Environments
Environment variable: the exponential-automerge App's client id (same value as in exponential-im/exponential) AUTOMERGE_APP_CLIENT_ID inside that environment
Environment secret: that App's private key, the whole .pem AUTOMERGE_APP_PRIVATE_KEY inside that environment
Install the exponential-automerge GitHub App on this repo (contents + pull requests: write) org Settings → GitHub Apps → exponential-automerge → Configure
Branch protection on main: require this status check Build, typecheck & test Settings → Branches
Label for acknowledging a blocker (I created no-automerge; this one is yours) pr-agent-ack Issues → Labels
Label the PRs that are already open, so a stray push to an old one can't land it no-automerge gh pr list --state open --json number -q '.[].number' | xargs -I{} gh pr edit {} --add-label no-automerge

allow_auto_merge stays off. The gate calls the merge API itself, pinned to the reviewed commit; it never queues GitHub auto-merge.

A workflow that names an environment creates it if it is missing, with no branch limit. So after this merges, pr-agent-automerge may already exist when you get to it: add the main limit before you store the key.

What happens before setup

  • CI needs nothing and runs from this PR onwards.
  • PR Agent review is red on this PR, and stays red here even once the secret exists: PR-Agent reads .pr_agent.toml from main, which has none until this merges, so it falls back to its default OpenAI model and stops for want of a key. After the merge it needs OPENROUTER_API_KEY, and is red on every PR without it. It is not meant to be a required check.
  • The gate does nothing at all while AUTOMERGE_APP_CLIENT_ID is missing: no comment, no merge, a notice in the run saying why.
  • The bump workflow does nothing either, for the same reason.

Checked

  • The CI steps, run locally as written in ci.yml on Node 22 from a clean npm ci: the SDK check, build, typecheck and 395 tests pass.
  • check-sdk-pin.mjs against 14 fixture lockfiles (unpublished range, unpublished pin, file: and git ranges, a linked SDK, a local tarball, a wrong integrity hash, a missing entry, a stale lockfile) and against main as it is today, where it fails with "package-lock.json doesn't match package.json".
  • actionlint is clean on all four workflows.
  • The gate's script, run against a fake GitHub API in 50 scenarios: it merges only on a clean review plus a green Build, typecheck & test on the same commit, and holds on a blocker, an untagged finding, a security concern, a partial or stale review, red, running or missing CI, a bot or fork PR, a draft, and no-automerge. pr-agent-ack from anyone but positonic is ignored, as is one older than the review, and it never overrides red CI. The 11 scenarios for the two fixes fail on the gate as first pushed and pass now.
  • The bump workflow's shell steps, run locally against a copy of main with gh and git push stubbed: it produces the same lockfile change as the first commit here, updates an open bump, leaves a closed one alone, and never lowers a range that is ahead of npm.
  • Not exercised for real: the merge itself, the App token, and the bump's push and PR, which need the setup above.

Not merged by me, and no pr-agent-ack label.

package.json on main asks for exponential-sdk ^1.32.0, but package-lock.json still pinned 1.31.0, so `npm ci` refused to install. Regenerated with `npm install --package-lock-only`; only the exponential-sdk entry and the root version fields change.
Copies the app repo's setup (exponential-im/exponential):

- ci.yml: npm ci, build, typecheck and test on every PR into main and on
  every push to main. One job, "Build, typecheck & test".
- pr_agent.yml + .pr_agent.toml: review-only PR-Agent on GLM 5.3 through
  OpenRouter, one call per push, findings tagged [blocker] or [minor]. The
  house rules are rewritten for a CLI.
- pr-agent-automerge.yml: squash-merges a PR into main once PR-Agent left
  nothing blocking on the latest commit and CI passed on it. Honours
  no-automerge and pr-agent-ack exactly as the app's gate does.

Differences from the app's gate: no database-schema rule; the CI check is
verified by name in the gate as well as by branch protection; and without
the App's credentials the gate does nothing, so this can merge before the
repo is set up.

New here, because this package depends on exponential-sdk:

- .github/scripts/check-sdk-pin.mjs runs in CI before npm ci and fails,
  saying what to do, when package.json's exponential-sdk range has no
  published version, when package-lock.json pins a version that is not on
  npm (or a local link or tarball), or when the two files disagree.
- sdk-bump.yml asks npm hourly for a newer exponential-sdk and opens a
  "chore: bump exponential-sdk to <version>" PR as the auto-merge App. A
  person merges it: the gate skips bot-opened PRs.

CLAUDE.md gains a "Who merges" section.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-15T22:07:07.572508Z bd84835 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bd848358d5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +378 to +380
const findings = [...body.matchAll(/<details><summary><a href='([^']*)'><strong>([\s\S]*?)<\/strong><\/a>/g)]
.map(m => finding(m[2].trim(), m[1], null));
if (!findings.length && !/No major issues detected/i.test(body)) return null;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Parse standalone findings without requiring a details wrapper

When a standalone review contains a [minor] finding with a code snippet and a [blocker] finding without one, this regex captures only the minor finding. PR-Agent's renderer emits a plain <a><strong>…</strong></a> when relevant lines are unavailable, rather than wrapping it in <details><summary>. Running readReview against that mixed output returns zero blockers; with green CI and no security concern, the gate therefore merges despite the explicit blocker. Parse both supported finding formats, or reject the review when findings cannot be fully accounted for.

Useful? React with 👍 / 👎.

…ding form

Two holes in the gate, both found by Codex's review and both inherited
from the app repo's copy:

- pr-agent-ack was only cleared by the `synchronize` run. Runs share one
  concurrency slot per branch and a queued run is replaced by the next one
  queued, so that run can be dropped and an old acknowledgement would then
  cover a newer commit. The label's timestamp is now checked against
  PR-Agent's review of the latest commit; an older one is ignored and
  removed.
- A standalone review (no state block) was parsed for findings in the
  `<details>` form only. PR-Agent also writes `<a><strong>` and bare
  `<strong>` findings when it can't quote the lines, so a [blocker] in
  that form was missed. Every finding in the focus-areas row now counts,
  and a row that can't be read holds the PR.

Also: CI runs on PRs into any branch, so a stacked PR already has its
check when it is retargeted to main; and the SDK bump never lowers a
range in package.json that is already ahead of npm.
@positonic
positonic merged commit ebd59b0 into main Nov 18, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant