Console-native PR verification: design and prior art

This page records the design rationale for the console-native, recipe-driven layer of workspace-native PR verification that Workspace-Native PR Verification defers past its shipped jackin-dev pr sync slice: the review-mode sidebar, the repo-committed verification recipe, isolation profiles, companion-role pinning, and documentation-preview automation, plus the prior-art survey that motivated building this instead of adopting an existing tool.

Problem

jackin❯ pull requests carry a hand-written Verify-locally block: clone or update the repo, fetch the PR branch, build the right binaries, point a run at isolated state, run a smoke command. The block is copy-paste boilerplate, drifts between PRs, and every review re-derives the same steps by hand. With one human contributor and no second reviewer, this manual loop runs on every PR. And the loop is not unique to jackin❯: every project developed inside a jackin❯ workspace has the same "check out this PR, build what it changed, run the smoke that proves it" ritual. jackin❯ verifying its own PRs is just the first case — the design is generic, and the per-project recipe that drives it lives in each project, not in jackin❯.

Three things make the console the right home for the richer flow, anchored to the instance the operator is already working in:

An instance already knows its pull request. When the operator is working on a PR inside an instance, jackin-capsule — the in-container daemon — already tracks the working directory's branch and resolves it to the matching open pull request, with title and check status (see crates/jackin-capsule/src/pull_request.rs and crates/jackin-capsule/src/pr_context.rs). The console can show those details the moment the instance is selected, so "verify this" is a natural next step from where the operator already is.
A PR sometimes needs a companion role branch. A change in the jackin❯ repo can require a matching change in a role repo. Verifying that PR means running it with a specific role pinned to a specific branch — something jackin❯ already knows how to do.
Testing jackin❯ itself needs clean state. Verifying a jackin❯ PR by running the freshly built binary must not read or mutate the operator's real configuration and state, and must not disturb the live instance they are working in. jackin❯ can already redirect its configuration and state to alternate locations, so an isolated verification environment is a matter of wiring, not new machinery.

Console-native review mode

The console already shows a left sidebar of workspaces, each expandable to its instances, with a right detail block for whatever row is selected. Verification reuses that layout — it is not a separate screen — and adds two things: pull request details in the detail block, and a review mode for the sidebar.

Step 1 — Select an instance that's on a PR

jackin-capsule reports the branch each instance's working directory is on and resolves it to the matching open pull request. Selecting such an instance shows its pull request details in the right block: number, title, base and head branches, open/draft/merged state, and CI check status. An instance not on a PR branch simply shows no pull-request section — the review action is offered only when there is a PR to review.

┌─ jackin ───────────────────────┬─ the-architect · running ──────────────────┐
│ ▾ jackin    ~/Projects/jackin   │ branch   feature/jackin-dev-roadmap          │
│   ▸ ● the-architect  running    │                                              │
│     ○ agent-smith    stopped    │ Pull request                                 │
│ ▸ agent-smith                   │   #480  docs(roadmap): workspace-native PR…  │
│ ▸ docs-site                     │   main ← feature/jackin-dev-roadmap          │
│                                 │   OPEN · checks passing · +135 −173          │
├─────────────────────────────────┴──────────────────────────────────────────┤
│ enter attach · R review pull request · s shell · q quit                       │
└──────────────────────────────────────────────────────────────────────────┘

Step 2 — Switch to PR review

Pressing the review key on a PR-bearing instance switches the left sidebar into review mode: the workspace/instance tree is replaced by the list of verification commands available for that PR, with the commands the PR's changes touch listed first. The right block becomes the output of the running or selected command, plus the PR context. A key returns to normal navigation.

┌─ Review · #480 docs(roadmap)… ─────────────┬─ docs-build · bun run build ──────┐
│ Relevant to this PR (docs)                  │ $ bun run build                    │
│  ▸ docs-build   [build]  ● passed           │ ✓ 142 pages built in 3.1s          │
│    docs         serve    ○ idle             │ ✓ link check ok                    │
│                                             │                                    │
│ Other commands                              │ ── verifying against ──            │
│    cli          [build]  ○ idle             │ isolated checkout of #480 head     │
│    console               ○ idle             │ config: your real config (copy)    │
│    test                  ○ idle             │ role: none · cache: shared         │
├──────────────────────────────────────────────┴───────────────────────────────┤
│ enter run · a all-relevant · A all · r refresh · s shell · o open · c settings · esc back │
└──────────────────────────────────────────────────────────────────────────────┘

Step 3 — Run and verify

Select a command and run it: build, test, start the docs server, or any command the repo's recipe defines. The operator runs the focused command, every relevant command in order, or all of them; opens a shell in the checkout to poke around or run something by hand; refreshes the PR; opens any served pages (below); and adjusts the verification settings (isolation, companion role, build cache — all with sensible defaults). Run-state badges show idle, running, passed, or failed; finishing shows a pass/fail summary. Commands the diff did not touch are still listed (never silently hidden), just not pre-selected.

Shell into the checkout. A shell action drops the operator into an interactive shell at the root of the isolated checkout — the same throwaway worktree the recipe commands run in, and with the same environment those commands get: the redirected configuration and state, the freshly built binary ahead on PATH (so jackin resolves to the binary under test), the companion-role selection, and every other variable the verification defines. So anything run by hand — git log, cargo build, jackin console, a one-off script — resolves the same binary, sees the same config and state, and behaves exactly like a recipe command, and still cannot touch the operator's real setup. On exit the terminal returns to review mode.

Refresh to rebuild. Refresh re-fetches the PR's latest pushed commits and hard-resets the local checkout to the new head — the right response to a force-pushed branch — so the next build runs against the updated code. The PR details (head commit, checks) update with it. Re-running the build set after a refresh is the local rebuild loop: pull the new commits in, rebuild, re-verify, without leaving the surface.

The live instance is never touched. Selecting the instance only identifies the pull request. Running review checks that PR's head out into a separate, throwaway host-side checkout and runs the recipe there — including the shell — against an isolated configuration and state. The operator's running session keeps going undisturbed, and jackin❯ can verify a PR against its own code without risking the real setup. Because the review surface is rich, debug mode does not paint over it: the full firehose of every command and its output is captured to a diagnostics run file, per Debug output never reaches a rich full-screen TUI. To triage a failed command, the operator shares the run id rather than pasting scrollback.

Repo recipe design

What jackin❯ can build, run, and serve for a repo is declared in a verification recipe committed at the root of the project repo, so it versions with the code — a PR that changes how the project is built or verified updates the recipe in the same diff. There is one command shape; optional fields select behaviour rather than forking into separate kinds:

mark a command as part of the build set (run in order, stop on first failure);
give it path globs, so jackin❯ can tell whether a PR's changes make it relevant;
mark it as a long-running server that opens the PR's changed docs pages (below).

The review actions are views over this one list: "build" runs every build-set command; "run" runs any command by name; trailing arguments are appended verbatim.

# jackin.tasks.toml — verification recipe for this repo
version = 1

[[task]]
name = "cli"
run = "cargo build --bin jackin"
build = true                         # part of the build set
paths = ["src/**", "Cargo.*"]        # relevance: lit up by code changes

[[task]]
name = "docs-build"
run = "bun run build"
cwd = "docs"                         # relative to the checkout root
build = true
paths = ["docs/**"]

[[task]]
name = "console"
run = "jackin console"               # resolves to the freshly built binary, not the installed one

[[task]]
name = "test"
run = "cargo nextest run"

[[task]]
name = "docs"
run = "bun run dev"                  # long-running; kept up until the operator quits
cwd = "docs"
# marking a command as a server tells jackin❯ to wait for it and open the PR's changed pages:
serve = { ready = "http://localhost:3000/", route_from = { dir = "docs/content/docs", strip_suffix = ".mdx", index = "index", base = "/" } }

Diff-driven relevance. jackin❯ reads the PR's changed file list once and matches it against each command's path globs. The matching commands are the PR's relevant set — surfaced first and run by "all relevant". Globs choose which command is relevant (the code build vs the docs build vs the smoke), not which source files a build recompiles — the compiler and the docs bundler already detect file-level changes inside a command. A manifest or lockfile edit lights up every code command; a docs edit lights up the docs build and the server. Non-matching commands are shown but not auto-selected; "all" runs everything; an empty or unknown change set falls back to treating everything as relevant. Nothing is silently skipped — what was de-selected, and why, is logged.

When jackin❯ verifies itself, a command that launches jackin❯ (a console smoke, a load) must run the freshly built binary, not the operator's installed one; the runner arranges that, and the launched binary's own runs are sandboxed by the chosen isolation profile.

This recipe is self-contained — each command carries its own invocation, so a repo is verifiable without any other tooling. It is related to other task-shaped surfaces on the roadmap (operator hotkeys, a future task-source abstraction) but deliberately stays its own thing: a maintainer verification recipe committed in the repo under test, with a different audience and different fields from an operator's in-console hotkeys. A shared resolver is worth revisiting only once task source abstraction defines a concrete shape. It is not a versioned jackin❯ schema: it lives in the repo under test, jackin❯ is pre-release, and a breaking shape change simply fails to parse.

Isolation profiles design

jackin❯ keeps two things separate: its configuration (workspaces, mounts, auth, trust) and its state (running instances, caches, role checkouts, agent homes). A verification run can redirect both to throwaway locations, chosen in the review-mode settings:

Against my real config (default) — verify against the operator's real configuration, with no state. jackin❯ takes a copy of the real configuration as read-only input and starts with empty state. Every write the built binary makes — schema migrations, "last role", trust-on-first-use, new instance state — lands in the throwaway copy and is discarded when the verification's disk is reclaimed. The real configuration is never written; the real state is never read.
Clean room — both configuration and state start empty. jackin❯ from zero, for first-run behaviour and defaults.

The configuration is always copied, never referenced: a PR that bumps a configuration or workspace schema migrates files in place on load, and pointing a verification run at the live configuration would let it rewrite the operator's real workspace files to the PR's schema version. The copy is what makes a schema-bumping PR safe to verify.

The shipped jackin-dev pr sync --config copy|blank flag (see Workspace-Native PR Verification) already implements the copy-vs-blank choice as a one-shot sync-time flag; this section's design is the review-mode setting that exposes the same choice interactively, per-run, from inside the console.

Companion role pinning design

A PR that pairs a CLI change with a role change needs jackin❯ inner launches to use a specific role at a specific branch. The companion-role setting records that pairing: any command that launches jackin❯ is invoked with the chosen role and, when a branch is given, that role checked out at that branch. With no companion role chosen, commands use whatever the isolated configuration would pick by default.

Documentation preview automation design

Some PRs change documentation, and the review need is "spin up the changed docs locally and open the pages this PR touched" — the PR template's documentation walk, automated. A command marked as a server is run like any other, but jackin❯ treats it specially: it starts the server, waits until it responds, then derives the URLs for the pages this PR changed (from the same changed-file list used for relevance) and opens each one. If none match, it opens the site root. The server stays up until the operator quits, then is shut down cleanly. The server command, port, and URL mapping are declared in the recipe, never hardcoded in jackin❯ — so a change of docs engine is a one-line edit to the recipe, not a jackin❯ code change. (This site's own move between docs frameworks changed exactly those values — the dev-server command and port — which is what the recipe absorbs.)

Per-pull-request bundle metadata (future)

For each PR it verifies, jackin❯ keeps a self-contained, throwaway bundle, isolated from the operator's working repos, the live instance, and the real setup:

a checkout of the PR at its head commit;
the isolated configuration (a copy of the real one, or empty) and the empty state home for the run;
enough metadata (number, title, branch, base, head commit, last-refresh time, state, chosen isolation) to show the verification's freshness without re-hitting the network.

Refreshing re-fetches the PR head and resets the checkout — the right response to a force-pushed branch. Reclaiming drops a PR's bundle and reports the disk freed. A single PR cannot be verified twice at once. The exact on-disk location is intentionally left to the implementation, since how jackin❯ stores console state is being reworked; the guarantee to preserve is that the bundle is isolated, reclaimable, and never mixed with the operator's real config, state, repos, or running instances.

The shipped jackin-dev pr status <PR_NUMBER> command already reports local/remote head SHAs and freshness by asking git and GitHub directly at call time (see crates/jackin-dev/src/main.rs); it does not yet persist its own metadata file, and jackin-dev pr clean does not yet report the disk freed. Both are open follow-ups, not blocking gaps, for this design.

Host-side guarantees

Per jackin❯ rule that it must never mutate the host machine silently, verification is operator-invoked, and its writes stay confined:

it writes only inside its own throwaway verification bundles and the role checkout cache — never the operator's working repos, never the live instance, never the real configuration or state;
the real configuration is copied, never written, so even a schema-bumping PR cannot migrate the operator's live workspace files;
detecting the instance's pull request, reading the PR's diff, and reading the host's GitHub auth are all read-only — verification never rewrites host git config or auth;
the saved workspace is never modified.

Risks, edge cases, failure modes

Instance not on a PR branch — no pull-request section in the detail block and no review action; the common "just working, no PR yet" case stays uncluttered.
PR lookup slow or failing — the detail block shows the branch with a loading or "no open PR" note; it never blocks instance selection. (jackin-capsule already caches the lookup per branch.)
GitHub CLI missing or unauthenticated — detection degrades to "branch only"; review, when entered, shows a one-line install/login hint instead of failing opaquely.
Which binary a command runs — when verifying jackin❯ itself, commands must run the freshly built binary, or a console smoke silently tests the wrong one.
Build cache — shared is fast but can in rare cases bleed state between PRs; the Fresh setting forces a cold build, and which was used is logged.
Fork, force-push, closed/merged — checkout uses the PR head ref directly; refresh resets hard; a closed/merged PR is flagged but still reviewable for post-merge inspection.
No recipe in the repo — review mode opens with a one-line "no recipe in this repo" notice and no runnable commands.
Server lifecycle — a server command is a child that must be torn down on quit, with its whole process group signalled so workers don't orphan, and a readiness timeout so a server that never binds fails loudly instead of hanging.
Shell / interactive child — the checkout shell hands the terminal over while it runs and must restore the review surface cleanly on exit; it inherits the full command environment (redirected configuration and state, built binary on PATH, companion role, any defined variables) so a hand-run command matches a recipe command exactly and defaults to the isolated bundle, not the operator's real setup.
Doneness — review shows a pass/fail summary and badges a failed command ✗. A server command is interactive, not pass/fail, so it does not count toward the verdict.

Schema-versioning note

If a future implementation needs to persist verification settings in the workspace configuration, that touches a versioned schema and pulls in jackin❯ schema-migration requirements. The design keeps verify state out of the workspace configuration (repo-committed recipe plus throwaway per-PR bundles) so the first slices need no schema bump.

Prior art

No existing tool covers the host-side-verify + companion-role-branch + isolated-jackin❯-state combination, reached from the instance the operator is working in; the closest pieces would have to be glued by hand:

gh pr checkout — checkout by number; no build/run recipe, no isolation.
gh-worktree — checks a PR out into a worktree; worktree only.
HighReview — local PR review via worktree with IDE/AI analysis; diff-review focus, no configurable build/run tasks.
gh-review — terminal diff/comment/approve UI; review surface, not build/verify.
Nx / Turborepo / Bazel / Buck2 — affected-target selective rebuild; prior art for diff-driven relevance, not PR-scoped review.
just / mise — repo-committed task runners; tasks but no PR checkout or isolation.

The near-term novel piece is a shipped developer binary that prepares a host-side PR checkout with isolated jackin❯ configuration and state so jackin❯ can verify itself safely (shipped as jackin-dev pr sync, see Workspace-Native PR Verification). The later console-native flow described on this page adds instance anchoring, recipe execution, documentation preview, and companion-role-branch pinning.

Future phases

Phase 1 — PR details in the instance view. Surface the pull request jackin-capsule already detects for an instance's branch in the right detail block: number, title, base/head, state, checks.
Phase 2 — Review mode + run-all baseline. The sidebar mode switch on a PR-bearing instance; the command list; check the PR head out host-side; run the whole build set against the freshly built binary, with a pass/fail summary.
Phase 3 — In-surface verbs + relevance (the diff-driven spine). Build / run-named / refresh / shell-into-checkout from review mode; refresh re-fetches the PR head and resets the checkout so a rebuild picks up new commits; map the PR's changed files to the relevant commands and list them first; suggestion-not-skip; de-selection logging.
Phase 4 — Documentation preview. The server command: start it, wait for ready, derive and open the PR's changed pages, tear down cleanly on quit.
Phase 5 — Isolation profiles. The review-mode setting "against my real config" vs "clean room".
Phase 6 — Companion role pinning. The review-mode setting of a role and optional branch, threaded into inner jackin❯ launches.

Console-native PR verification: design and prior art

On this page