ResearchOperator SurfaceHost Affordances

Host Affordance Bridge Preflight

Status: Partially implemented — Phase 0 evidence captured; core host screenshot/image paste is ready for operator opt-in testing behind JACKIN_HOST_ATTACH=1; opt-in Phase 1A host attach path added with terminal-color parity, image paste with chunked large-image transfer, explicit host image paste, explicit image-path staging, first GitHub host-open actions, visible URL open with modifier-hover disclosure, command-palette cursor URL open, host-link opt-out, Debug info host diagnostics path copy/reveal, launch-failure artifact reveal/open, minimal chunked file export, explicit export-and-reveal/open, cursor-path export/reveal/open, selected-text export/reveal/open, and modified-click visible-file export

Purpose

This page defines what jackin must prove before promoting the Host Affordance Bridge from opt-in implementation to default behavior. The bridge roadmap is the product/architecture contract. This preflight page is the measurement checklist: terminal events, OrbStack transport, clipboard readers, Amp xclip behavior, binary transfer shape, and GitHub CI URL lookup. The first bounded implementation slice has reached the operator-testable milestone for screenshot/image paste; remaining items here are promotion gates and live-validation gates, not blockers for testing that core opt-in path.

Required Outputs

The preflight/promotion trail must produce these artifacts. Phase 0 evidence plus the landed tests/docs are enough for the current opt-in operator test; default-on promotion still requires the live-validation artifacts named below.

  • A terminal-event report for Ghostty on macOS.
  • An OrbStack transport report for direct socket vs stdio attach-proxy, gathered from code inspection plus a separate throwaway non-jackin socket probe when live evidence is needed.
  • A clipboard-backend report for macOS, Linux X11, and Linux Wayland. macOS Phase 0 evidence is captured; Linux Wayland/X11 remain live-validation items.
  • An Amp clipboard tooling report that distinguishes helper-binary presence from a real display/clipboard bridge.
  • A binary-transfer recommendation for image paste and file export.
  • A GitHub CI URL query recommendation.
  • Roadmap updates recording every measured result and any implementation decision changed by the measurements.

Captured Evidence

The June 11, 2026 Phase 0 probes produced enough evidence to start implementation with a bounded first slice. The raw probe files were written under /tmp/jackin-hab-preflight/ during the measurement run; they are scratch artifacts, not committed product files.

AreaEvidenceImplementation consequence
Ghostty text pasteThe raw terminal probe saw ordinary pasted text bytes for text clipboard paste. Focus events appeared as ESC [ I / ESC [ O.Existing terminal byte-stream paste must keep working; image paste cannot be implemented by replacing text paste handling.
Ghostty image pasteFor raw screenshot, copied browser image, and copied Finder image file paste gestures, the terminal process did not receive usable image bytes.Image paste must use a host clipboard reader and typed transfer; Capsule input parsing alone cannot recover bytes that never arrive.
Ghostty mouse modifiersPlain motion/clicks emitted SGR mouse frames. Option motion used modifier-coded frames such as cb=43; Ctrl motion used frames such as cb=51. Cmd was not distinguishable in the process-visible stream in the manual probe.The first visible-URL gesture binds Alt/Option and Ctrl modified left-click only. Cmd-click must not be assumed until a future probe proves a process-visible signal.
macOS raw screenshot clipboardNative pasteboard probe returned public.png, 218596 bytes, magic 89 50 4e 47 0d 0a 1a 0a, plus a public.file-url item.macOS backend can read raw screenshot image bytes directly and should prefer PNG when available.
macOS browser copied imageProbe returned public.tiff 34262 bytes, magic 4d 4d 00 2a, plus com.apple.flat-rtfd, com.apple.webarchive, and public.html.macOS backend must handle TIFF source data or coerce browser clipboard images to PNG before staging.
macOS Finder copied image fileProbe returned public.file-url, UTF-16 and UTF-8 filename text, and com.apple.icns; it did not return image file contents as public.png.Finder copy should be treated as a file-reference source: resolve the file URL or require explicit path staging rather than assuming raw image data.
Plain text pathProbe returned public.utf8-plain-text bytes for /tmp/example.png plus public.html.Plain path remains ordinary text unless the operator invokes an explicit image-stage action.
OrbStack direct socketThrowaway OrbStack socket evidence showed the host can see the mounted socket inode but direct UnixStream::connect can fail.Host attach needs transport selection: direct socket where connect succeeds, docker exec -i ... attach-proxy fallback where it does not.
Amp without clipboard toolAmp showed Failed to copy selection: No clipboard tool found. Install xclip (X11) or wl-clipboard (Wayland), e.g. sudo apt install xclip.Construct image should include normal Linux clipboard tools for runtime compatibility.
Amp after installing xclipDirect xclip -selection clipboard -t image/png -o and xclip -selection clipboard -o returned Error: Can't open display: (null). Amp then showed Failed to write to clipboard via xclip/wl-copy.xclip/wl-clipboard presence is not a macOS clipboard bridge. It requires DISPLAY/Wayland access and, for X11, usually xauth plus an exposed display socket. jackin must not silently mount those host display sockets.
GitHub CI URL queriesgh pr view <url> --json statusCheckRollup exposes check objects with detail URLs. gh pr checks <url> --json bucket,completedAt,description,link,name,startedAt,state,workflow uses link; detailsUrl is not a valid field for gh pr checks.Implemented lookup asks for bucket,link,name,workflow, stores a validated HTTP(S) CI link, and prefers failed links before cancelled, pending, passing, and skipped links. If no link is useful, it falls back to statusCheckRollup.detailsUrl by failed/cancelled/timed-out, pending, success, skipped priority, then the PR checks tab.

Handled In This Preflight PR

  • Operator-facing test instructions now live in Host Affordances, so JACKIN_HOST_ATTACH=1 behavior can be validated from normal docs instead of only from roadmap notes.
  • The construct image now installs xclip, xauth, and wl-clipboard. This makes Amp and other Linux CLIs see expected clipboard helper binaries, but it deliberately does not expose the host display server or host clipboard.
  • The attach protocol is shared through jackin-protocol, so host runtime code and Capsule decode the same frame shapes.
  • jackin-capsule attach-proxy exists as a byte-blind stdio relay from docker exec -i to /jackin/run/jackin.sock, with a binary relay regression test.
  • jackin-runtime has host attach transport-selection helpers that choose direct socket when host connect succeeds and attach-proxy when the socket path is missing or connect fails.
  • JACKIN_HOST_ATTACH=1 opts foreground attach paths into the host-owned attach loop for reconnect, hardline, new shell sessions, and new agent sessions. The default path still uses the in-container interactive client until live validation proves parity.
  • The opt-in host path currently owns raw mode, stdin/stdout relay, SIGWINCH resize, focus/mouse bytes, cleanup, direct-socket transport, attach-proxy transport, and shared OSC 10/11 terminal-color probing. Live validation remains the gate before promoting it from opt-in to default.
  • The attach protocol now has a bounded ClipboardImage frame. Capsule validates image magic bytes, stages accepted images under /jackin/run/clipboard/, and pastes the staged container path into the focused pane when no dialog owns input. The staged path uses bracketed-paste wrappers when the focused pane has bracketed paste enabled.
  • Capsule now renders transient image-paste feedback in the constrained toast area for staged, dialog-blocked, no-writable-focused-pane, and rejected image payloads, so the operator sees the container path or rejection reason without overwriting pane content or chrome rows.
  • The opt-in host path now treats a lone Ctrl+V as a macOS image-paste probe: it uses osascript to read PNG-coerced host clipboard data first, TIFF image data second, then copied Finder file URLs read-only, sends a typed image frame when a valid in-cap image is available, and forwards the original Ctrl+V byte otherwise. Small images use the single-frame ClipboardImage path; larger images use ClipboardImageStart / ClipboardImageChunk / ClipboardImageEnd with 1 MiB chunks, a 64 MiB cap, offsets, and SHA-256 verification before Capsule staging. Capsule shows a transient receiving notice as soon as a chunked image transfer starts, so live validation can distinguish a slow transfer from a missing gesture. Browser TIFF, Finder file-url behavior, and Linux backends still need live validation.
  • The command palette now has Paste image from host clipboard. This explicit action sends a typed host request to run the same host clipboard image reader used by the Ctrl+V path, which gives operators a discoverable path when terminal paste gestures are consumed or unclear.
  • The command palette now has Stage image without pasting. This explicit action sends a typed host request to run the same host clipboard image reader used by the Ctrl+V path, then Capsule consumes the next image response in stage-only mode: it stages the image under /jackin/run/clipboard/, reports the staged path, and leaves focused pane input untouched.
  • The command palette now has Stage image from clipboard path. This explicit action handles the Phase 0 plain-text-path result without changing ordinary text paste: the host attach client reads host clipboard text only for this action, requires an absolute readable image path or file:// image URL, validates image magic bytes, sends the same image frames as direct paste on success, and returns ClipboardImageError for visible failure feedback on rejection.
  • Explicit host clipboard image/path requests now preserve host probe errors in the returned Capsule feedback. On Linux, missing WAYLAND_DISPLAY/DISPLAY, missing host-side wl-paste, and missing host-side xclip are visible to the operator; installing those tools inside the container remains runtime compatibility only unless a real display bridge is exposed.
  • GitHub context now records a best-effort CI URL from gh pr checks link fields, statusCheckRollup.detailsUrl, or the PR checks tab; renders separate copy/open PR/open CI actions; and emits a scheme-limited host-open attach frame handled by the opt-in host attach client. Live browser-open validation remains pending.
  • Alt-click and Ctrl-click on visible http, https, and mailto URLs in mouse-disabled panes now resolve through the same URL-token rules as double-click selection and emit the existing scheme-limited host-open attach frame. OSC 8 targets are preferred when available and use the same scheme allowlist. Unsupported visible and OSC 8 schemes consume the explicit host-open gesture, show Host link rejected: unsupported URL scheme, and emit no host-open frame. Alt/Ctrl passive hover over a safe URL renders a compositor-owned Open link: ... notice, emits the terminal pointer-shape cue when supported, and never writes hover bytes into the pane. The command palette also has Open link under cursor, which resolves the focused pane's terminal cursor through the same URL-token rules and reports a transient no-link notice when there is no host-open link under the cursor. Non-URL modified clicks keep the raw mouse fallback. Cmd-click remains unimplemented until live terminal evidence proves the correct signal shape.
  • JACKIN_OPEN_LINKS=deny / off / no now disables jackin host browser-open actions without changing normal terminal output or OSC 8 passthrough. Capsule suppresses GitHub/context and cursor-link host-open frames with visible feedback; modified-click falls back to normal mouse dispatch. The host attach client also refuses to spawn the host opener under the same opt-out.
  • Debug launches now inject JACKIN_RUN_DIAGNOSTICS_PATH alongside JACKIN_RUN_ID. The launch cockpit Debug info dialog and Capsule's Debug info dialog both copy that exact host JSONL path for the diagnostics row, with Capsule retaining the previous in-container path derivation only as a fallback for older launches. Both dialogs expose explicit R / O reveal shortcuts and a Reveal diagnostics row so Debug info owns a distinct copy target and reveal target before any in-pane agent UI is involved.
  • The launch failure popup now has host-side R reveal and O open actions for the visible run diagnostics and docker output artifact rows. It uses the host desktop reveal/open helpers, keeps the failure popup open, and shows visible feedback.
  • The command palette now has Export file, Export file and reveal, Export file and open, Export file under cursor, Export file under cursor and reveal, Export file under cursor and open, Export selected file, Export selected file and reveal, and Export selected file and open actions. The prompt actions canonicalize the requested container path; the cursor actions extract the visible token under the focused pane cursor; the selected-file actions use the active selection text; and Alt/Ctrl modified-click in mouse-disabled panes exports a visible token only after URL-open resolution declines and the same file-export validator accepts it. Capsule allows only workspace paths and /jackin/run/, rejects non-regular files, caps the first implementation at 64 MiB, and streams FileExportStart / FileExportChunk / FileExportEnd frames with 1 MiB chunks, SHA-256, and explicit reveal/open flags. The opt-in host attach client writes to ~/Downloads/jackin/<instance>/ through a temporary file, verifies size and digest, then renames to a collision-resistant final path. It removes temporary files after interrupted, malformed, or idle transfers, and asks the host OS file manager/opener to reveal or open the copy only for the explicit post-export action. Live validation through a throwaway jackin instance remains pending.
  • Host-open, clipboard-image, and file-export outcomes now return bounded feedback to Capsule so success and rejection are visible in the TUI instead of only in host logs. The host attach client clamps both HostNotice and ClipboardImageError messages before encoding, so an overlong host-side error cannot make the visible feedback path fail. Host desktop open/reveal helpers now wait for the host command exit status and report nonzero exits without logging stdout/stderr or private payloads. Clipboard-image compact logs record the host paste trigger, no-image text-forward fallback, extension, byte count, staged container path, transfer stage, and coarse rejection reason with full rejection detail gated behind debug logs. File-export debug and compact logs record source category, basename, byte count, destination category, destination basename, and coarse rejection reason instead of full container or host paths; operator-visible notices still show the exported destination.
  • The current implementation milestone stops at opt-in operator testing of the core screenshot/image paste path. The unfinished items below remain intentional follow-up work for live validation, default-on promotion, Linux host backends, and broader affordance hardening.

1. Terminal Event Probe

Build or add a small host-side probe that runs outside Docker in Ghostty and logs raw terminal input bytes plus decoded terminal events. It must not depend on Capsule internals.

Measure:

  • Ctrl+V with plain text clipboard.
  • Cmd+V with plain text clipboard.
  • Ctrl+V with raw screenshot clipboard.
  • Cmd+V with raw screenshot clipboard.
  • Ctrl+V with copied browser image.
  • Cmd+V with copied browser image.
  • Ctrl+V with copied Finder image file.
  • Cmd+V with copied Finder image file.
  • Cmd-click, Option-click, and Ctrl-click on a visible URL.
  • Mouse move while holding Cmd, Option, and Ctrl.

Record:

  • Raw bytes received.
  • Whether bracketed paste wrappers appear.
  • Whether modifier state reaches the process.
  • Whether Ghostty consumes the gesture and sends nothing.
  • Whether behavior differs in normal screen vs alternate screen.

Pass condition: implementation knows which gestures jackin can bind directly and which must use command palette actions because the terminal consumes them.

2. OrbStack Transport Probe

Do not inspect or mutate existing jackin role containers. Use code inspection first. If live evidence is required, use a separate throwaway non-jackin container with a private host temp directory mounted into it, create one Unix socket under that mount, test host connect behavior, then remove only that throwaway container and temp directory.

Measure:

  • Host direct connect to ~/.jackin/sockets/<container>/jackin.sock.
  • Attach Hello handshake over direct host socket if connect succeeds.
  • docker exec -i <container> /jackin/runtime/jackin-capsule attach-proxy relay once attach-proxy prototype exists.
  • Binary relay integrity using non-text bytes, not only printable text.
  • Throughput/latency for representative image payloads: 1 MiB, 4 MiB, 16 MiB.

Record:

  • Direct socket works or fails.
  • Exact failure error if direct socket fails.
  • Whether stdio relay preserves attach frames.
  • Whether stdio relay has payload/latency limits that affect image paste or file export.

Pass condition: transport selection is measured: direct socket when usable, attach-proxy when not. OrbStack must not be guessed from Docker Desktop behavior.

3. Clipboard Backend Probe

Clipboard image reading belongs on the host side. Container tools cannot read the macOS host clipboard unless a separate host display/clipboard bridge is explicitly exposed.

macOS

Measure:

  • Raw screenshot copied to clipboard.
  • Browser image copied to clipboard.
  • Finder image file copied to clipboard.
  • Plain text path copied manually.

Backends to test:

  • Native pasteboard access through Rust/ObjC/Swift or a maintained crate if available.
  • osascript PNG/TIFF coercion as fallback/test oracle.

Record:

  • Whether bytes were read.
  • Format detected.
  • Magic-byte validation result.
  • Whether source was raw image data, file URL, file promise, or text.

Linux X11

xclip must be tested as host-side X11 backend:

xclip -selection clipboard -t image/png -o

Also test JPEG/GIF/WebP MIME variants if available.

Record:

  • Whether xclip exists.
  • Whether DISPLAY and XAUTHORITY allow clipboard access.
  • Exit status and stderr when no image exists.
  • Exact MIME types that work.

Linux Wayland

Test wl-paste from wl-clipboard:

wl-paste --type image/png

Record:

  • Whether wl-paste exists.
  • Whether WAYLAND_DISPLAY and runtime socket access allow clipboard access.
  • Exit status and stderr when no image exists.
  • Exact MIME types that work.

Pass condition: host clipboard reader backend order is known, missing-tool messages are known, and jackin can distinguish image clipboard from text/path clipboard.

4. Amp And xclip

Amp image support accepts image file paths in the CLI: paste a file path, drag a file into the terminal, or use @ file mentions. See Amp CLI image support. That supports jackin's generic bridge: stage image into container, then paste container-readable path.

Separate question: Amp may also use xclip for direct clipboard integration on Linux. jackin must prove this before relying on it.

Required experiments:

  • Run Amp in a jackin container without xclip.
  • Run Amp in a jackin container with xclip.
  • Put a fake xclip earlier in PATH that logs argv, env, stdin size, stdout behavior, and exit code.
  • Press Amp's image/text paste gesture in the Amp TUI.
  • Test with DISPLAY unset.
  • Test with invalid DISPLAY.
  • Test with valid X11 forwarding if available.
  • Test with XAUTHORITY absent and present if X11 forwarding is used.

Record:

  • Whether Amp invokes xclip.
  • Exact xclip argv.
  • Whether Amp asks for plain text or image MIME.
  • Whether Amp needs xauth, DISPLAY, mounted X11 socket, or other env.
  • Whether Amp degrades cleanly when xclip is missing.
  • Whether installing xclip alone changes behavior without X11 access.

Decision rules:

  • xclip should be installed in the construct image if Amp or another built-in runtime expects it as a normal Linux dependency.
  • xauth should be installed with xclip if measured Amp behavior or X11 clipboard access requires it.
  • Installing xclip in the container does not replace the host clipboard bridge. It only helps in Linux/X11 sessions where the container has explicit X11 access.
  • Do not silently mount /tmp/.X11-unix, Wayland sockets, XAUTHORITY, or host display sockets. Those are host-side exposure and require separate opt-in surfaced behavior.

Pass condition: jackin knows whether xclip is required runtime tooling, whether xauth is also required, and whether Amp uses xclip for text, image, or neither.

5. Binary Transfer Probe

The current attach/control payload cap is 4 MiB. Image paste and file export need measured transfer behavior.

Evaluate:

  • Chunked attach frames with transfer id, chunk index, total bytes, cancellation, cap, and digest.
  • Negotiated side stream.
  • Host write into the bind-mounted host backing dir for /jackin/run/clipboard/, followed by Capsule validation and path paste.

Measure:

  • 1 MiB PNG.
  • 4 MiB PNG.
  • 16 MiB PNG.
  • 64 MiB regular file export sample.
  • Cancellation mid-transfer.
  • Digest mismatch handling.

Pass condition: image paste and file export have separate transfer recommendations. File export must use chunking or side stream; it must not depend on one oversized attach frame.

Current result: passed for the current implementation slice. Image paste uses the bounded single-frame ClientFrame::ClipboardImage path below the 16 MiB frame cap, then switches to chunked client frames with transfer id, offsets, size, 1 MiB chunks, SHA-256 verification, a visible receiving notice, idle-transfer cleanup, and a 64 MiB first-pass cap. Capsule cancels a malformed image-chunk transfer id so a clean retry can reuse it. File export uses chunked server frames with transfer id, offsets, size, 1 MiB chunks, SHA-256 verification, interrupted/malformed/idle temp cleanup, and a 64 MiB first-pass cap. Remaining probe work is live interrupted-transfer behavior, live export/image latency, and whether very large transfers should move from attach chunks to a negotiated side stream.

6. GitHub CI URL Probe

Current Capsule GitHub context has PR URL and check bucket counts, not failed job/check URLs. Probe exact gh shape.

Test:

gh pr view <pr-url> --json statusCheckRollup
gh pr checks <pr-url> --json bucket,completedAt,description,link,name,startedAt,state,workflow

Record:

  • Fields available for one failed check.
  • Fields available for several failed checks in one workflow run.
  • Fields available for failures across multiple workflow runs.
  • Whether gh pr checks link, statusCheckRollup.detailsUrl, or another field is the best browser target. detailsUrl is not a valid gh pr checks field in the measured CLI shape.
  • Behavior when gh pr checks exits with failure status because checks failed.

Pass condition: GitHub context can store or resolve a real CI target URL. Open CI button must not ship with only bucket counts.

Current result: passed for the first implementation slice. Capsule stores one validated HTTP(S) CI target from gh pr checks link, statusCheckRollup.detailsUrl, or the PR checks tab, and the dialog exposes an explicit Open CI action. Remaining work is live host-browser validation.

Observed PR #565 sample on June 12, 2026:

{
  "gh_pr_checks": [
    {
      "bucket": "pass",
      "state": "SUCCESS",
      "name": "ci-required",
      "workflow": "CI",
      "link": "https://github.com/jackin-project/jackin/actions/runs/27393524623/job/80957222650"
    },
    {
      "bucket": "pass",
      "state": "SUCCESS",
      "name": "docs-required",
      "workflow": "Docs",
      "link": "https://github.com/jackin-project/jackin/actions/runs/27393524611/job/80956520066"
    }
  ],
  "gh_pr_view_statusCheckRollup": [
    {
      "name": "Select runner lane",
      "status": "COMPLETED",
      "conclusion": "SUCCESS",
      "detailsUrl": "https://github.com/jackin-project/jackin/actions/runs/27393524623/job/80955960676"
    }
  ],
  "gh_pr_checks_detailsUrl_error": "Unknown JSON field: \"detailsUrl\"; available fields include bucket, completedAt, description, event, link, name, startedAt, state, workflow."
}

Minimal Manual Proof

Before promoting host attach from opt-in to default, prove this flow manually or with throwaway prototype code:

  1. Host reads image from clipboard.
  2. Host transfers or writes image into running container.
  3. Capsule or prototype validates bytes and stages file at /jackin/run/clipboard/<name>.png.
  4. Host/Capsule pastes that container path into Amp, Codex, or Claude Code.
  5. Agent accepts path as image input.
  6. OrbStack transport path is known: direct socket or stdio relay.
  7. Amp has measured xclip behavior.

Promotion Gate

The host affordance bridge may become default behavior only after every required output above is either:

  • measured and recorded in this page or the main bridge roadmap;
  • converted into a concrete V1 implementation decision; or
  • marked blocked by a proven platform/tool limit with exact evidence.

Do not replace missing measurements with assumptions. The bridge exists to remove host/container ambiguity; default behavior must be promoted from measured boundaries.

On this page