# 01 — Host affordance evidence (https://jackin.tailrocks.com/research/product/operator-experience/host-affordances/host-affordance-bridge-design/01-host-affordance-evidence/)



## Summary [#summary]

Host clipboard images do not reliably enter a container attach stream. Existing terminal and remote-session tools support a host-owned, typed bridge that stages validated data across the isolation boundary.

## Question and scope [#question-and-scope]

Which host, terminal, and remote-session mechanisms can safely restore clipboard, link, and file affordances for containerized agent sessions?

## Method [#method]

The evidence compares public Codex and Claude behavior with terminal, clipboard, remote-session, and multiplexer mechanisms, including license and isolation constraints.

## Findings [#findings]

### Host CLIs [#host-clis]

The public Codex CLI documentation says Codex supports image inputs in two ways: paste images into the interactive composer, or pass image file paths with `--image` / `-i`. It does not document the internal paste implementation, but the architecture implication is clear: a local Codex process has access to the host clipboard and local image files before it constructs the model request. See [Codex CLI features](https://learn.chatgpt.com/docs/codex/cli).

Claude Code exposes similar behavior in practice: when run directly on the host, it detects an image in the clipboard and inserts an image placeholder into its prompt composer. That does not prove a public protocol jackin❯ can rely on; it proves the host-local process has a capability the in-container process lacks.

### herdr [#herdr]

[herdr](https://github.com/ogulcancelik/herdr) is the closest public reference because it is a Rust terminal-native agent multiplexer. It is not embedded in jackin❯ because herdr is AGPL-3.0 while jackin❯ is Apache-2.0, but its design is directly informative.

The relevant herdr shape:

1. The host-side client watches terminal input for a paste trigger such as `Ctrl+V` or an empty bracketed-paste sequence.
2. On that trigger, it asks the host platform for an image clipboard payload.
3. macOS reads the pasteboard through AppleScript by coercing clipboard data to PNG.
4. Linux reads image MIME types through `wl-paste` on Wayland or `xclip` on X11.
5. The client validates and caps the image payload, then sends a typed `ClipboardImage { extension, data }` message over herdr's client/server protocol.
6. The server stages the image to a private temp file and routes the staged path as a paste event to the active pane.

The files that matter in herdr are `src/client/mod.rs`, `src/platform/macos.rs`, `src/platform/linux.rs`, `src/server/clipboard_image.rs`, `src/server/headless.rs`, and `src/server/terminal_attach.rs`. jackin❯ borrowed the architecture and validation ideas, not the code.

### Ghostty and terminal behavior [#ghostty-and-terminal-behavior]

Ghostty already supports terminal graphics display through the Kitty graphics protocol, and it can paste file-backed images in some flows, but there is an explicit Ghostty discussion showing a gap for screenshots captured directly to the macOS clipboard: [ghostty-org/ghostty discussion #10478](https://github.com/ghostty-org/ghostty/discussions/10478). That distinction matters. A file copied from Finder can become a pasteable path or terminal image indicator; a screenshot copied directly to the pasteboard may never become a path or byte stream for the child process.

This confirmed the jackin❯ design should not assume "image paste" is one terminal protocol. It has to handle at least three host clipboard shapes:

| Clipboard shape                       | Example                                                                          | Correct jackin❯ behavior                                                                                     |
| ------------------------------------- | -------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| Raw image data                        | macOS screenshot copied with `Shift+Cmd+Ctrl+4`, image copied from a browser     | Read host clipboard bytes, validate, stage inside the container, paste readable container path.              |
| File promise / file URL / copied file | Screenshot saved to disk then copied in Finder, image copied from a file manager | Resolve the host file if permitted, copy or bind/stage it into the container, paste readable container path. |
| Plain text path                       | Operator copied `/path/to/image.png` text                                        | Treat as ordinary text unless an explicit image-paste command says "stage this path."                        |

### Existing jackin❯ clipboard surface [#existing-jackin-clipboard-surface]

jackin-capsule already had clipboard behavior that had to remain intact:

* Pane programs can write text to the host clipboard through OSC 52 passthrough only when the operator opted in with `JACKIN_OSC52=allow`.
* jackin-owned actions such as text selection, double-click word copy, Debug info copyable values, and launch-failure copy targets emit OSC 52 from the multiplexer.
* Those copy actions render visible feedback, such as the `Selection copied` toast, because terminals may accept, deny, or silently ignore clipboard writes.
* Bracketed paste from the host terminal is mirrored from the focused pane and forwarded as ordinary input bytes.

The screenshot-driven lesson is that image paste had to join this surface deliberately. It should not be a special case hidden in the input parser, and it should not replace existing OSC 52 text copy behavior. The correct model is a clipboard boundary with explicit direction, ownership, and permission:

| Direction                                | Mechanism                                                                                     | Boundary owner                                      |
| ---------------------------------------- | --------------------------------------------------------------------------------------------- | --------------------------------------------------- |
| Pane writes text to host clipboard       | OSC 52 when `JACKIN_OSC52=allow`                                                              | Capsule output policy and host terminal policy.     |
| jackin❯ UI writes text to host clipboard | OSC 52, with visible copy feedback                                                            | Capsule-owned output path and host terminal policy. |
| Host text paste enters focused pane      | Terminal byte stream, bracketed-paste-aware                                                   | Host terminal plus attach client.                   |
| Host image paste enters focused pane     | Host clipboard reader, typed `ClipboardImage` frame, container staging, pasted container path | Host attach client plus Capsule staging.            |
| Agent reads host clipboard directly      | Not allowed                                                                                   | No implementation path.                             |

This keeps copy-out, text paste-in, and image paste-in separate. That separation is a correctness property, not just an implementation preference: text copy is terminal-mediated, text paste is byte-stream-mediated, and image paste requires a host API read plus a file transfer into the container.

### Adjacent host affordance: opening links [#adjacent-host-affordance-opening-links]

The same host/container boundary appears with hyperlinks. Capsule already forwards safe OSC 8 hyperlinks (`http`, `https`, and `mailto`) to the outer terminal, and it also renders jackin-owned hyperlink regions for chrome and dialogs. That is correct but incomplete: not every terminal exposes a convenient clickable-link gesture, and some visible links are plain text rather than OSC 8 regions.

The design goal was a host attach client that owns an explicit "open link under cursor" affordance:

1. The operator performs an explicit gesture, such as `Cmd`-click on macOS if the terminal exposes that modifier to the program, `Ctrl`-click where that is the portable terminal modifier, or a command palette action such as "Open link under cursor".
2. The host attach client asks Capsule which safe link is under the cursor, or derives the visible URL from the current pane snapshot using the same URL-token semantics as double-click selection.
3. The host attach client opens the URL with the host OS browser action (`open` on macOS, `xdg-open` or a configured opener on Linux).
4. Capsule renders a compact feedback cue only if the action fails or is blocked by policy.

This had to be an explicit host action, not automatic browser opening when an agent prints a link. A compromised or noisy agent can emit unlimited OSC 8 hyperlinks; it must not be able to launch host browser windows without a user gesture.

`Cmd`-click is desirable on macOS, but it is a terminal capability question, not an assumption. Some terminals consume `Cmd`-click for their own hyperlink handling and never send it to the child program. Preflight measurements cover what Ghostty and other supported terminals send for `Cmd`-click, `Option`-click, and `Ctrl`-click. Because `Cmd` did not reach the host attach client, jackin❯ provides the capability through a command palette action and modifier sequences the terminal exposes. See the [preflight page](/research/product/operator-experience/host-affordances/host-affordance-bridge-preflight/).

The design covers two visible agent experiences:

| Experience                       | Examples                                                                                  | Link-open behavior                                                                                                                                                                                                                                    |
| -------------------------------- | ----------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Plain CLI transcript             | Codex-style output, shell output, logs, command results                                   | Scan visible row text and scrollback snapshots for URL-shaped tokens; when the modifier is held and the cursor is over a safe URL, draw a subtle hover highlight and open on modified click.                                                          |
| Rich TUI / alternate-screen mode | Claude Code, Amp, Grok, and other TUIs that render panels, mouse targets, and OSC 8 links | Prefer OSC 8 hyperlink regions and jackin-known chrome link regions; fall back to visible URL-token detection on the current rendered grid. Normal mouse input continues to the TUI. Only an explicit link-open gesture on a safe URL is intercepted. |
| jackin-owned chrome and dialogs  | GitHub context dialog, Debug info dialog, bottom PR target, launch-failure popup          | Use first-class actions because jackin❯ already owns the URL and click target.                                                                                                                                                                        |

The hover highlight is an overlay in Capsule's compositor, not bytes injected into the pane. If the focused TUI has mouse reporting enabled, ordinary motion, clicks, and drags still belong to the TUI. The link-open path is active only while jackin❯ can positively identify the operator's modifier or while an explicit command palette action is running. If the terminal cannot report `Cmd` hover state, the correct result is "command palette action works, modifier hover is unavailable in that terminal", not guessing from unmodified mouse movement and stealing rich-TUI input.

The GitHub context dialog was the canonical first chrome target. It already displays branch, PR number, PR title, GitHub URL, and CI status, so the host-affordance bridge added separate actions for "Open PR in browser" and "Open CI in browser"; when CI is failing, the CI action prefers the most specific failing check/run URL available rather than the generic repository Actions page.

This required new data in Capsule. The prior PR context lookup only asked `gh pr list` for `number`, `title`, `url`, and `isDraft`, then asked `gh pr checks` for bucket counts. V1 extended that state with check/run URLs.

### tmux and OSC 52 [#tmux-and-osc-52]

tmux's clipboard integration is mostly about text clipboard synchronization. Its wiki describes OSC 52 and external clipboard tools as "blunt" and inconsistently supported, and calls out the security problem where allowing arbitrary applications to set the outside clipboard means anything that can write to a pane can affect the system clipboard. See the [tmux clipboard wiki](https://github.com/tmux/tmux/wiki/Clipboard).

OSC 52 is useful for jackin❯ text copy out of the container and maybe for text clipboard reads in terminals that explicitly support queries. It is not a correct foundation for image paste. Many terminals do not support reading the clipboard through OSC 52; where reads exist, they typically require user consent; and the payload model is text-oriented rather than "give the remote program the host's screenshot as a file it can read."

### Kitty graphics [#kitty-graphics]

The [Kitty terminal graphics protocol](https://sw.kovidgoyal.net/kitty/graphics-protocol/) lets a terminal application display raster images in a compatible terminal. That is the opposite direction from this feature. It answers "how can a program draw an image in Ghostty/kitty?" not "how can a program inside a Docker container receive an image from the host clipboard?" jackin❯ may eventually need Kitty graphics passthrough for visual previews, but it is not the clipboard image bridge.

### Remote-terminal analogs [#remote-terminal-analogs]

Remote SSH tools solve the same class of problem with an out-of-band upload: read the local clipboard, write an image file on the remote machine, then paste or copy the remote path. [clipssh](https://samuellawrentz.com/blog/clipssh/) is a concise example of that model. jackin❯'s container boundary is not SSH, but the correct abstraction is the same: host image bytes must be deliberately transferred to the environment where the agent can read them, and the thing pasted into the agent should be the destination path.

Ghostty has an open SSH-image-paste discussion that describes the same desired user flow: detect a local clipboard image, upload it to the remote session, and insert the remote path so tools like Claude Code can read it. The proposed path there is SCP over an existing SSH control socket. jackin❯ does not need SSH for local Docker, but the product invariant is identical: paste/upload creates a destination-local file path, not a host-only path. See [ghostty-org/ghostty discussion #10517](https://github.com/ghostty-org/ghostty/discussions/10517).

### Link handling references [#link-handling-references]

WezTerm has a first-class `OpenLinkAtMouseCursor` action that can be bound to `Ctrl`-click, plus `hyperlink_rules` that turn visible text patterns into clickable links. This supports jackin❯'s planned split between explicit OSC 8 regions and implicit URL detection. See [WezTerm `OpenLinkAtMouseCursor`](https://wezterm.org/config/lua/keyassignment/OpenLinkAtMouseCursor.html) and [WezTerm `hyperlink_rules`](https://wezterm.org/config/lua/config/hyperlink_rules.html).

xterm.js documents the common safety rule for link activation: terminals often require a modifier, usually `Ctrl` or `Cmd` on macOS, to avoid accidental window opening. It also calls out hover URL display as a safety feature, especially for OSC 8 links where the displayed text can differ from the actual target. jackin❯ follows that model: show enough target information on hover/debug surfaces, but do not log full query strings by default. See [xterm.js Link Handling](https://xtermjs.org/docs/guides/link-handling/).

Rio's hyperlink hints are useful because they make modifier keys configurable and default to `Command` on macOS and `Alt` elsewhere. That argues against hard-coding `Cmd` as the only jackin❯ link-open gesture. jackin❯ prefers platform defaults but keeps a configurable modifier/open command path. See [Rio Hyperlinks](https://rioterm.com/docs/features/hyperlinks).

xterm mouse protocols can report Shift, Meta, and Control modifiers with mouse events, but the xterm documentation notes that Shift and Control are often unavailable because terminals reserve them for selection or menus. Actual terminal measurements, especially Ghostty on macOS, therefore govern whether modifier-hover reaches the attach client. See [xterm control sequences — mouse tracking](https://www.xfree86.org/current/ctlseqs.html).

### Shell integration and file transfer references [#shell-integration-and-file-transfer-references]

iTerm2's shell integration can download a file referenced in remote output and can upload a dragged Finder file to the remote host with SCP when Option is held. This is not a direct implementation target for jackin❯, but it proves the UX class is established: terminal chrome can turn a visible remote/container path or host drag/drop into an explicit transfer action. jackin❯'s equivalent is typed Capsule staging under `/jackin/run/clipboard/` or a future `/jackin/run/uploads/`, not ad hoc shell text injection. See [iTerm2 shell integration](https://iterm2.com/3.0/documentation-shell-integration.html).

WezTerm shell integration uses OSC 7, OSC 133, and OSC 1337 to mark working directories, prompt/input/output zones, and pane state. jackin❯ already blocks OSC 7 passthrough to avoid host cwd pollution, but the idea of prompt/output zones is still useful: when shell sessions emit OSC 133, Capsule can use those zones to make transcript-mode link hover, command-output selection, and future "open artifact from last command" more precise. See [WezTerm shell integration](https://wezterm.org/shell-integration.html).

Kitty's clipboard kitten can read and write arbitrary clipboard MIME types, including raster images, and can work over SSH with permission prompts. That is a useful optional backend/protocol reference, but it should not become jackin❯'s primary path because it depends on Kitty-compatible terminal support and terminal-level clipboard permissions. The primary path remains a host attach client with platform readers. See [Kitty clipboard kitten](https://sw.kovidgoyal.net/kitty/kittens/clipboard/).

Kitty's remote-file kitten supports opening, editing, and downloading files from a remote SSH session. The terminal can use hyperlinks from commands such as `ls --hyperlink=auto`, then a modified click chooses an action. This is the closest UX precedent for jackin❯'s container-file export path: visible file path plus explicit operator action plus host-side destination. See [Kitty remote files](https://sw.kovidgoyal.net/kitty/kittens/remote_file/).

Kitty's transfer kitten is a more protocol-shaped precedent: it can copy files in either direction over the TTY device, including remote-to-local download and local-to-remote upload. jackin❯ does not depend on Kitty's implementation, but it validates a typed bidirectional file-transfer capability as a terminal-adjacent primitive. See the [transfer kitten documentation](https://sw.kovidgoyal.net/kitty/kittens/transfer/).

iTerm2's utilities include `it2dl`, which downloads a remote file into the local Downloads folder. This reinforces the correct host destination model for jackin❯: a predictable operator-owned download location with visible feedback, not silent writes to arbitrary host paths. See [iTerm2 utilities](https://iterm2.com/documentation-utilities.html).

## Implications for jackin❯ [#implications-for-jackin]

A host-owned typed bridge is required because in-container processes cannot reliably observe host-only clipboard or desktop actions.

## Limitations and unknowns [#limitations-and-unknowns]

Terminal and clipboard behavior varies by platform and host application. The architecture must preserve explicit fallback behavior when a capability is absent.

## Sources [#sources]

Primary repositories, vendor documentation, and protocol references are linked beside the findings they support.

## Related work [#related-work]

* [Host Affordance Bridge overview](/research/product/operator-experience/host-affordances/host-affordance-bridge-design/)
* [Architecture and security](/research/product/operator-experience/host-affordances/host-affordance-bridge-design/03-architecture-and-security/)