Test infrastructure: shared test-support crate, behavioral specs, and parser fuzzing

Summary

The maintainable verification strategy combines shared test helpers, executable behavioral specifications, property tests, parser fuzzing, deterministic clocks, and CI gates. The jackin-test-support crate and behavioral specs hold the durable implementation contracts.

Research question

Three testing gaps increase the risk of silent regressions and make source-file ownership changes harder to verify.

Test file duplication mirrors source duplication. crates/jackin/tests/manager_flow.rs and crates/jackin/tests/dind_e2e.rs are each 1,500+ lines. Mock runners remain independently defined in several places, while configuration and resolved-workspace fixtures recur across jackin-console, jackin, jackin-runtime, and jackin-config. The shared jackin-test-support crate establishes the ownership boundary; remaining local duplicates should move there only when their semantics are genuinely shared.

Behavioral specs are incomplete. Specs exist for runtime/launch.rs and the console op_picker, but not for jackin-capsule's daemon (crates/jackin-capsule/src/daemon.rs — the in-container PID 1 control plane) or for the operator console's TUI state machine. Behavioral specs should be a general practice for every major component, not a one-off exercise per refactor.

No property-based or fuzz testing for parser surfaces. crates/jackin-manifest/src/validate.rs, crates/jackin-env/src/env_resolver.rs, and the migration chains (crates/jackin-config/src/migrations.rs, crates/jackin-manifest/src/migrations.rs) all parse operator- or role-author-supplied input, and are tested only by example-based unit tests. An input the developer didn't think of isn't covered. cargo-fuzz already exists for jackin-term (the damage_grid_process target, scheduled nightly in .github/workflows/hygiene.yml); no equivalent exists for any parser crate, and proptest is not part of the dependency graph at all.

Findings and current evidence

Shared fixtures and focused test files

Rust projects that maintain large integration test suites (tokio, bevy, cargo) commonly extract a dedicated test-support crate rather than let helpers proliferate per test file. Apply the same pattern here:

• jackin-test-support is the shared home for deterministic fixtures, fake runners, workspace builders, and role seeding that have identical semantics across crates. Crate-specific helpers stay with their consumers.
• Large integration-test files should be divided by scenario ownership. The boundary is useful when each resulting file can name its fixture needs without recreating shared mocks.
• insta already supports several crates. Shared snapshot helpers belong in jackin-test-support only for cross-crate determinism concerns such as fixed dimensions, clocks, themes, and redaction. Visual snapshot testing owns artifact format and rendering assertions.

2. Behavioral specs for major components

Behavioral specs already exist for runtime/launch.rs (published at runtime/launch.rs Behavioral Spec) and for the console op_picker (published at op_picker Behavioral Spec), plus a differently-scoped spec for alternate agent credential/config folder sync (Auth Source-Folder Sync). The gap is the capsule daemon and the operator console:

1. Capsule daemon (crates/jackin-capsule/src/daemon.rs) — PID 1 contract, single-attach-client invariant, control channel dispatch, session lifecycle, PTY mutex poison recovery, attach framing, OSC passthrough, mode-state restore, sessions persistence and reattach.
2. Operator console — the TUI state machine: workspace selection to role selection to agent selection to instance lifecycle to session management, keybinding dispatch, dialog stack.

Each spec should list invariants, give the failure mode for each violation, and link to the source line where the invariant is enforced — the same INV-format shape used by the existing published specs under /reference/developer-reference/specs/.

3. Property and fuzz tests for parser surfaces

Two testing layers, neither of which exists today for the parser crates:

proptest for invariant testing:

• env_resolver: ${env.VAR} interpolation is associative, idempotent on no-op substitutions, escapes correctly for every Unicode input.
• manifest::validate: a manifest that parses successfully always round-trips through serialize to parse unchanged.
• Migration chain: any input that parses at version N migrates successfully to N+1 and the resulting output parses at N+1.

cargo-fuzz targets, run alongside the existing jackin-term nightly job:

• manifest::validate (TOML input, expect no panic).
• env_resolver::resolve (any Unicode env string).
• Each migration step (any input that parses at version N).

Non-goals

• Do not couple unrelated readability refactors to test-support consolidation; shared helpers are required only where a split would otherwise duplicate them.
• Do not add mutation testing yet. Re-evaluate once cargo-llvm-cov (from Rust CI tooling & dependency hygiene) shows where coverage is weakest.
• Do not fuzz the TUI rendering layer or the Docker client — those surfaces change too fast for fuzzing to be cost-effective.

Decision implications

• Consolidate helpers by semantic ownership, not by a migration sequence; each move should delete a duplicate and preserve consumer behavior.
• Contributor-facing behavioral specs remain the durable home for component invariants. Roadmap items own any undelivered Capsule daemon or console coverage.
• Parser property tests and fuzz targets should join the existing nightly fuzz authority when their invariants and time budgets are explicit.

Limitations and open questions

1. Which remaining helpers have truly identical semantics across consumers?
2. What nightly CI time budget should parser fuzz targets share with the existing jackin-term target?

Related work

• jackin-test-support crate — shared deterministic fixtures and fakes produced by this work
• Visual snapshot testing (CLI & TUI) — the SVG golden-artifact format that would build on the shared test-support crate
• Rust CI tooling & dependency hygiene — cargo insta for snapshot tests, cargo-llvm-cov for coverage
• runtime/launch Behavioral Spec — the established format for other component contracts