# Test infrastructure: shared test-support crate, behavioral specs, and parser fuzzing (https://jackin.tailrocks.com/research/engineering/testing/test-infrastructure-consolidation/)



## Summary [#summary]

The maintainable verification strategy combines shared test helpers, executable behavioral specifications, property tests, parser fuzzing, deterministic clocks, and CI gates. The [`jackin-test-support` crate](/reference/crates/jackin-test-support/) and [behavioral specs](/reference/developer-reference/specs/) hold the durable implementation contracts.

## Research question [#research-question]

Three testing gaps increase the risk of silent regressions and make source-file ownership changes harder to verify.

**Test file duplication mirrors source duplication.** <RepoFile path="crates/jackin/tests/manager_flow.rs">crates/jackin/tests/manager\_flow\.rs</RepoFile> and <RepoFile path="crates/jackin/tests/dind_e2e.rs">crates/jackin/tests/dind\_e2e.rs</RepoFile> are each 1,500+ lines. Mock runners remain independently defined in several places, while configuration and resolved-workspace fixtures recur across `jackin-console`, `jackin`, `jackin-runtime`, and `jackin-config`. The shared <RepoFile path="crates/jackin-test-support/Cargo.toml">`jackin-test-support` crate</RepoFile> establishes the ownership boundary; remaining local duplicates should move there only when their semantics are genuinely shared.

**Behavioral specs are incomplete.** Specs exist for `runtime/launch.rs` and the console `op_picker`, but not for `jackin-capsule`'s daemon (<RepoFile path="crates/jackin-capsule/src/daemon.rs">crates/jackin-capsule/src/daemon.rs</RepoFile> — the in-container PID 1 control plane) or for the operator console's TUI state machine. Behavioral specs should be a general practice for every major component, not a one-off exercise per refactor.

**No property-based or fuzz testing for parser surfaces.** <RepoFile path="crates/jackin-manifest/src/validate.rs">crates/jackin-manifest/src/validate.rs</RepoFile>, <RepoFile path="crates/jackin-env/src/env_resolver.rs">crates/jackin-env/src/env\_resolver.rs</RepoFile>, and the migration chains (<RepoFile path="crates/jackin-config/src/migrations.rs">crates/jackin-config/src/migrations.rs</RepoFile>, <RepoFile path="crates/jackin-manifest/src/migrations.rs">crates/jackin-manifest/src/migrations.rs</RepoFile>) all parse operator- or role-author-supplied input, and are tested only by example-based unit tests. An input the developer didn't think of isn't covered. `cargo-fuzz` already exists for `jackin-term` (the `damage_grid_process` target, scheduled nightly in <RepoFile path=".github/workflows/hygiene.yml">.github/workflows/hygiene.yml</RepoFile>); no equivalent exists for any parser crate, and `proptest` is not part of the dependency graph at all.

## Findings and current evidence [#findings-and-current-evidence]

### Shared fixtures and focused test files [#shared-fixtures-and-focused-test-files]

Rust projects that maintain large integration test suites (tokio, bevy, cargo) commonly extract a dedicated test-support crate rather than let helpers proliferate per test file. Apply the same pattern here:

* <RepoFile path="crates/jackin-test-support/Cargo.toml">`jackin-test-support`</RepoFile> is the shared home for deterministic fixtures, fake runners, workspace builders, and role seeding that have identical semantics across crates. Crate-specific helpers stay with their consumers.
* Large integration-test files should be divided by scenario ownership. The boundary is useful when each resulting file can name its fixture needs without recreating shared mocks.
* `insta` already supports several crates. Shared snapshot helpers belong in `jackin-test-support` only for cross-crate determinism concerns such as fixed dimensions, clocks, themes, and redaction. [Visual snapshot testing](/research/engineering/testing/visual-snapshots/) owns artifact format and rendering assertions.

### 2. Behavioral specs for major components [#2-behavioral-specs-for-major-components]

Behavioral specs already exist for `runtime/launch.rs` (published at [runtime/launch.rs Behavioral Spec](/reference/developer-reference/specs/runtime-launch/)) and for the console `op_picker` (published at [op\_picker Behavioral Spec](/reference/developer-reference/specs/op-picker/)), plus a differently-scoped spec for alternate agent credential/config folder sync ([Auth Source-Folder Sync](/reference/developer-reference/specs/auth-source-folder-sync/)). The gap is the capsule daemon and the operator console:

1. **Capsule daemon** (<RepoFile path="crates/jackin-capsule/src/daemon.rs">crates/jackin-capsule/src/daemon.rs</RepoFile>) — PID 1 contract, single-attach-client invariant, control channel dispatch, session lifecycle, PTY mutex poison recovery, attach framing, OSC passthrough, mode-state restore, sessions persistence and reattach.
2. **Operator console** — the TUI state machine: workspace selection to role selection to agent selection to instance lifecycle to session management, keybinding dispatch, dialog stack.

Each spec should list invariants, give the failure mode for each violation, and link to the source line where the invariant is enforced — the same INV-format shape used by the existing published specs under `/reference/developer-reference/specs/`.

### 3. Property and fuzz tests for parser surfaces [#3-property-and-fuzz-tests-for-parser-surfaces]

Two testing layers, neither of which exists today for the parser crates:

**`proptest` for invariant testing:**

* `env_resolver`: `${env.VAR}` interpolation is associative, idempotent on no-op substitutions, escapes correctly for every Unicode input.
* `manifest::validate`: a manifest that parses successfully always round-trips through serialize to parse unchanged.
* Migration chain: any input that parses at version N migrates successfully to N+1 and the resulting output parses at N+1.

**`cargo-fuzz` targets**, run alongside the existing `jackin-term` nightly job:

* `manifest::validate` (TOML input, expect no panic).
* `env_resolver::resolve` (any Unicode env string).
* Each migration step (any input that parses at version N).

## Non-goals [#non-goals]

* Do not couple unrelated readability refactors to test-support consolidation; shared helpers are required only where a split would otherwise duplicate them.
* Do not add mutation testing yet. Re-evaluate once `cargo-llvm-cov` (from [Rust CI tooling & dependency hygiene](/research/engineering/ci/rust-tooling/)) shows where coverage is weakest.
* Do not fuzz the TUI rendering layer or the Docker client — those surfaces change too fast for fuzzing to be cost-effective.

## Decision implications [#decision-implications]

* Consolidate helpers by semantic ownership, not by a migration sequence; each move should delete a duplicate and preserve consumer behavior.
* Contributor-facing behavioral specs remain the durable home for component invariants. Roadmap items own any undelivered Capsule daemon or console coverage.
* Parser property tests and fuzz targets should join the existing nightly fuzz authority when their invariants and time budgets are explicit.

## Limitations and open questions [#limitations-and-open-questions]

1. Which remaining helpers have truly identical semantics across consumers?
2. What nightly CI time budget should parser fuzz targets share with the existing `jackin-term` target?

## Related work [#related-work]

* [`jackin-test-support` crate](/reference/crates/jackin-test-support/) — shared deterministic fixtures and fakes produced by this work
* [Visual snapshot testing (CLI & TUI)](/research/engineering/testing/visual-snapshots/) — the SVG golden-artifact format that would build on the shared test-support crate
* [Rust CI tooling & dependency hygiene](/research/engineering/ci/rust-tooling/) — `cargo insta` for snapshot tests, `cargo-llvm-cov` for coverage
* [runtime/launch Behavioral Spec](/reference/developer-reference/specs/runtime-launch/) — the established format for other component contracts