# 01 — Goals and non-goals (https://jackin.tailrocks.com/reference/research/jackin-context-engine/01-goals-and-non-goals/) # 01 — Goals and non-goals [#01--goals-and-non-goals] ## Goals [#goals] 1. **Out-of-box token optimization for every capsule.** A freshly launched jackin❯ container is born with the negative-cost set active (tool-search deferral intact, Edit-over-Write guards, observation filtering, subagent model/effort pins) — zero operator discipline required. This is the dossier's K16 "jackin-baked optimization pack" (ESTIMATE 20–35% integrated) made concrete. 2. **Cover both interception points.** Write-time compression must reach shell output (RTK's slot) *and* native reads (headroom/lean-ctx's slot), because the measured Bash share of observation tokens ranged 16.5% (docs workload) to dominant (test-heavy workloads). One engine, both boundaries, exactly one shell-rewrite path per capsule. 3. **Reversible by default.** Every lossy transform leaves a content-addressed verbatim copy retrievable by handle on *all* exit codes — fixing RTK's success-path gap and adopting headroom-CCR/lean-ctx-archive semantics. Recovery hints ride on every compressed payload (lean-ctx #625 doctrine: agents fight irreversible compression; undiscoverable escape hatches are the root cause of "too compressed" failures). 4. **Deterministic core, no ML in the hot path.** Typed deterministic transforms (filters, crushers, outlines, extractive prose scoring) only. This is now the *converged* industry position: headroom itself is demoting its ModernBERT stage after timeout cascades, and CompressionAttack (arXiv 2510.22963, ≤80% ASR) turned hot-path ML compressors into a security finding, not just a latency cost. 5. **Cache-safety by construction.** All input-side work happens at content-ingestion time (append-only), never by rewriting history or the cached prefix. The engine ships the prefix-stability regression tests headroom pioneered and surfaces `cache_read` continuity as a monitored invariant. 6. **Retrieval before reading.** Bounded, task-shaped retrieval (outline → section/symbol → span) replaces whole-file reads where it measurably wins: −91% outline / −98% symbol-search (T1 local, [tools hub 19 — Code-intelligence tools: codedb, fff, CodeGraff, and alternatives](/reference/research/token-optimization-tools/19-code-intelligence-tools/)), 12–53× markdown section retrieval (measured on mq, [tools hub 25 — sem and mq: entity-level retrieval and structural markdown queries](/reference/research/token-optimization-tools/25-sem-and-mq-structural-and-markdown-retrieval/)), with size gates so small files are returned whole. 7. **Fleet-aware economics.** Spawn staggering, prefix pre-warm, `excludeDynamicSections`, batch-lane routing, effort tiering, and advisor escalation are orchestrator-enforced. Dual cost model: dollars *and* tasks-per-cap, selected by the capsule's auth mode. 8. **Prove it or lose it.** Every layer reports through one bounce-netted, Claude-tokenizer-honest ledger; features that fail the acceptance bar (below) get disabled per role, automatically. The engine is its own validation harness. 9. **Agent-agnostic by layer.** \~80% of the stack ports as discipline across Claude Code/Codex/other runtimes ([techniques 23 — Cross-agent / cross-provider portability matrix](/reference/research/token-optimization-techniques/23-cross-agent-portability/)); the engine abstracts effort tier, routing, rules-file compilation, output caps, and cache-stability per `Agent` adapter, and degrades gracefully where a runtime lacks a hook surface (Codex: instructions-only). ## Non-goals (the graveyard, enforced) [#non-goals-the-graveyard-enforced] Each entry below was killed by the research with evidence; the engine treats these as tripwires in review, not options: | Non-goal | Killed by | | ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Whole-prompt / mid-stream proxy compression (LLMLingua-class) in front of Claude Code | Break-even \~5.5× mixed / \~10× cacheable prefix; 358-run Sonnet 4.5 RCT: aggressive compression **+1.8% cost**; double-stabilization conflicts ([techniques 13 — Infrastructure-level (self-hosted / gateway tier)](/reference/research/token-optimization-techniques/13-infrastructure-level/); [tools hub 21 — Headroom and the context-compression layer (vs the caveman ecosystem)](/reference/research/token-optimization-tools/21-headroom-and-context-compression/), 22) | | Cache-aligned whole-prompt modality re-encoding (pxpipe-class "optical context compression") as a **default** | Different verdict class: pxpipe's prefix-stability engineering (marker relocation, frozen slab, quantized history boundary) defeats the break-even/cache-shred argument above, so this is **not a tripwire** — it stays a non-goal as a default because it is structurally lossy with a silent-confabulation failure shape (12-char hex recall 13/15 Fable 5 vs 0/15 Opus 4.8), its economics rest on undocumented provider vision behavior (`px/750`, resample caps) that already flipped across model generations, and the whole-prompt-rewrite proxy form conflicts with the engine's no-proxy invariants. Explicitly **eligible as a harness-gated experiment** on gist-tolerant workloads behind the full evidence bar ([tools hub 17](/reference/research/token-optimization-tools/17-pxpipe-design/)) | | ML compression model in the hot path | CompressionAttack ≤80% ASR; headroom P99 4.17 s + executor-saturation outages; deterministic TextCrusher-class reaches \~94% retention (13-reuse ch.) | | API gateway that re-serializes requests | Custom `ANTHROPIC_BASE_URL` disables tool-search deferral; one nondeterministic serializer = broken cache = \~3.8× session cost ([techniques 06 — Context architecture — what enters the window at all](/reference/research/token-optimization-techniques/06-context-architecture/), 13) | | Semantic response caching for coding traffic | Zero published coding hit rates; repetition lives in the prefix already priced 0.1×; 3–7% confidently-wrong on code ([techniques 08 — Retrieval, memory, and state offloading](/reference/research/token-optimization-techniques/08-retrieval-memory-and-state/), 13) | | base64/gzip/glyph-DSL/wenyan "compression" | base64 = 4.33× MORE tokens; gzip+base64 = 2.68× MORE; glyphs 3.9–4.9 tok each with 26% instruction-fidelity; wenyan char/token illusion ([techniques 04 — Style and Language Compression — Beyond Caveman](/reference/research/token-optimization-techniques/04-style-and-language-compression/), 05, 13) | | Cache-keepalive pinger on the interactive main loop | Subscription main loop already 1h-TTL; 1h break-even \~19 h idle ([techniques 14 — Frontier — unrealistic but maybe real](/reference/research/token-optimization-techniques/14-frontier-ideas/) K13) — only SDK/API fleet lanes with sub-40-min gaps qualify | | Vector DB / embeddings for **code navigation** as default | Flagship agents dropped code embeddings; acceptance bar ≥20% tokens-per-solved-task over `rust-analyzer + ast-grep + structural + fff` never met publicly ([tools hub 19 — Code-intelligence tools: codedb, fff, CodeGraff, and alternatives](/reference/research/token-optimization-tools/19-code-intelligence-tools/), 20) | | A second memory store, a second shell-rewrite path, a second output policy | Doubling rules from the combining page: pure overhead, double-compression risk ([tools hub 06 — Combining them: is there one product?](/reference/research/token-optimization-tools/06-combining/)) | | Renaming existing identifiers for token shape | Cache bust + blame destruction; design-time-only lever ([techniques 05 — Tokenizer arbitrage](/reference/research/token-optimization-techniques/05-tokenizer-arbitrage/)) | | KV-cache eviction/quantization, soft prompts, speculative decoding | Self-host-only; no hosted API channel ([techniques 24 — Fresh literature & market delta (clean-room re-sweep)](/reference/research/token-optimization-techniques/24-fresh-literature-and-market-delta/), 14 K1/K2) | | Trusting per-payload percentages as savings | Per-payload ≠ whole-bill: the correction that killed every vendor headline ([tools hub 07 — Evidence, benchmarks, and the claim graveyard](/reference/research/token-optimization-tools/07-evidence-and-claims/)) | | `max_tokens` as a spend governor | Truncates `tool_use`, bills the attempt, forces retry at higher cap ([techniques 09 — Output discipline and structured generation](/reference/research/token-optimization-techniques/09-output-discipline/), 25) | ## Evidence bars (every feature, no exceptions) [#evidence-bars-every-feature-no-exceptions] * **Adoption bar** (from [techniques 16 — Validation Harness: the No-Quality-Loss Proof Protocol](/reference/research/token-optimization-techniques/16-validation-harness/) + [tools hub 07 — Evidence, benchmarks, and the claim graveyard](/reference/research/token-optimization-tools/07-evidence-and-claims/)): task/test success ≥ baseline (non-inferiority, 5pp margin, n=12 screening / n=30 confirmation); `cache_read` continuity preserved; command re-run / bounce rate not worse; **total tokens per solved task ≤ baseline by ≥20% net of the feature's own overhead** (schema rent, injected instructions, retrieval round-trips, handles). * **Net-of-injection accounting** (the tokbench lesson from lean-ctx's first independent benchmark): any standing prefix the engine injects is a first-class cost in its own ledger, CI-gated like lean-ctx's `doctor overhead --gate` (its headline self-footprint metric is \~2.1K tokens/session — ours must be lower and gated from day one). * **Canary gate**: the six compression-failure canaries (negation, ordering, numeric precision, don't-do-X, multi-step, caveat retention — [techniques 16 — Validation Harness: the No-Quality-Loss Proof Protocol](/reference/research/token-optimization-techniques/16-validation-harness/) §3) run in both arms of every A/B; one Arm-B-only failure vetoes the feature regardless of savings. * **Quality-risk features run behind holdouts**: aggressive registers and effort downgrades ship with a headroom-style holdout control group and an online sampled canary ([techniques 25 — Meta layer: the cost of optimizing, budget governance, and online quality guarding](/reference/research/token-optimization-techniques/25-meta-cost-governance-and-online-quality/): alarm, don't block), so the "does register compression degrade agentic success" open question finally gets an answer on live traffic.