# 06 — Routing and fleet economics (L4) (https://jackin.tailrocks.com/research/context/engine/06-routing-and-fleet-economics/)



## Summary [#summary]

Routing should occur at task boundaries, with explicit model and effort pins, because mid-session switches invalidate caches and inherited defaults are not a cost policy.
The only levers that touch the thinking bucket (20% of dollars) live here, and the biggest multipliers in the dossier's composed stacks are routing multipliers. jackin❯ is the launcher, supervisor, and orchestrator — every technique in this layer is a config-generation or scheduling decision the runtime already has the authority to make.

## Question and scope [#question-and-scope]

When do model routing, delegation, and fleet-wide prefix reuse reduce accepted-task cost rather than shift it?

## Method [#method]

The analysis composes per-model token counts, cache behavior, delegation overhead, and acceptance gates; it rejects any route whose retries, cold context, or quality loss erase the apparent saving.

## Findings [#findings]

### Effort policy [#effort-policy]

* Per-role default effort written into the launch env/flags (`build_agent_command`), per-task-class overrides at session boundaries only (INV-9: mid-session `/effort` busts the cache; \~9-turn break-even).
* Anchor evidence: Opus 4.5 at `medium` matched Sonnet 4.5's best SWE-bench at **−76% output tokens** (T1 vendor); `max → high` is negative-cost per vendor docs ("prone to overthinking"). Transfer to newer models is **unvalidated — the engine's own harness must produce the per-level effort curve**, which nobody (including the vendor) has published; `usage.output_tokens_details.thinking_tokens` now makes it measurable.
* Subagent effort pins: exploration/read-only agents get explicit effort and model frontmatter. Inheritance is not a cost policy; generated pins are the stable way to keep exploration in the intended lane. Reverify built-in defaults before relying on them ([provider-sensitive claim verification](/research/context/techniques/28-drift-delta/)).

### Model routing [#model-routing]

* **Advisor-pattern first** (the only routing with published negative-cost numbers: Sonnet-main + Opus-advisor = +2.7pp SWE-bench Multilingual AND −11.9% cost/task, T1): role-level `advisorModel` wiring where the runtime supports it.
* **Cheap-fleet pinning**: read-only fan-out to Haiku-tier (5-worker fan-out $2.75 → \~$0.20–0.28 class, −90–93% ESTIMATE); code-writing teammates stay Sonnet-tier (Haiku SWE-bench gap \~6pp — the quality-delta map). The advisor and fan-out evidence is Sonnet-4.5-era; per the spec's own per-model-validation rule, the harness re-produces these curves on the current lineup (Sonnet 5) before pinning fleet defaults.
* **Tokenizer-aware routing** ([techniques 05 — Tokenizer arbitrage](/research/context/techniques/05-tokenizer-arbitrage/)/[techniques 10 — Model routing and tiered delegation](/research/context/techniques/10-model-routing-and-delegation/)): tokenizer families can produce materially different counts for the same corpus. The engine counts routed corpora against the exact target model before committing a route and never ports counts across families. Reverify model-family membership, price, and availability together ([provider-sensitive claim verification](/research/context/techniques/28-drift-delta/)).
* Route changes happen at task/session boundaries via new sessions or subagents (cache is model-scoped); the orchestrator, not the operator, owns the boundary.

### Spawn choreography (fleet cache) [#spawn-choreography-fleet-cache]

* **Stagger or pre-warm**: a cache entry exists only after the first response begins streaming — N simultaneous same-prefix spawns pay N full writes; await-first-then-fan-out cuts the wave's prefix cost \~3.8× at n=5 / \~8.6× at n=25. Pre-warm pattern for cold fleets: one `max_tokens:0`-class request, then fan out (25×30k prefix: \~$9.4 → \~$0.94 ESTIMATE).
* **`excludeDynamicSections` enforced fleet-wide** with an SDK version floor: six per-container fields (\~111 tokens: cwd, git flag, platform, …) placed ahead of the prefix otherwise un-share the entire \~28–34k system prefix across every container in the fleet — the single biggest hosted-fleet lever ([techniques 22 — Fleet, team & multi-tenant cache economics (hosted)](/research/context/techniques/22-fleet-and-multitenant-cache/)).
* **Byte-identical prefix discipline**: same workspace, same model/effort/tool set across a fleet lane; any tool-definition change busts the whole fleet's prefix. Subagent-caching behavior is version-dependent (issue #29966 vs measured cavecrew writes) — the ledger audits `cache_read_input_tokens > 0` per subagent class rather than assuming.
* Depth/fan-out caps: subagents nest 5 deep and agent teams run ≈7× tokens; the orchestrator enforces role-level caps and tears down idle teammates.

### Lane scheduler [#lane-scheduler]

Three lanes per capsule, chosen by human-attachment (which jackin❯ knows from the attach state):

| Lane                          | Pricing                                                                                                              | Routed there                                                                     |
| ----------------------------- | -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| Interactive (human attached)  | standard; fast mode only by the `v·t·s > Δ$` rule at session start (never mid-session — re-bills context at premium) | operator sessions                                                                |
| Autonomous (no human blocked) | standard                                                                                                             | agent-driven work                                                                |
| Offline                       | **Batch API: 50% off everything, stacks with caching (0.05× batched cache reads)**                                   | nightly sweeps, eval/harness runs, doc regen, subagent fleets that tolerate ≤24h |

### Quota and budget governance [#quota-and-budget-governance]

* **Dual cost model per capsule auth mode** (headroom's auth-mode policy, driven by jackin❯ credential provisioning): API-key capsules optimize dollars; subscription capsules optimize **tasks per limit window**. The engine detects the active billing regime at launch instead of hard-coding plan, model, or headless-usage policy. Exact cap weights remain unpublished and therefore bounded ([provider-sensitive claim verification](/research/context/techniques/28-drift-delta/)).
* The cap denominator is unpublished: the engine ships the **V2 prober** — regress token-class consumption against `anthropic-ratelimit-unified-*` header utilization (captured pass-through, `cache_control`-preserving) to fit per-account weights, closing the dossier's biggest INCOMPLETE.
* **Governors that actually govern** ([techniques 25 — Meta layer: the cost of optimizing, budget governance, and online quality guarding](/research/context/techniques/25-meta-cost-governance-and-online-quality/)): workspace rate limits and monthly credit caps (real stops), gateway-tier budgets where a gateway exists; the vendor "spend limit" is alert-only and treated as such. Degrade-don't-die: budget triggers downgrade model/effort on routine lanes, never on critical lanes, and never truncate via `max_tokens`.
* Engine self-cost is metered in the same ledger (meta-governance: the meter must cost less than what it measures; `count_tokens` runs against its own separate RPM pool, batched, never inline per-step).

### Additional routing constraints [#additional-routing-constraints]

* **Mutation-gain arithmetic**: any decision to rewrite a deep (already-cached) block uses headroom's *corrected* `net_mutation_gain`: the warm-case rewrite penalty applies over the saved tokens *plus* the downstream tail (`S+ΔT`), not `S` alone. Property tests ship with the formula.
* **Learning never destabilizes prefixes**: any adaptive policy (mode selection, register level, effort tier) selects by **deterministic argmax over posterior means by default**; stochastic exploration (Thompson/ε-greedy) is opt-in per role — lean-ctx's #498 doctrine, adopted as an invariant: a learner that randomizes prompt-visible behavior is a cache-buster.
* **Subscription write-quarantine**: capsules on subscription credentials consume learned recommendations read-only and never write into shared learning pools (headroom's `toin_read_only` pattern) — UX-bound traffic must not mutate policy trained for PAYG economics.
* **Bandit spec baseline** (if/when adaptive mode selection ships): Beta(2,1) arms over (entropy, jaccard, budget) triples; success ⇔ savings-ratio > 0.3 AND task completed; ×0.98 session-end decay floored at 1 — lean-ctx's \~300-line spec, adopted as the reference design rather than invented fresh.

## Implications for jackin❯ [#implications-for-jackin]

These findings define the routing and fleet economics (l4) contract for jackin❯; any implementation must preserve the acceptance gates and boundaries above.

## Limitations and unknowns [#limitations-and-unknowns]

The design remains bounded by the evidence and acceptance gates above; volatile dependencies and provider behavior require revalidation before implementation.

## Sources [#sources]

Evidence and repository references are cited inline beside the claims they support.

## Related work [#related-work]

* [Context engine dossier](/research/context/engine/)
* [Token-optimization techniques](/research/context/techniques/)
* [Token-optimization tools](/research/context/tools/)