06 — Routing and fleet economics (L4)

Summary

Routing should occur at task boundaries, with explicit model and effort pins, because mid-session switches invalidate caches and inherited defaults are not a cost policy. The only levers that touch the thinking bucket (20% of dollars) live here, and the biggest multipliers in the dossier's composed stacks are routing multipliers. jackin❯ is the launcher, supervisor, and orchestrator — every technique in this layer is a config-generation or scheduling decision the runtime already has the authority to make.

Question and scope

When do model routing, delegation, and fleet-wide prefix reuse reduce accepted-task cost rather than shift it?

Method

The analysis composes per-model token counts, cache behavior, delegation overhead, and acceptance gates; it rejects any route whose retries, cold context, or quality loss erase the apparent saving.

Findings

Effort policy

• Per-role default effort written into the launch env/flags (build_agent_command), per-task-class overrides at session boundaries only (INV-9: mid-session /effort busts the cache; ~9-turn break-even).
• Anchor evidence: Opus 4.5 at medium matched Sonnet 4.5's best SWE-bench at −76% output tokens (T1 vendor); max → high is negative-cost per vendor docs ("prone to overthinking"). Transfer to newer models is unvalidated — the engine's own harness must produce the per-level effort curve, which nobody (including the vendor) has published; usage.output_tokens_details.thinking_tokens now makes it measurable.
• Subagent effort pins: exploration/read-only agents get explicit effort and model frontmatter. Inheritance is not a cost policy; generated pins are the stable way to keep exploration in the intended lane. Reverify built-in defaults before relying on them (provider-sensitive claim verification).

Model routing

• Advisor-pattern first (the only routing with published negative-cost numbers: Sonnet-main + Opus-advisor = +2.7pp SWE-bench Multilingual AND −11.9% cost/task, T1): role-level advisorModel wiring where the runtime supports it.
• Cheap-fleet pinning: read-only fan-out to Haiku-tier (5-worker fan-out $2.75 → ~$0.20–0.28 class, −90–93% ESTIMATE); code-writing teammates stay Sonnet-tier (Haiku SWE-bench gap ~6pp — the quality-delta map). The advisor and fan-out evidence is Sonnet-4.5-era; per the spec's own per-model-validation rule, the harness re-produces these curves on the current lineup (Sonnet 5) before pinning fleet defaults.
• Tokenizer-aware routing (techniques 05 — Tokenizer arbitrage/techniques 10 — Model routing and tiered delegation): tokenizer families can produce materially different counts for the same corpus. The engine counts routed corpora against the exact target model before committing a route and never ports counts across families. Reverify model-family membership, price, and availability together (provider-sensitive claim verification).
• Route changes happen at task/session boundaries via new sessions or subagents (cache is model-scoped); the orchestrator, not the operator, owns the boundary.

Spawn choreography (fleet cache)

• Stagger or pre-warm: a cache entry exists only after the first response begins streaming — N simultaneous same-prefix spawns pay N full writes; await-first-then-fan-out cuts the wave's prefix cost ~3.8× at n=5 / ~8.6× at n=25. Pre-warm pattern for cold fleets: one max_tokens:0-class request, then fan out (25×30k prefix: ~$9.4 → ~$0.94 ESTIMATE).
• excludeDynamicSections enforced fleet-wide with an SDK version floor: six per-container fields (~111 tokens: cwd, git flag, platform, …) placed ahead of the prefix otherwise un-share the entire ~28–34k system prefix across every container in the fleet — the single biggest hosted-fleet lever (techniques 22 — Fleet, team & multi-tenant cache economics (hosted)).
• Byte-identical prefix discipline: same workspace, same model/effort/tool set across a fleet lane; any tool-definition change busts the whole fleet's prefix. Subagent-caching behavior is version-dependent (issue #29966 vs measured cavecrew writes) — the ledger audits cache_read_input_tokens > 0 per subagent class rather than assuming.
• Depth/fan-out caps: subagents nest 5 deep and agent teams run ≈7× tokens; the orchestrator enforces role-level caps and tears down idle teammates.

Lane scheduler

Three lanes per capsule, chosen by human-attachment (which jackin❯ knows from the attach state):

Quota and budget governance

• Dual cost model per capsule auth mode (headroom's auth-mode policy, driven by jackin❯ credential provisioning): API-key capsules optimize dollars; subscription capsules optimize tasks per limit window. The engine detects the active billing regime at launch instead of hard-coding plan, model, or headless-usage policy. Exact cap weights remain unpublished and therefore bounded (provider-sensitive claim verification).
• The cap denominator is unpublished: the engine ships the V2 prober — regress token-class consumption against anthropic-ratelimit-unified-* header utilization (captured pass-through, cache_control-preserving) to fit per-account weights, closing the dossier's biggest INCOMPLETE.
• Governors that actually govern (techniques 25 — Meta layer: the cost of optimizing, budget governance, and online quality guarding): workspace rate limits and monthly credit caps (real stops), gateway-tier budgets where a gateway exists; the vendor "spend limit" is alert-only and treated as such. Degrade-don't-die: budget triggers downgrade model/effort on routine lanes, never on critical lanes, and never truncate via max_tokens.
• Engine self-cost is metered in the same ledger (meta-governance: the meter must cost less than what it measures; count_tokens runs against its own separate RPM pool, batched, never inline per-step).

Additional routing constraints

• Mutation-gain arithmetic: any decision to rewrite a deep (already-cached) block uses headroom's corrected net_mutation_gain: the warm-case rewrite penalty applies over the saved tokens plus the downstream tail (S+ΔT), not S alone. Property tests ship with the formula.
• Learning never destabilizes prefixes: any adaptive policy (mode selection, register level, effort tier) selects by deterministic argmax over posterior means by default; stochastic exploration (Thompson/ε-greedy) is opt-in per role — lean-ctx's #498 doctrine, adopted as an invariant: a learner that randomizes prompt-visible behavior is a cache-buster.
• Subscription write-quarantine: capsules on subscription credentials consume learned recommendations read-only and never write into shared learning pools (headroom's toin_read_only pattern) — UX-bound traffic must not mutate policy trained for PAYG economics.
• Bandit spec baseline (if/when adaptive mode selection ships): Beta(2,1) arms over (entropy, jaccard, budget) triples; success ⇔ savings-ratio > 0.3 AND task completed; ×0.98 session-end decay floored at 1 — lean-ctx's ~300-line spec, adopted as the reference design rather than invented fresh.

Implications for jackin❯

These findings define the routing and fleet economics (l4) contract for jackin❯; any implementation must preserve the acceptance gates and boundaries above.

Limitations and unknowns

The design remains bounded by the evidence and acceptance gates above; volatile dependencies and provider behavior require revalidation before implementation.

Sources

Evidence and repository references are cited inline beside the claims they support.

Related work

• Context engine dossier
• Token-optimization techniques
• Token-optimization tools