Context & efficiencyToken-optimization techniques

27 — Current verdict and stack guidance

Current conclusions and stack guidance across quota, multimodal, latency, fleet, portability, governance, and quality evidence.

Summary

The current evidence preserves the central dollar verdict: about 2.5× is defensible without a quality trade, and about 5–6.2× is conditional on a routing change passing the acceptance harness. No reviewed mechanism supports an honest 10× claim at equal quality. Frontier-model thinking billed as output and the cache-read floor of genuinely useful context remain binding constraints.

For subscription users, the primary metric changes from dollars per task to accepted tasks per usage cap. That reorders the levers: prefix stability, context-window discipline, and request volume rise; uncontrolled fan-out can lose despite appearing cheap in API-dollar arithmetic. Provider quota weights remain partly opaque, so quota conclusions stay bounded where direct measurement is unavailable.

Question and scope

What reduction remains defensible after composing the durable findings, quota constraints, and unresolved frontier ideas?

Method

The verdict recomposes only independent, quality-gated effects from the durable chapters, then reports subscription and API-priced outcomes separately with unresolved inputs left explicit.

Findings

Current evidence by topic

TopicCurrent findingDetail
Subscription quotaUse tasks per cap; apply API dollars only to credits or explicitly off-cap traffic19 — Subscription and quota economics
MultimodalVision-tier routing can reduce image tokens by 67%; text-over-PDF can save about 50% on document tokens in measured cases20 — Multimodal token economics
LatencyA token-saving path is unacceptable when added wall-clock or human wait exceeds its value21 — Latency and time economics
FleetPrefix identity, dynamic sections, start timing, and provider cache scope determine cross-agent reuse22 — Fleet and multi-tenant cache
PortabilityInstruction-level techniques travel broadly; hooks, provider betas, and storage surfaces do not23 — Portability matrix
LiteratureKV-cache compression is meaningful for self-hosted inference but does not reduce hosted API-billed tokens24 — Literature and market scan
GovernanceMeasurement overhead, retries, raw recovery, and live quality checks belong in net accounting25 — Governance and online quality
FrontierConditional ideas remain gated by feasibility, provider support, and paired quality tests26 — Frontier mechanisms

Dollar verdict

The added mechanisms change choices at the margins but do not remove the two structural floors:

  1. Thinking from a frontier model is billed as output and is reachable mainly through effort control or model routing.
  2. Context the model genuinely needs still incurs a cache-read floor even under stable-prefix caching.

Multimodal substitution matters when media is a material share of the workload. Cached-context retrieval can reduce fresh-input cost, but it remains a context-architecture and caching lever rather than a new independent multiplier. Savings from overlapping layers must not be multiplied as if each acted on the full baseline.

Quota verdict

For a capped subscription, use a two-part objective:

accepted tasks per cap = accepted tasks / provider-accounted usage
overage cost           = credits or API traffic × applicable rates

The provider does not publish a stable conversion from all token classes to cap consumption. Measure relative changes from the provider surface and session records; do not present an inferred quota weight as a provider fact. Cache stability and fewer requests are robust priorities because both reduce avoidable repeated work even when the exact cap denominator is hidden.

Conservative stack

  • Stabilize system, tool, and project prefixes.
  • Keep raw recovery for every transformed observation.
  • Use retrieval to avoid irrelevant reads before compressing relevant ones.
  • Apply deterministic output discipline.
  • Measure fresh input, cache write, cache read, thinking, visible output, turns, rereads, latency, and accepted-task quality separately.

Quota-bound subscriber stack

  • Add request-volume and context-window budgets.
  • Stagger or pre-warm fleet starts only when measurement shows prefix reuse.
  • Gate fan-out by expected task value and cap pressure.
  • Attribute each capsule's provider-reported usage where the provider exposes it.

Conditional routing stack

  • Route only task classes that pass the paired acceptance harness.
  • Preserve an advisor or escalation path for hard cases.
  • Recompute savings for the actual current model pair and tokenizer family.
  • Reject the route when retries, cache fragmentation, or quality loss erase the nominal price benefit.

Cross-layer caveats

Server-side provider cache scope is not the same as a local file cache. Local recovery stores make compressed observations reversible; they do not prove shared provider-prefix reuse. Subagent cache behavior is version- and execution-path-dependent, so inspect current session records before relying on it.

Absolute dollar examples tied to one model price can age while ratios and structural constraints remain valid. Apply the provider-sensitive claim verification contract before using provider-sensitive numbers.

Implications for jackin

Current tier adjustments

MechanismCurrent treatmentReason
Prefix stability and cache preservationPromote for quota-bound workRepeated prefix writes and cache misses consume avoidable allowance
Request-volume disciplinePromoteExtra turns repeat fixed context and create more opportunities for cache misses
Uncontrolled subagent fan-outDowngrade for quota-bound workParallel agents can multiply requests and fixed-prefix consumption
Vision-tier routing and croppingConditionalStrong local savings only when media is present and still legible
Text extraction before full PDF/image ingestionConditionalUseful when structure and exact text survive extraction
Per-request gateway routingKeep harness-gatedModel savings can be erased by cache fragmentation or quality loss
Hosted KV-cache compression claimsRejectSelf-host GPU-memory savings are not hosted API token savings
Compression without raw recoveryRejectExact identifiers, errors, configuration, and security evidence require recovery

Current stack guidance

For jackin, the current verdict and stack guidance evidence identifies which mechanisms are safe to adopt directly and which still require workload-specific validation.

Limitations and unknowns

Rejected or downgraded claims

  • No evidence supports stacking every reported percentage into a 10× accepted-task claim.
  • A payload-reduction benchmark is not a task-level, quota-level, or quality result.
  • Self-host KV eviction does not lower billed hosted tokens.
  • A flat-rate subscription does not make unlimited retries or fan-out free; the usage cap is the scarce resource.
  • A cheaper route is not a saving when quality failures cause escalation, rereads, or repeated turns.

Sources

Evidence and repository references are cited inline beside the claims they support.

On this page