# 14 — The caveman ecosystem: flagship verified, cavemem and caveman-code deep-dived (https://jackin.tailrocks.com/reference/research/token-optimization-tools/14-caveman-ecosystem/)



# 14 — The caveman ecosystem: flagship verified, cavemem and caveman-code deep-dived [#14--the-caveman-ecosystem-flagship-verified-cavemem-and-caveman-code-deep-dived]

Round-2 source sweep (2026-07-03; clones: caveman @ `25d22f8`, caveman-code @ `74d599a`, cavemem @ `1fe41e9`). The flagship's hub claims were verified line-by-line against source; **cavemem** and **caveman-code** — previously one-line entries in the family diagram — get their first equal-depth analysis. Companion verification pass for RTK/headroom/lean-ctx/codedb: [15 — Verification pass](/reference/research/token-optimization-tools/15-verification-pass-and-extended-matrix/).

## TL;DR [#tldr]

* **The flagship holds up.** Every load-bearing claim in [01 — caveman design](/reference/research/token-optimization-tools/01-caveman-design/) confirmed at source: six levels, two hooks, verbatim guards, Auto-Clarity, the eval methodology, the honest \~50%-over-terse-control number (re-run: +50% median). One wording fix: the installer hand-validates hook shapes *to survive* Claude Code's Zod, it doesn't run Zod itself. The README now states the 65% per-task mean on its own line — cite it directly. **+6.8k stars in the 15 zero-commit days after v1.9.0 (2026-06-12)** — the promotion-driven-adoption signal, re-confirmed. v1.9.1 shipped 2026-07-03 (50 changes: headline rebased to 65%, hook/installer hardening, `docs/HONEST-NUMBERS.md` — [https://github.com/JuliusBrussee/caveman/releases/tag/v1.9.1](https://github.com/JuliusBrussee/caveman/releases/tag/v1.9.1)), so the signal now rests on the star velocity alone, not on a frozen repo. First published **null result** exists (GrowwStacks: "no measurable token savings", via the servicesground roundup) alongside Pillitteri's 61–68%-on-discursive-text ≈ 4–10% whole-session — both consistent with the dossier's \~4–6% modeling.
* **cavemem is refuted as shipped.** Its compression is **deterministic regex + a 5.9 KB lexicon, not the model**; measured on its own eval corpus it delivers **10% (lite) / 33% (full, the default) / 36% (ultra)** against the marketed 30/60/75 — and even those savings mostly never reach the model, because the MCP read tool **re-expands by default**, refunding abbreviations at read time. Compounding defects: FTS indexes *compressed* text but queries arrive raw (vocabulary mismatch breaks its own lexical retrieval), retrieval is global across every project on the machine, the summaries tier is unsearchable, `excludePatterns` privacy config is dead code, growth is unbounded, and hedge-stripping deterministically converts speculation into assertion ("It could potentially break prod" → "It break prod"). **Dormant 2026-05-17 → 2026-07-03 (44 open issues, zero maintainer replies); maintenance resumed 2026-07-03** — issues triaged 44 → 4, npm `dist/index.js` packaging bug fixed, opt-in SSRF/ReDoS-hardened web-enrichment MCP tool added, Windows PowerShell 7 hooks, in-repo version 0.3.0 (npm still 0.2.1) — [https://github.com/JuliusBrussee/cavemem/commits/main](https://github.com/JuliusBrussee/cavemem/commits/main). The measurement-based refuted-as-shipped verdict stands until re-measured. One keepable artifact: the preserved-segment tokenizer (fence/code/URL/path/version/number spans byte-exact through any transform) — \~100 lines, the correct safety contract for prose transforms.
* **caveman-code was mischaracterized.** Not a "code variant" of the skill — a **full terminal coding agent** (open fork of pi-code, 9-package TS monorepo, TUI + CLI + SDK, 20+ providers) whose token story is **four stacked layers**: a 3-level register (\~120–195 prompt tokens, hard exceptions for code/commit-messages/PR-text/security), per-tool head/tail line budgets ("Flint Chipper": bash 80, read 300, grep 120), read-dedup-by-fingerprint stubs, and **RTK integration on-by-default when the binary is present** — plus opt-in LLMLingua-2 (default off). Its "\~2× fewer tokens vs Codex" is a real, artifact-backed benchmark (honest-bench: 524,703 vs 1,010,185 fresh tokens, 25 tasks, real child processes, committed CSV/JSON/logs) — **but** it measured the full stack at ultra+ML-on (more aggressive than shipped defaults), Codex's token accounting is asymmetric (printed totals may include cache), and caveman-code passed **1–2 fewer tasks** than Codex in both runs. First-party-reproducible tier; the best benchmark hygiene in the family, with real caveats.
* **Family growth**: **cavegemma** (73★ 2026-07-03; 79★ 2026-07-05, still frozen since 2026-05-17) — a LoRA fine-tune of Gemma to "speak caveman natively", i.e. bake the register into weights and drop the prompt (self-host-only lever; watch, don't adopt). cavekit 1,068★ (2026-07-03); 1,074★ (2026-07-05), now at v4.1.0 (tag 2026-06-18, [https://api.github.com/repos/JuliusBrussee/cavekit/tags](https://api.github.com/repos/JuliusBrussee/cavekit/tags)). caveman-shrink is real and guarded (27k npm downloads/month at the 07-03 sweep; 23.3k/month as of 2026-07-05 — and its published tarball was missing `spawn-options.js` until fixed alongside v1.9.1, #597). The README now cites arXiv 2604.00025 ("+26 points under brevity") — the dossier's records ledger already marked that paper do-not-propagate (solo, unreviewed, cherry-picked subset).

## Flagship verification (claims → verdicts) [#flagship-verification-claims--verdicts]

| Hub claim                                                    | Verdict        | Note                                                                                                                                                                                           |
| ------------------------------------------------------------ | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Six levels; ultra never abbreviates code symbols             | ✅              | `skills/caveman/SKILL.md:34-41`; repo default = **full**                                                                                                                                       |
| Two hooks (SessionStart inject / UserPromptSubmit re-assert) | ✅ + richer     | activate hook injects the *full level-filtered SKILL.md* (drift fix); tracker adds NL-trigger detection; installer gained orphan-hook pruning (#471), JSONC-tolerant reads, atomic 0600 writes |
| "Zod-validated" installer                                    | ⚠ wording      | hand-validation *to survive* Claude Code's strict Zod (one malformed hook silently discards the whole settings file)                                                                           |
| \~940-token skill listing                                    | ⚠ envelope     | recount: 576 tokens bare; marketplace installs register skills **twice** (standalone + namespaced) ≈ 1,150 with scaffolding — environment-dependent ±40%                                       |
| Eval: o200k, terse control, \~50% over control               | ✅ re-run       | +50% median / +46% mean (n=10, snapshot); benchmarks/run.py baseline is a verbose "helpful assistant" prompt — banks the generic terse effect, as recorded                                     |
| README 75% pooled / 65% per-task                             | ✅              | README now itself states "Average 65% output reduction (range 22–87%)" — cite `README.md:136`                                                                                                  |
| caveman-shrink #474 guarded, not live                        | ✅              | issue closed; installer rejects the broken-stub form; regression-tested                                                                                                                        |
| cavecrew −43.9% measured vs 60% claimed                      | ✅ (claim side) | repo contains **zero** cavecrew measurement; the dossier's −43.9% remains the only number in existence                                                                                         |
| Issue #484 (over-shrinking PR titles)                        | ✅ open         |                                                                                                                                                                                                |
| No agentic quality benchmark                                 | ⚠ rescope      | true **for the skill**; the family now has one — caveman-code's honest-bench publishes pass rates (and shows the caveman arm 1–2 tasks *behind* Codex)                                         |

## cavemem (first deep dive) [#cavemem-first-deep-dive]

**Identity.** TypeScript pnpm monorepo (7 packages + MCP server + CLI + embedding worker), MIT, created 2026-04-18, **dormant 2026-05-17 → 2026-07-03** — the initial project lived one month, then went quiet; maintenance resumed 2026-07-03 (issues triaged 44 → 4, packaging fix, in-repo 0.3.0 — npm still 0.2.1; see the TL;DR). npm `cavemem@0.2.1`, \~2.6k downloads/month. 581★ / **1 watcher** / 44 unanswered issues at the 07-03 sweep.

**Architecture.** SQLite WAL at `~/.cavemem/data.db`: `sessions`, `observations` (every prompt + every tool call, 4 KB cap), `summaries` (turn/session), `observations_fts` (FTS5 porter), `embeddings` (raw Float32 blobs, **no ANN index** — brute-force cosine over *all* embeddings per query). Compression = **code, not model**: priority-resolved span tokenizer (fence > inline-code > URL > heading > path > date > version > number > identifier, byte-exact pass-through) + four regex drop-passes (pleasantries/hedges/fillers/articles) + a 5.9 KB abbreviation lexicon. `expand()` reverses abbreviations only. Retrieval: FTS5 BM25 + cosine, min-max + fixed-alpha blend (not RRF); embeddings computed on the *expanded* text (the one genuinely thoughtful design choice). MCP surface: **4 tools ≈ 300 tokens** (genuinely lean). Five hooks capture prompts/tool-calls/turn-summaries; session-start injects up to 3 same-cwd prior session summaries.

**Measured vs marketed.** Marketing "\~75%" is a config-schema description string ("lite ≈30%, full ≈60%, ultra ≈75%"), benchmarked nowhere. The repo's own spec targets 30/40/55. An `evals/` bench exists (202-word corpus, chars/4 counter). Live run on that corpus: **lite 10% / full 33% / ultra 36%** — misses even its own targets. And `get_observations` defaults `expand: true`, so stored-compression savings are largely **refunded before the model reads them**; the at-rest savings mostly save disk, not context.

**Failure inventory** (each verified in source): FTS vocabulary mismatch (indexes compressed "db", queries raw "database" — porter can't bridge abbreviations); cross-project retrieval leakage (cwd-scoping parameter exists, no caller passes it); summaries tier unsearchable (no FTS, no embedding, no MCP tool); no dedup/TTL/eviction anywhere; no recency/supersede semantics (a stale April guess outranks a June correction); `privacy.excludePatterns` documented but referenced nowhere in executable code; hedge-stripping is deterministic modality destruction — every hedged memory is systematically overclaimed on recall, and `expand` can rewrite words the author wrote ("auth" → "authentication"); injection cost unmeasured and structurally uncapped (3 × 20-turn concatenated summaries at session start).

**Corrections to the prior hub description**: compression is *not* model-side; "lossy, no recovery" → *partially reversible, semantically unsafe*; "single-agent" → cross-IDE by design (Claude/Cursor/Gemini/OpenCode/Codex into one DB) but single-machine and unintentionally global-across-projects; "no benchmark" → trivial benchmark exists; "\~75% (expect 55–60%)" → **33–36% measured, mostly refunded at read time**; stats: 581★/1 watcher; dormant 7 weeks until maintenance resumed 2026-07-03 (see above).

**Verdict.** Architecturally superseded by any reversible-store design (headroom CCR / lean-ctx archive class): storing lossy prose with no ground-truth path makes corrupted memories unverifiable forever. The keepable artifact is the **preserved-segment tokenizer as a transform safety contract**, and the four self-inflicted wounds convert into requirements for any engine: index the canonical form (or transform queries identically), embed canonical text, scope retrieval per project by default, budget and *measure* injection.

## caveman-code (first full analysis) [#caveman-code-first-full-analysis]

**Identity.** Standalone coding agent, TS monorepo of 9 packages, MIT, binaries `caveman`/`caveman-code`; an open, credited **fork of pi-code** with Claude Code-compatible config surfaces (`~/.cave/settings.json`, `commands/`, `skills/`, `agents/*.md`, `.mcp.json`; observer-only hooks). Plan mode, autonomous goal loop, architect/editor model split, session branching + shadow-git checkpoints, 5 builtin worktree-isolated subagents. Created 2026-04-08; **671★ / 3 watchers** (the family's un-promoted member); npm `@juliusbrussee/caveman-code` 0.65.2, \~20.7k downloads/month, single published version.

**The four compression layers** (defaults: register `full` on, budgets on, dedup on, RTK on-when-present; LLMLingua-2 ML layer **off**):

1. **Cave Mode register** — 3 levels (no wenyan), 120–195 system-prompt tokens, hard exceptions at every level: code blocks, **commit messages and PR descriptions**, security warnings, destructive-op confirmations. Note the family's internal contradiction: the caveman plugin ships `/caveman-commit` that *compresses* commit messages; caveman-code *exempts* them — the same author reached the hub's own conclusion (order-sensitive/durable text should not be register-compressed).
2. **Per-tool line budgets** ("Flint Chipper") — head/tail caps per tool: `bash 80 (50h/30t)`, `read 300 (200/100)`, `grep 120 (80/40)`, `find/ls 60`, fallback 150; + ANSI strip, blank-line collapse, semantic JSON/XML extraction. Claimed −67…94% per fixture (offline bench in-repo).
3. **Read-dedup** — per-session content fingerprints; re-reads return a stub ("−99% on repeats") — the sem/lean-ctx attention-ledger idea, independently converged on.
4. **RTK bridge** — detects the external Rust `rtk` binary and rewrites bash through `rtk rewrite` via a spawn hook, **enabled by default**: an agent vendor shipping RTK integration on-by-default is direct prior art for the layered-stack thesis.

**The 2×-vs-Codex claim, audited.** honest-bench (2026-05-18, committed harness + raw artifacts): 25 verify-scripted tasks, caveman and Codex CLI spawned as real child processes, same `gpt-5.5`/`xhigh`: **524,703 vs 1,010,185 fresh tokens (1.93×), 14/25 vs 15/25 resolved**. A same-day `thinking=high` variant: 2.00× but **−2 tasks**. Caveats that must ride with the number: the caveman arm ran `rtk=on, cave-mode=ultra, tool-compression=on, ml=on` — more aggressive than shipped defaults; Codex tokens are stdout-scraped with a fallback path that books totals as input (`cache_read=null`) so the denominator may include cached tokens; single run per task, no CIs; pass rate below Codex in both runs. Tier: **first-party but genuinely reproducible** — the best benchmark hygiene in the family (config fingerprints in the CSV, per-task logs, one-command repro), well above the skill's README benchmark; still self-run and quality-negative.

**Reusable for a native engine**: the register exception list; the per-tool line-budget table (portable constants); read-dedup stubs; the subagent output contract ("file:line over prose, no emojis, terse"); and the honest-bench harness design (real processes, per-task `verify.sh`, fresh-token accounting, config-fingerprint column, committed artifacts) as the template for comparative benches.

## What this changes in the hub [#what-this-changes-in-the-hub]

1. **Family matrix rows updated**: cavemem = *refuted as shipped* (33–36% real, refunded at read time; unmaintained); caveman-code = *full agent, 4-layer stack, reproducible-but-caveated 1.93×* — not a register variant.
2. The memory either/or table in [06 — Combining](/reference/research/token-optimization-tools/06-combining/) tilts further against cavemem: it now loses on *measurement* too, not just architecture.
3. "No agentic quality benchmark anywhere in the family" is obsolete — honest-bench exists and its pass-rate gap (−1/−2 tasks) is the first quality datapoint for register-style compression on agentic work: directionally negative, n too small to settle.
4. Evidence ledger additions: GrowwStacks null result; Pillitteri 4–10% whole-session; cavegemma as the register-in-weights frontier (self-host-only).