docs/features/polykv_api.md

# PolyKV Control-Plane API — Design (F5 M6-S5 / #712)

**Status:** design (2026-07-16). Implementation is phased (§10); nothing here
is shipped yet.
**Inputs:** [polykv_api_research.md](polykv_api_research.md) (design-space
survey), [poly_kv.md](poly_kv.md) (the KV mechanism this sits on top of),
[llamafile-usage.md](../llamafile-usage.md) §3.4 (SharedKVPool user surface).
**Single rule:** additive / opt-in. Every endpoint and field lives under the
existing `opencoti` JSON namespace or new `/polykv/*` routes; no upstream
behavior changes when PolyKV control is unused.

---

## 1. Goals

A rich, opt-in control plane that satisfies every multi-agent serving journey
on top of the existing SharedKVPool (`shared_pool_slot` +
`shared_prefix_n_tokens`, server-context.cpp:3085). Four requirements, plus
two the requirements imply:

1. **Full status** of pools + the sessions attached to each.
2. **tps reporting** — aggregate system throughput *and* per-session
   generation tps. (All-new: upstream llama.cpp reports per-*response*
   `timings` but has no per-session/slot running instrumentation.)
3. **Autonomous capacity/admission loop** — a main session sets a per-session
   gen-tps target (e.g. 5 tps); when the average across active sessions
   approaches the floor the orchestrator holds off spawning sub-agents.
   Behavioral knobs: reject a new session vs. warn.
4. **LangGraph extensions** — a custom `BaseChatModel`, the pool-control API
   exposed as LangGraph tools, and PolyKV-aware agent-state routing.
5. **Nesting** (implied by 3): a sub-agent that discovers fan-out work can
   become a pool parent for its own sub-agents, so parallelizable tasks run
   in parallel instead of being forced sequential.
6. **Fork / branch** (COW): a sub-agent group that shares an *ancestor* prefix
   but then diverges (different toolset/context) gets its own shared pool
   branched at the divergence point.

## 2. Decisions locked (2026-07-16)

| # | Decision | Choice |
|---|----------|--------|
| API home | where the control plane lives | **Self-contained in the C++ engine** — extends `/props`, `/slots`, adds `/polykv/*`. Works standalone; no `opencoti-server` in the path. |
| Admission | enforce the tps floor or advise | **Both** — advisory (report) by default; optional per-pool enforced mode with `reject`\|`warn`\|`queue`. |
| Pool model | how pools are created | **Explicit first-class objects** — `create`/`fork`/`attach`/`release`, immutable from birth (§3.3). Contiguous-prefix contract checked at fork. Boilerplate hidden by the opencoti plugin. |
| Scope | deliverable shape | **Design the full surface now; implement in gated phases** (§10). |
| Fork | divergent child prefix | **First-class in v1** (copy-free COW, §3.2 / §7) — *not* deferred. |

**Defaulted (override any):** transport = HTTP on the engine + SSE for the
live tps stream (poll fallback); tenancy = single trust domain (no per-pool
ACL); tps = per-session *experienced* decode tps (EWMA) + aggregate decode
throughput, floor = mean of active sessions' experienced tps (prefill
excluded); nesting depth uncapped; interior pools pinned while any descendant
is attached + grace TTL; seq-ids for pools+leaves sized at boot.

---

## 3. Data model

### 3.1 A pool is a tree node

```
Pool {
  id            : int32          // seq_id in the unified cache
  parent        : int32 | ROOT   // -1 for a root pool
  branch_pos D  : int32          // shared-ancestor length inherited from parent
  own_range     : [D, L)         // tokens this pool materialized itself
  prefix_len L  : int32          // total shared prefix = D + |own_range|
  tokens        : [llama_token]  // full prefix token array [0, L), host-side (auto-P + fork validation)
  prefix_hash   : [u64]          // cumulative rolling hash per position (validates any D; ~8 B/token host RAM)
  pin           : bool           // resident-pinned (auto while descendants attached)
  children      : [Pool]         // interior pools branched/extended from this one
  sessions      : [Session]      // leaf sharers attached at this pool's prefix
  last_access   : ts
}
Session {                        // a leaf = an ordinary decoding slot
  id            : int32          // seq_id
  pool          : int32          // the pool it attached to
  suffix_range  : [L, L+S)       // its own private, mutable continuation
  tps_ewma      : f32            // experienced decode tps
  ttft_ms, state, ...
}
```

The **root** pool (`parent = ROOT`, `D = 0`) is what today's
`shared_pool_slot` request already produces. Everything else is new.

### 3.2 Three edge types, one mechanism

A child pool inherits `[0, D)` from its parent and owns `[D, L)`. `D` selects
the edge type:

- **extend** — `D = parent.L`. Child = parent prefix **+** appended shared
  tokens. This is *nesting*: a sub-agent adds its own shared context and hands
  it to its sub-agents.
- **branch (COW fork)** — `D < parent.L`. Child shares the common ancestor
  `[0, D)` and diverges with its own `[D, L)`. This is the fork: same system
  prefix, different toolset after `D`.
- **root** — `parent = ROOT`, `D = 0`.

**Why the fork is copy-free.** `seq_cp` in the unified cache is additive
membership: for each cell where `seq_has(cell, parent)` in `[0, D)` it calls
`seq_add(cell, child)` (llama-kv-cache.cpp:2007-2044). No cell is duplicated —
the ancestor cells simply gain the child in their sequence set. Because
prefixes are **immutable** (FROZEN, §3.3), the shared range is never mutated,
so the "copy" a classic COW would eventually pay never happens. The child only
computes new KV for `[D, L)`, prefilled at RoPE position `D` attending to the
shared `[0, D)` already in cache — exactly how attaching with a shorter
`shared_prefix_n_tokens` works today. **Extend and branch are the same code
path parameterized by `D`.**

### 3.3 Immutability & versioning

**Pools are immutable from birth.** A pool's prefix is fully materialized at
`create`/`fork` and never changes afterwards — there is no OPEN/extend-in-place
state. "Extending" a pool *always* means creating a child (`fork` with
`D = parent.L`). This is simpler than a freeze-on-first-attach state machine
and loses nothing: the "build context incrementally" journey is covered by the
main agent decoding in its **own session** and then forking `from_session`
(§4.1), which snapshots its current context as a new pool in one call — no
in-place mutation ever needed.

This makes "add a tool to the shared prefix after agents attached"
well-defined — it is a *new versioned pool* (a fork), and the old one keeps
serving its sharers unchanged. (This is the precise answer to the HuggingFace
question: the prefix is static per pool; a tool/MCP added later is a new pool
version, valid only for sessions that attach to it — never mutated under
sessions already on the old one.)

### 3.4 Lifetime

Cell freeing is implicit-refcounted: a cell is reclaimed only when its sequence
set empties, so an ancestor's cells physically survive as long as *any*
descendant references them — even if the ancestor's own slot is torn down. On
top of that:

- **Pin policy.** Interior pools are auto-pinned (`--no-clear-idle` semantics,
  per-node) while any descendant is attached. Without this a new leaf attaching
  to an idle-cleared interior pool hits the residency guard
  (server-context.cpp:3116) and silently falls back to full reprocess —
  correct but slow. A **grace TTL** delays reclaim after the last descendant
  detaches (cheap re-attach for bursty fan-out).
- **seq-id budget.** Every pool *and* every leaf consumes one `n_seq_max` slot.
  A deep/wide tree spends seq-ids on interior nodes too; the boot-time reserve
  accounts for both. Exposed in `/capacity`.

### 3.5 Materialization — the slot/seq decoupling (P2's real work)

Today a "pool" is just an ordinary server **slot** that happened to serve a
request: `shared_pool_slot` names a slot index, the pool's tokens live in
`slots[pool].prompt.tokens`, and the pool seq **is** the slot's seq. First-class
pools break that coupling, and this is the largest single implementation lift
in the milestone:

- **seq-ids beyond slots.** Pool seq-ids come from a boot-time reserve above
  the slot range: `n_seq_max = n_parallel + --polykv-max-pools` (default
  proposed: 16). A pool occupies a seq-id but **no slot** once materialized.
- **Prefill-only internal task.** `create`/`fork` materializes `[D, L)` via an
  internal task that runs through the normal batching path (so it composes
  with every KV feature) but emits **no logits and no generation**; it
  transiently borrows a slot for the prefill, then the KV stays on the pool's
  seq and the slot is returned. `seq_cp(parent → pool, 0, D)` runs first for
  fork/extend.
- **Host-side pool state.** The Pool object (tokens, cumulative hash, tree
  links, pin/TTL) lives in server memory beside the slot table — *not* in a
  slot — and is what `/pools` reports and attach validates against.
- **Cache layout (REVISED c6, bug-2248).** Originally pools required
  `--kv-unified` (the split-stream `seq_cp` overload hard-asserts
  `is_full`, so a partial cross-stream share was undefined — this was a
  boot-time refusal). Since patch `0145` pools work on BOTH layouts:
  unified shares stay zero-copy set-membership; on the split cache
  (`--parallel` without `--kv-unified` — the layout the rolling-KV
  position window requires) every prefix share runs as a PHYSICAL copy
  via `oc_seq_share_prefix()` (full-stream `seq_cp` + trim to `[0,P)`,
  positions preserved; cross-stream copies include the window's host
  tail). Trade-off: no VRAM dedup off unified — each sharer stream
  holds its own prefix copy (window mode caps device use at
  `window_cells`/stream). The boot check is now an INFO stating the
  copy-share semantics. `/capacity` is STREAM-AWARE on the split cache
  (patch `0146`): the verdict checks the per-stream budget and free
  streams ("no free stream (split cache)" hard-stop; headroom clamped
  to free streams) and the payload adds `kv_streams_total` /
  `kv_streams_free` / `kv_cells_per_stream` (null on unified).

---

## 4. API surface

All additive. Reuses the `opencoti` JSON namespace (`get_opencoti_props()`
:3787) and the `/slots` accumulator precedent (:183). Endpoint shapes follow
research §6.

### 4.1 Pool lifecycle (imperative, explicit)

| Verb | Endpoint | Body → result |
|------|----------|---------------|
| create root | `POST /polykv/pools` | `{tokens? \| from_session?}` → `{pool_id, prefix_len}`. `from_session` snapshots a live session's current context as the pool prefix — the **primary** creation path for agents (no token round-trip). |
| fork (extend or COW branch) | `POST /polykv/pools/{id}/fork` | `{branch_pos D, tokens[D:L] \| from_session?}` → new child `{pool_id, parent:id, branch_pos, prefix_len}`. `D == parent.L` ⇒ extend (nesting); `D < parent.L` ⇒ COW branch. |
| attach | *(request path, unchanged mechanism)* | per-request `pool_id` (legacy `shared_pool_slot` kept). **`shared_prefix_n_tokens` becomes optional**: the server auto-computes `P` as the longest hash-match between the request prompt and the pool's stored tokens (**auto-P**) — killing the mis-set-P footgun and the HF-question confusion. An explicit `P` is still accepted and validated. |
| pin/evict/flush | `POST /polykv/pools/{id}/{pin\|evict\|flush}` | retention control (research §6 Req 1) |
| release | `DELETE /polykv/pools/{id}` | detach + reclaim when refcount hits 0 (respects grace TTL) |

Contiguous-prefix contract is checked at `fork`: the child's declared
`tokens[0:D]` must hash-match the parent's `[0:D)` (cumulative `prefix_hash`
at `D`), else `409 Conflict` — the caller then either re-roots or
full-prefills. All routes live under a `/polykv/` prefix so a future upstream
llamafile/llama.cpp bump can never collide with an unprefixed `/pools`.

### 4.2 Status (research §6 Req 1)

- `GET /polykv/pools` → `{pools:[{pool_id, parent, branch_pos, prefix_len,
  own_kv_bytes, subtree_kv_bytes, residency, pinned, last_access_ts,
  children:[pool_id], sessions:[{session_id, suffix_n_tokens,
  marginal_kv_bytes, tps_ewma, ttft_ms, state}]}], tree_depth, seq_ids_used,
  seq_ids_max}`. The `parent`/`children` fields make the **tree** explicit.
  **KV attribution is by ownership, never by membership**: a shared cell
  belongs to many seqs, so summing per-seq bytes would double-count — a pool
  reports `own_kv_bytes` = its `[D, L)` range only, plus a `subtree_kv_bytes`
  rollup (own + all descendants + attached sessions' suffixes). Σ own over the
  tree == physical bytes, by construction.
- `GET /polykv/pools/{id}` → one subtree.
- `/slots` gains `pool_id` + `tps_ewma` per slot (extends #677).
- `/props` `opencoti` block gains a `polykv` sub-object (config: floor,
  admission mode, pin/TTL policy).

### 4.3 tps (research §6 Req 2)

- **Per-response** (already partly present): `timings{predicted_per_second,
  prompt_per_second, cache_n}`.
- **Headers** (TGI model): `x-session-tps`, `x-cached-prefix-tokens`,
  `x-pool-id`. **Deferred to P2+**: HTTP headers must be emitted before the
  body, but the slot (and thus the live tps value) is assigned only after the
  task is queued — a header set at request time would always be stale, and
  trailers aren't portable. The per-response `timings` JSON already carries the
  authoritative per-response rate; headers add value only once pool attach
  happens at request routing (P2), where `x-pool-id`/`x-cached-prefix-tokens`
  are known up front.
- **Live stream:** `GET /polykv/tps` (SSE) — periodic
  `{ts, aggregate_tps, sessions:[{session_id, pool_id, tps_ewma}], mean_active_tps}`.
  This is the signal the admission loop subscribes to.
- **`/metrics`** (Prometheus, low-cardinality per research §4): gauges
  `opencoti_polykv:gen_throughput_tps{pool}`, `pool_kv_bytes{pool}`,
  `pool_sessions{pool}`; counters `prefix_cache_hits_total{pool}`,
  `queries_total{pool}`. Label = pool only (never session — cardinality).

### 4.4 Admission (research §6 Req 3; decision: both modes)

- `POST /polykv/pools/{id}/admission` `{target_tps_per_session, mode:"advisory"|
  "enforced", on_saturation:"reject"|"warn"|"queue", guarantee_min_sessions,
  settle_tokens, settle_max_ms}` — per-pool policy (last three = P7 §14,
  defaults 1 / 48 / 5000).
- `GET /polykv/pools/{id}/capacity` →
  `{can_admit, projected_mean_tps_if_admitted, headroom_sessions, kv_headroom_pct, reason}`
  + P7 (§14): `{settling, settle_remaining_ms, n_warming, n_pool_sessions,
  guaranteed, projected_mean_tps_model, projected_mean_tps_measured,
  drop_per_admit_ewma, known_session}`. The projection is measured-drop
  based once the pool has absorbed ≥1 admit; the raw mean excludes
  warming (EWMA-cold) sessions.
  `headroom_sessions` is computed against **marginal** per-session KV
  (`S_i·bpt`), not full context — the basis of the ~7× reduction (research §5).
- **Advisory (default):** endpoints report; the orchestrator decides. Nothing
  is rejected server-side.
- **Enforced (opt-in):** on a new attach with projected `mean_active_tps <
  floor` OR `kv_headroom` exhausted → `reject` = `429`/`503` + `Retry-After`
  (RFC 6585/9110); `warn` = admit + `X-Sessions-Remaining` header; `queue` =
  hold until headroom returns. P7 (§14): the gate applies to **NEW sessions
  only** (an existing session→slot affinity = continuation, never gated);
  `"overcommit": true` in the request body bypasses it explicitly; and
  `reason:"measurement settling"` maps to a bounded HOLD (≤ `settle_max_ms`
  + 1 s) rather than a 429.
- **Throttle-toward-floor is explicitly NOT v1.** Andes' insight (over-floor
  generation is wasted GPU → throttle fast sessions back toward the floor to
  reclaim capacity) requires per-slot rate control inside the continuous-
  batching decode loop — i.e. skipping a slot in some decode batches. That is
  scheduler surgery with its own correctness/perf gates, not an API feature;
  it is banked as a P4-follow candidate, and v1 `warn` is headers-only.

---

## 5. tps instrumentation (all-new, C++ server-side)

Upstream has no per-session running tps. Design:

- Extend the `/slots` lifetime accumulators (server-context.cpp:183) with a
  per-slot decode-token counter + wall-clock, updated at each decode-batch
  completion for the tokens attributable to that slot.
- **Experienced tps** per session = EWMA over a rolling window (default: last
  `W` decoded tokens or `T` ms, whichever first) of that slot's decode
  throughput. This is the quantity that *falls as concurrency rises* even while
  aggregate climbs (attention doesn't batch-amortize — research §5), so it is
  the correct SLO signal.
- **Aggregate** = system decode throughput (all slots' decode tokens / wall).
- **`mean_active_tps`** = mean of experienced tps across currently-decoding
  sessions — the value the floor compares against (matches "average sessions
  gen tps").
- Prefill tokens are excluded from "gen tps" (counted separately as
  `prompt_per_second`).
- Cost: a handful of integer counters + one EWMA update per slot per batch —
  negligible; gated behind the PolyKV block so off-path is byte-identical.

---

## 6. Admission control

Two coupled signals (research §5):

1. **Soft / low-lag:** `mean_active_tps` vs the floor. Directly measures the
   SLO; leads `num_requests_waiting` (which lags).
2. **Hard wall:** pool KV-occupancy %. Reject *before* this forces
   preempt-oldest eviction.

`projected_mean_tps_if_admitted` estimates the post-admit floor from the
`B_sat` knee model (below the knee admitting is ~free; above it every admit
costs everyone's TPOT ~linearly). On high-VRAM cards (RTX 6000 96 GB) the tps
knee is hit **before** the KV wall, so signal (1) dominates on opencoti's
hardware — which is exactly why the floor, not occupancy, is the headline.

Framing: "5 tok/s/session" = **TPOT SLO 200 ms/token; maximize goodput s.t.
TPOT ≥ floor** (a floor, not a maximize — Andes). The default is advisory: the
LangGraph capacity-gate node (§8) reads `/capacity` and decides. Enforced mode
is the same math with server-side `reject`/`warn`/`queue`.

---

## 7. Nesting & COW-fork — worked journeys

**Journey A — sub-agent parallelization (nesting / extend).** Main agent M
runs on root pool `P0 = [system+tools]` (len N0, FROZEN). M builds task context
and `POST /pools/P0/fork {branch_pos:N0, tokens:[N0:N1]}` → interior pool `P1 =
[system+tools+task]`. M spawns k sub-agents that each attach to `P1` and decode
their own suffix in parallel. Physically, `P1`'s `[0:N0)` cells are P0's cells
(now in seq-set {P0,P1,leaf…}); `[N0:N1)` is prefilled once. The k sub-agents
cost only their marginal suffixes. The fan-out work that used to run
sequentially now runs concurrently under one shared branch.

**Journey B — divergent toolset (COW fork).** Two sub-agent groups share the
system preamble `[0:D)` but need different toolsets. Group B does
`POST /pools/P0/fork {branch_pos:D, tokens:[D:D']}` (D < N0) → `P1' =
[system+toolsetB]`, sharing `[0:D)` zero-copy, prefilling `[D:D')` fresh. Its
sub-agents attach to `P1'`. No physical copy of the shared preamble.

**Invariants enforced.** (a) contiguous-prefix hash check at `fork`; (b)
immutability from birth — a pool's prefix is fully materialized before any
descendant can attach (§3.3), so no attach ever races a prefix change; (c)
recursive pin — interior pools stay resident while descendants attached; (d)
seq-id budget counts interior nodes. Divergence that is *not* a clean prefix of any
existing pool → new root (or full prefill) — reported, never silently wrong.

**Hybrid/recurrent guard (bug-2203) is recursive.** Every attach/fork on a
recurrent or hybrid model still hits the exact-full-state guard
(server-context.cpp:3116) at every level.

---

## 8. LangGraph integration (research §6 Req 4)

Package home: a new **top-level `opencoti/` non-bun root** (per the repo's
"new non-bun roots" rule) — Python package `opencoti-langgraph` (pip). Kept out
of the bun workspace. Pin `BaseChatModel` signatures to a specific
`langchain-core` version tag (the API is mid-migration — research caveat).

- **`OpencotiChatModel(BaseChatModel)`** — implement `_generate` + `_llm_type`
  (required), `_stream`/`_agenerate`/`_astream`; override `bind_tools`.
  Populate `usage_metadata` and
  `response_metadata{pool_id, session_tps, backpressure}` from the headers/§4.3.
- **Pool-control tools** — `create_pool`, `fork_pool`, `pool_status`,
  `session_tps`, `capacity` exposed as LangChain tools via `bind_tools` /
  `ToolNode`, so an agent can manage its own pool tree.
- **Capacity-gate node** — reads `/capacity`; returns
  `Command(update={"capacity_ok":ok, "headroom_sessions":n}, goto=...)` to
  route the graph (spawn vs. hold). `interrupt()` for block-until-free.
  `InMemoryRateLimiter` is explicitly insufficient — it can't see per-session
  tps or KV headroom.
- **Agent-state routing** — a reducer surfaces `headroom_sessions` /
  `mean_active_tps` into graph state so conditional edges branch on live PolyKV
  capacity.

---

## 9. Composition with the KV stack

PolyKV control is orthogonal to the KV *mechanics* and must not perturb them:

- **rolling-KV / DCA / turbo / TCQ / quant-KV** — pool sharing is `seq_cp`
  membership on whatever cell type is configured; the tps/admission plane is
  read-only over the existing decode loop. Off-path byte-identical (gate).
- **MTP / speculative** — per-session tps counts *accepted* tokens (the user-
  visible generation rate), not draft steps.
- **`--parallel ≥ 2`** — the multi-slot requirement is already the
  SharedKVPool baseline; the tree just adds interior seq-ids.
  `--kv-unified` is optional since c6 (zero-copy shares with it,
  copy-shares without it — see the layout note in §3.5).
- **hybrid/recurrent** — bug-2203 guard, recursive (§7).

## 10. Phased implementation plan (gated)

Full surface designed here; implementation lands in gated phases, each with a
correctness + off-path-identity gate before the next:

- **P1 — tps instrumentation** (§5). Per-slot counters + EWMA + `/slots`/`/props`
  exposure + `/polykv/tps` SSE + `/metrics`. Prereq for everything.
  **✅ IMPLEMENTED 2026-07-16** (live in vendored tree, patch capture pending):
  gates green — sustained EWMA −3.2% vs `timings` ground truth; 2-session
  concurrency shows correct `aggregate_tps` ≈ Σ sessions + `mean_active_tps`;
  `/metrics` 3 series render. bug-2205: aggregate MUST come from
  Σ `n_decoded_cum` (per-token), not `n_tokens_predicted_total`
  (completion-time only).
- **P2 — pool objects + status API** (§3, §4.1-4.2). The slot/seq decoupling
  (§3.5): seq-id reserve, prefill-only materialization task, host-side Pool
  state, **auto-P attach**; `GET /polykv/pools`. Root pools only.
  **✅ IMPLEMENTED 2026-07-16** (live in vendored tree, patch capture pending):
  gate 23/23 on Qwen3-4B — boot check (`--polykv-max-pools` without
  `--kv-unified` fails clean; NOTE `--parallel` must be explicit, unset
  auto-sets `kv_unified=true`), create-from-prompt prefill-only P=95,
  auto-P exact attach `cache_n=95`, divergence attach `cache_n=42<95`,
  `from_session` zero-copy `seq_cp` snapshot (affinity is evict-on-reuse —
  only the most-recent session per slot resolves), pin/unpin/GET-one,
  release + released-id clean fallback, disabled path inert, pooled vs
  unpooled greedy text IDENTICAL (bit-shared cells). Mutations only on the
  server thread via `SERVER_TASK_TYPE_POLYKV`; recurrent guard per bug-2203.
- **P3 — nesting & COW-fork** (§7). `fork` with arbitrary `D` (extend + COW
  branch), cumulative-hash contract check, recursive pin, seq-id accounting.
  **✅ IMPLEMENTED 2026-07-16** (live in vendored tree, patch capture pending):
  gate 18/18 substantive on Qwen3-4B — extend fork (D=parent.L, own=22),
  COW branch (D=47<95, zero-copy `seq_cp` of `[0,D)` + fresh prefill),
  contract violation → clean 4xx, auto-P attach to a child (cache_n=117),
  tree reporting (parent/children/branch_pos/own_len/tree_depth), release
  ordering enforced (parent-with-children refused, leaves-first drains),
  seq exhaustion at max_pools, `from_session` fork. The §4.1 prefix
  contract is enforced **token-exact** (`std::equal` against the parent's
  stored tokens — strictly subsumes the cumulative-hash check; `prefix_hash`
  kept as identity fingerprint). Pooled-vs-unpooled greedy divergence on the
  branch was classified benign by first-token probe (same top-5 ranking,
  max |Δlogprob| 0.112 — batch-shape numerics, #495 class). Recurrent
  bug-2203 guard is recursive: hybrid/recurrent forks must extend at
  D == parent.L.
- **P4 — admission** (§6). `/capacity`, advisory metrics, then enforced mode
  (`reject`/`warn`/`queue`).
  **✅ IMPLEMENTED 2026-07-16** (live in vendored tree, patch capture pending;
  `queue` deferred per §11 default): gate 18/18 on Qwen3-4B — /capacity shape
  (idle: can_admit, projected null, kv cells by OWNERSHIP: pools' own ranges +
  slots' marginal suffixes via new `slot.n_pool_shared`), advisory floor never
  rejects, **enforced+reject → 429 + Retry-After** under an active decoding
  session (tps-floor arm needs n_active ≥ 1), **enforced+warn → 200 +
  X-Sessions-Remaining**, off-path (no pool_id / unknown pool_id) untouched.
  Projection = conservative fully-saturated knee: `mean_active · n/(n+1)`
  (never over-admits below the real B_sat knee). Fast path: an atomic
  enforced-pool count — ordinary traffic pays zero; the gate itself is a
  high-priority OP_CAPACITY control task from `handle_completions_impl`
  (same pattern as /slots' METRICS task).
- **P5 — LangGraph package** (§8). `opencoti-langgraph` under top-level
  `opencoti/`. Runs in the first-class **`opencoti` conda env** (py3.11,
  langchain-core 1.4.x + langgraph 1.2.x pinned; the same env will later
  host the consultants council alongside opencoti-server). Includes the P6d
  `CompactionNode` (§13).
  **P5 ✅ IMPLEMENTED 2026-07-16**: offline 21/21 (pytest, httpx MockTransport
  FakeServer), live gate **19/19** on Qwen3-4B `:8231 --parallel 2 --kv-unified
  --polykv-max-pools 4` — pool tree (token-path fork D=65), `OpencotiChatModel`
  invoke + streaming with pool attach proven (`cache_n=68 ≥ root prefix 65`),
  `session_tps`/usage metadata, capacity context arm + gate-node routing,
  enforced 429 → `PoolSaturatedError(retry_after)` → advisory recovery,
  **P6c compact-by-refork e2e** (old pool released, migrated session coherent
  on the new pool, `cache_n=71`), tps `ctx_*` fields. Contract lessons
  (bug-2208): fork `prompt`/`tokens` = the child's **FULL prefix [0,L)**
  token-exact vs the parent (never suffix-only — build via `/tokenize`:
  parent tokens + suffix `add_special=false`); a pool prefix meant for
  `/v1/chat/completions` attach must be the **chat-templated** system block
  (raw text shares 0 tokens → `cache_n=0`); Qwen3 thinking models need
  `chat_template_kwargs: {enable_thinking: false}` for content-based gates.
  Client accordingly ships `tokenize()` + `compact_by_refork(compacted_tokens=…)`.
- **P6 — context exhaustion & compaction** (§13). **P6a (ctx-shift pool
  guard) + P6b (context visibility + /capacity context arm) land BEFORE P5**
  per user decision 2026-07-16; P6c/P6d ship with P5.
  **P6a+P6b ✅ IMPLEMENTED 2026-07-16** (live in vendored tree, patch capture
  pending): gate 23/23 on Qwen3-4B at `-c 4096 --parallel 1 --context-shift`
  (the exact §13.1-regime-3 configuration) — boot WARN; pooled session forced
  past n_ctx = graceful `truncated=true` stop at 4021 tokens (position wall
  fires one token before unified-cell exhaustion when the prefix is
  pool-shared), NO shift, pool provably coherent afterwards (fresh attach
  cache_n=60, needle answered); non-pooled ctx-shift still engages and
  completes after pools released (off-path intact); `?expected_tokens` context
  arm rejects with `"context headroom exhausted"`; `compaction_pressure`
  0 at idle, ∈[0,1]; `orphaned_pin` true for a pinned never-attached leaf;
  `/polykv/tps` sessions carry `ctx_used/ctx_total/ctx_headroom_tokens`.
  Guard bookkeeping: `slot.n_pool_shared` is clamped to the retained prefix at
  prompt-cache decision time (`min(n_pool_shared, n_past)`) — NOT blind-reset,
  because cache reuse keeps pool-shared cells across later non-pool requests
  on the same session. NOTE the geometry lesson: with `--kv-unified` the slot
  n_ctx is the FULL `-c` (not divided by `--parallel`), and multi-slot cell
  exhaustion precedes the positional wall — gates must use `--parallel 1`.

## 11. Open sub-decisions (defaulted; flag to change)

- EWMA window `W`/`T` defaults (proposed: 128 tokens or 2 s).
- Grace TTL for interior-pool reclaim (proposed: reuse `--slot-shrink-idle-ms`).
- `/metrics` opt-in flag name + default-off.
- Whether `queue` (admission) is v1 or P4-follow (proposed: `reject`+`warn` v1,
  `queue` follow).
- `--polykv-max-pools` default (proposed: 16; each reserved seq-id costs KV
  bookkeeping but no cells until materialized).
- Hash algorithm for the cumulative `prefix_hash` (proposed: xxhash64 rolling;
  8 B/token host RAM — ~800 KB for a 100k-token prefix, negligible).
- SSE tps-stream cadence (proposed: 500 ms, configurable; stream is opt-in so
  off-path cost is zero).

## 12. Verification

- Off-path identity: PolyKV-control-unused boot is byte-identical (DSO + decode).
- tps accuracy: instrumented per-session tps vs. an external wall-clock
  token-rate measurement, ±few %.
- Fork correctness: a branched pool's decode is logit-equivalent to the same
  session full-prefilled (real_frac=0 / KLD), never greedy-needle.
- Tree lifetime: ancestor cells survive descendant-only references; reclaim on
  refcount 0; recursive pin prevents the residency-miss fallback.
- Admission: projected vs. realized `mean_active_tps` under a spawn ramp;
  `reject`/`warn` behavior + `Retry-After`/`X-Sessions-Remaining` headers.
- Composition gates: rolling-KV / DCA / turbo / MTP / `--parallel 2` unchanged.

## 13. Context exhaustion & compaction orchestration (P6)

*Added 2026-07-16 on user review: the design above admits sessions by tps floor
and marginal KV, but had no story for what happens when the shared context
actually fills — and continuous agentic frameworks WILL fill it. Decision:
P6a+P6b land BEFORE P5 (the LangGraph capacity gate builds on the context arm);
P6c/P6d ship with/after P5.*

### 13.1 What the engine does today (verified, three regimes)

1. **Prompt too long at admission** → clean `ERROR_TYPE_EXCEED_CONTEXT_SIZE`.
2. **Generation reaches `n_ctx`, `--ctx-shift` off (default)** → clean stop
   with `truncated=true` (`server-context.cpp:1898`). Not corrupting, but
   silent mid-thought truncation is exactly the "fails badly" mode for agents:
   no advance warning, no orchestration signal.
3. **`--ctx-shift` on + pool-attached session → CORRUPTION.** The shift does
   `seq_rm` + `seq_add(…, -n_discard)` (`server-context.cpp:3261-3262`); in
   the unified cache `seq_add` mutates the **per-cell position**, and
   pool-shared cells are members of many seqs — the shift re-RoPEs the pool's
   prefix for every other session and every descendant pool. Upstream guards
   this for parent/child shared prompts (`server-context.cpp:3192`) but a
   pool attach is neither, so the guard does not fire. The rest-kv-eviction
   variant uses the same primitives and has the same hazard.

The "safe" naive alternative — COW-copy shared cells before shifting —
allocates a duplicate of the prefix at the exact moment the cache is full.
Both intuited failure modes (corruption / usage inflation) are real.

### 13.2 Design principle: compaction is a prompt REWRITE, never an in-place KV op

LangGraph (and every agentic framework) compacts at the prompt-assembly
level: `trim_messages`, `RemoveMessage` reducers, or a summarization
`pre_model_hook` that replaces old messages with a running summary. The
compacted context is **new text** — its tokens cannot match the old cells, so
cell-level "compaction" is not even meaningful. Therefore the server must
never mutate pooled cells; the correct primitive already exists in the tree:

> **Compaction = re-root via fork.** The system prompt + tool definitions
> survive compaction verbatim — that is the stable ancestor pool
> `[0, D_sys)`. Compacted context = `fork(ancestor, D=D_sys,
> tokens = summary + recent tail)` — a new sibling branch, prefilled once.
> Sessions migrate to the new pool; the old working subtree is `release`d.
> Pool immutability is preserved by construction, no cell position is ever
> touched, and net cells go **down** (old subtree reclaimed) instead of up
> (COW duplication).

Pinned pools under this model are a **leak hazard, not a corruption hazard**:
a pin left on the abandoned subtree blocks reclaim. Discipline is
unpin-after-migrate; the server reports orphans (pinned + zero sessions +
zero children) so the orchestrator can sweep.

### 13.3 Phases

- **P6a — corruption guard (before P5).** Extend the ctx-shift refusal to
  pool-attached slots (`slot.n_pool_shared > 0` ⇒ clean error, same as the
  `:3192` shared-prompt refusal) + boot WARN when `--ctx-shift` is combined
  with `--polykv-max-pools`. Rest-kv-eviction path included. Off-path inert.
- **P6b — visibility + spawn-gate parity (before P5).** Per-session
  `ctx_used`/`ctx_headroom_tokens` in `/polykv/tps` + `/slots`; `/capacity`
  gains `?expected_tokens=N` and a **context arm** to `can_admit`
  (reason `"context headroom exhausted"`) so sub-agent spawns are gated on
  context exactly like the tps floor; plus a `compaction_pressure` field
  (0..1, driven by free-cell fraction and largest-session share) so the
  orchestrator compacts BEFORE the wall.
- **P6c — compact-by-refork protocol (with P5).** Documented two-step:
  summarize via normal completion → `fork` the stable ancestor at `D_sys`
  with the compacted suffix → migrate sessions → `release` old subtree.
  Orphaned-pin flag in `/polykv/pools`. Atomic convenience endpoint only if
  the two-step proves racy in practice.
- **P6d — LangGraph side (inside opencoti-langgraph).** `CompactionNode`
  (pre_model_hook pattern: watch `response_metadata` headroom /
  `compaction_pressure` → summarize → re-root → swap `pool_id` in graph
  state); the capacity-gate node checks the context arm from day 1.

### 13.4 Non-goals

- In-place KV compaction/merging of pooled cells (meaningless under rewrite
  semantics; corrupting under sharing).
- Server-side self-summarization (the orchestrator owns the summary — it has
  the conversation semantics; the server only has tokens).
- Growing `n_ctx` at runtime (allocation is boot-time; capacity planning is
  the admission plane's job).

---

## 14. P7 — measurement-settled admission, measured-drop forecast, guaranteed minimum

> **Status: design locked (2026-07-23, user).** Driven by the
> `courier-qwopus9b-mtp-s55-p10` run: floor 15 tps, spawns every 5 s on a
> tps EWMA (tau 2 s) that had not yet absorbed the previous admit
> (38→31.7→21.7→15.9 at successive ticks), overshoot to n=6 at 11.6 tps,
> and the corrective shrink actuated **22 minutes later** (workers retire
> only at episode end). Decisions: pacing lives in BOTH layers (server-side
> hold + orchestrator pacing); overflow valve = per-pool
> `guarantee_min_sessions` (default 1) + per-request `overcommit` honored.

### 14.1 Root causes (from the run + code audit)

1. **Admit-on-stale-measurement.** Nothing marks a pool "still absorbing the
   last admit"; every 5 s tick admitted again before the EWMA settled.
2. **Warming bias.** `mean_active_tps` averages `tps_ewma` over all
   processing slots, including just-admitted ones still at `tps_ewma == 0`
   — right after a spawn the mean is dragged toward 0, corrupting both the
   floor comparison and the projection.
3. **Optimistic projection.** `n/(n+1)` (flat-aggregate knee model) under-
   predicted the real drop at every step of the run (e.g. predicted 17.4
   post-admit, reality 15.9). No measured feedback.
4. **Courier bypassed the projection.** `scheduler()` compares raw
   `mean_active_tps` to the floor instead of
   `projected_mean_tps_if_admitted` (bug — fixed in P7's courier pass).
5. **Enforced gate cannot tell admission from continuation.** The gate at
   request routing fires on ANY request carrying `pool_id`, so a pool under
   floor 429s its own running sessions' next turns — livelock risk. The gate
   must gate only NEW sessions (no `session_to_slot` affinity yet).
6. **Shrink actuation lag is structural** (episode-end retire). P7 does not
   change it; it makes overshoot rare instead (prevention, not cure).

### 14.2 Server design

**Settle window (per pool).** On each NEW-session admit record
`{t_admit_us, admit_slot, n_decoded_at_admit, mean_at_admit}`. The pool is
**settling** until the admitted slot has decoded `settle_tokens` (default 48)
more tokens OR `settle_max_ms` (default 5000) elapsed. While settling:
- `/capacity` → `settling: true`, `settle_remaining_ms`, `can_admit: false`,
  `reason: "measurement settling"` (advisory clients wait, not shrink).
- Enforced mode → the gate HOLDS the attach (bounded sleep-poll ≤
  `settle_max_ms`, then re-evaluates) instead of 429 — "slow the spawn just
  enough", never a hard reject for settling alone.

**Warming-aware mean.** Slots with `tps_ewma == 0` while processing are
**warming**: excluded from `mean_active_tps`/projection; reported as
`n_warming` in `/capacity` and `/polykv/tps`.

**Measured-drop forecast.** When a settle completes, fold
`drop = max(0, mean_at_admit − mean_settled_now)` into a per-pool EWMA
(`drop_ewma`, alpha 0.3). Projection:
`projected_measured = mean_settled − drop_ewma` (once ≥1 sample),
`projected_model = mean·n/(n+1)` (fallback / always reported). The floor arm
uses `projected_measured` when available, else the model.
`/capacity` reports both plus `drop_per_admit_ewma`.

**Idle-gap estimate.** When every session is between turns (tool calls in
flight) `n_active` is 0 and "no projection" reads as free capacity — the
hole that still ramped the first P7 gate run to n=8. If the pool has live
sessions and a FRESH last settled mean (< 30 s), `/capacity` projects from
it instead (`projected_idle_estimate: true`; measured-drop variant when
available, else `last_settled_mean·n_pool/(n_pool+1)`), and the floor arm
applies (`reason: "projected mean tps below floor (idle estimate)"`). A
long-idle pool (stale sample) genuinely has capacity and stays
unprojected.

**Guaranteed minimum (the anti-deadlock valve).** Admission policy gains
`guarantee_min_sessions` (default **1**, settable via
`POST /polykv/pools/{id}/admission`). In `/capacity`, when the pool's
currently-attached session count `< guarantee_min_sessions`, `can_admit` is
**true** with `reason:"guaranteed minimum"`, `guaranteed:true` — bypassing
the tps-floor and kv-headroom arms. The **context arm stays hard**
(`expected_tokens > free_cells` still refuses — cells are physical). A brand
-new or freshly forked (nested) pool therefore ALWAYS gets its first agent,
even oversubscribed past the target; deadlock is impossible by construction.
Once at/over the minimum, replacements follow the normal floor/target checks
— an oversubscribed pool converges back down as episodes end.

**Per-request `overcommit`.** `"overcommit": true` in the completion body
(threaded like `pool_id`) skips the enforced gate for that request —
explicit caller-controlled oversubscription, logged.

**Continuation bypass.** The enforced gate resolves the request's
`session_id` against `session_to_slot` (via the capacity control task, which
runs on the server thread); a known session is a continuation → never gated.

### 14.3 Courier/orchestrator pass (both-layer pacing)

- `scheduler()` gates spawns on `projected_mean_tps_if_admitted` (not raw
  mean) AND `settling == false`; a settling tick is a no-op (never a shrink
  signal). `n_warming > 0` also defers judgment.
- After `spawn_worker`, the scheduler does not evaluate again until the
  server reports the pool settled (one poll of `/capacity` suffices — the
  server owns the settle clock).
- Report gains settle/guarantee/overcommit event rows + a settling band on
  the tps chart.

### 14.4 Compatibility & phases

All fields additive; pools without a policy behave exactly as before except
the warming exclusion (a pure measurement fix) and the continuation bypass
(strictly less rejection). Off-path (no PolyKV) byte-identical.

- **P7a** — warming-aware mean + settle state + measured-drop forecast +
  `/capacity` fields (server). **DONE 2026-07-23.**
- **P7b** — `guarantee_min_sessions` + `overcommit` + continuation bypass +
  settling hold in the enforced gate (server). **DONE 2026-07-23.**
- **P7c** — courier scheduler/report pass + rerun the courier gate.
  **PASS 2026-07-23** (solidPC 3090, Qwopus3.5-9B MTP, 10 pkg × 55 steps,
  floor 15): vs the baseline run, wall 2187→1919 s (−12%), time-under-floor
  1648→790 s (−52%), deep sub-floor (<7.5 tps) 706→122 s (−83%), delivery
  p50 1350→764 s (−43%), same 100% score. Two gate iterations: the first
  (settle+guarantee+projection only) got −23% under-floor but still ramped
  to n=8 through idle-gap "no projection = free capacity" spawns; the
  idle-estimate arm closed that hole.